Predicted: Chair, Truth: Chair
*FAILURE CASE* - Predicted: Lamp, Truth: Chair
Zero late days used
Accuracy of Best Model: 98.01%
My model performed well with a best accuracy of 98.01% on the test set. By looking at the number of failure cases, my model does not perform well on classifying lamps. Maybe due to the fact that most have similar underlying shapes to that of a case. Above are some positive and failure cases for each class.
Accuracy of Best Model: 89.88%
My model performed well with an accuracy of 89.88% on the test set. There are some cases where the model finds it difficult to segment complex overlapping structures. However, for general cases, my model performs well. Above are some good and bad examples of segmentation
Robust Analysis 1 - Point Cloud Rotation
For this task, I rotated the input point clouds by a certain number of degrees on the z-axis. To do this I multiplied the rotation matrix by the point clouds. For both segmentation and classificatioon parts, it seems like my model is not robust to rotations, especially at 90 degrees.
Rotation Matrix
| cos(theta) | -sin(theta) | 0 |
|---|---|---|
| sin(theta) | cos(theta) | 0 |
| 0 | 0 | 1 |
| Degrees | Segmentation | Classification |
|---|---|---|
| 0 | 89.88% | 98.01% |
| 37 | 68.81% | 74.08% |
| 45 | 65.54% | 65.27% |
| 90 | 45.11% | 29.38% |
Robust Analysis 2 - Different Number of Points
For segmentation, as I increase the number of points, the accuraacy decreases. For classification, decreasing the number of points decreasees the accuracy slightly but not drastically.
| # of Points | Segmentation | Classification |
|---|---|---|
| 1,000 | 90.17% | 97.78% |
| 2,000 | 90.14% | 97.59% |
| 4,000 | 90.00% | 97.80% |
| 8,000 | 89.90% | 97.80% |
| 10,000 | 89.88% | 98.01% |
I implemented edge convolution from the Dynamic Graph CNN paper. Edge convolution captures the local geometry information of the point clouds by generating edge features that describes the relationship between a point and its nearest neighbors. This is done by applying a channel wise aggregation (max or summation) on the edge features. I used max for my implementation. The edge features are learned through an MLP. For my implementation, I used the same network architecture from question 1 and 2, however I used edge convolution layers before each shared MLP layer. I also used k=20 nearest neighbors. The accuracy for classification and segmentation is below.
Accuracy of Best Model: 97.69%
Accuracy of Best Model: 90.75%
In terms of accuracy, for the classification task, the accuracy is slightly less when addiing locality. I do think it's hard to get a better accuracy that 98% percent. For the segmentation class, adding locality performs better. I believe that adding locality would improve the segmentation task due to the nature of edge convolutions capturing local geometric structure. I also limit the number of points tto 1024 for memory issues.