Submitted by Ambareesh Revanur (arevanur)
Number of late days used is 0.
Test Accuracy of best model: 97.3%
Visualize a few random test point clouds and mention the predicted classes for each.
Predicted Chair for the below point cloud. The point cloud clearly indicates legs/seat/armrest that only appears in a chair. Hence the model is able to easily classify it into a chair.
Predicted Vase for the below point cloud. The below point cloud has a leaf and flower usually associated with a vase.
Predicted Lamp for the below point cloud. The below point cloud has distinctive lamp holder and straight rods with lamp base. These are not present in other classes such as chair and vase.
Also visualize at least 1 failure prediction for each class (chair, vase and lamp), and provide interpretation in a few sentences.
Interpretation: The model predicted that below object is a vase although it is a lamp. This lamp shape is relatively rare in the training dataset and looks like some of the vase that is in shape of a inverted bowl.
Interpretation: The model predicted that below object is a lamp although it is a chair. This chair shape is relatively rare in the training dataset and looks like some of the lamps that have a tall and slender structure.
Interpretation: The model predicted that below object is a lamp although it is a vase. This vase shape is relatively rare in the training dataset and looks like some of the lamps that have a tall and slender structure.
Q: Test Accuracy of best model: 88.68%
Q: Visualize segmentation results of at least 5 objects (including 2 bad predictions) with corresponding ground truth, report the prediction accuracy for each object, and provide interpretation in a few sentences.
In the below set of images left is Ground truth and the right is prediction.
The model is able to achieve high accuracy on this point cloud since there are many point clouds that are similar to this in the training set.
(Good prediction) Prediction accuracy for the below prediction is 99.2%. The training data contains many chairs with 4 legs that are distinctly visible.
(Good prediction) Prediction accuracy for the below prediction is 99.2%. This chair has a structure that is easy to predict since there is an obvious demarcation between seat and legs.
(Bad prediction) Prediction accuracy for the below prediction is 45.3%. The model has hard time trying to demarcate between legs and arms since they continue till the bottom.
(Bad prediction) Prediction accuracy for the below prediction is 49.1%. This structure is difficult since it not obvious even for humans if the structure resembles legs. Further, there is a pillow inside the chair that is part of the sear. This is not immediately obvious since the training dataset does not have enough data resembling this.
(Bad prediction) Prediction accuracy for the below prediction is 50.9%. It is not immediately obvious if the chair has legs or if the seat is large extending till the bottom.
I used the look_at_view_transform function to play with elevation parameter. This generates a rotation matrix which I use to multiply with the test point cloud.
Classification
Segmentation
There is a consistent drop in accuracy with higher angle of deviation. This is not surprising since the model is trained on the 3d models that are not rotated. One way to retain higher accuracy levels is by data augmentation.
Next, I used the --num_points parameter to tune the model as hinted in the question.
Classification
Segmentation
The results show that PointNet is able to remarkly retain improvement to certain degree inspite of changing input. In all the experiments, the accuracy drops with higher modification magnitude. In comparison to Q1 and Q2, it is not surprising that the results are worse.
Implemented PointNet++ architecture in models.py. I used Kmeans clustering to cluster the points. Owing to its iterative nature of the clustering algorithm, the training step was slower and I was able to train only a few epochs on the GPUs. Therefore, we cannot compare these results with the ones we obtained in Q1 and Q2 (trained for 240 epochs).
Prelimnary results: Classification accuracy after 2 epochs of training is 84.0%