Course: 16-889 Learning for 3D Vision

Name: Soyong Shin

Due by: Apr. 14 (Fri)

I used 3 late days for this assignment

Contents:

Q1. Classification Model (40 points)

Following predefined dimensions of hidden layers (Slide L17, page 30), I implemented on models.py with the class of cls_model

Training Log

Best Test Accuracy

Within 100 epoch training, the best accuracy was reported as 96.85 %

Visualization (Successful cases):

Visualization (Failure cases):

Q2. Segmentation Model (40 points)

Following predefined dimensions of hidden layers (Slide L17, page 32), I implemented on models.py with the class of seg_model

Training Log

Best Test Accuracy

Within 100 epoch training, the best accuracy was reported as 89.58 %

Visualization (Successful cases):

Here I visualize the top 5 accurate samples. The accuracy was obtained as the percentage of correct points.

Top 1. Accuracy: 99.58 %

Top 2. Accuracy: 99.52 %

Top 3. Accuracy: 99.26 %

Top 4. Accuracy: 99.12 %

Top 5. Accuracy: 98.98 %

Visualization (Failure cases):

Here I visualize the top 5 worst accurate samples.

Worst 1. Accuracy: 40.62 %

Worst 2. Accuracy: 49.37 %

Worst 3. Accuracy: 51.35 %

Worst 4. Accuracy: 53.71 %

Worst 5. Accuracy: 54.16 %

Q3. Robustness Analysis (20 points)

A. Rotation transformation

Here, I applied random rotation to the points and check the robustness of the network on the rotational transformation. The implementation is on analysis.py

I tested the models on 10 different rotational magnitudes, 0 - 45 with an interval of 5 degrees. In every iteration, a random rotation vector angle with each magnitude is sampled and transformed to the rotational matrix rMat. And the points are rotated by the matrix.

a. Classification

b. Segmentation

Conclusion

Both networks clearly show performance decreases as the magnitude of rotation increases. The segmentation model, on the other hand, shows better robustness to the arbitrary point cloud rotation.

B. Number of points

Here I changed the size of the point cloud and check the robustness of the models on the size. The implementation is on analysis.py

Here I used 1000 to 10000 points with an interval of 1,000.

a. Classification

b. Segmentation

Conclusion

Both networks show great robustness to the number of input points. It means 1,000 points are still a sufficient number for both tasks. Therefore, I further conducted the experiments on much smaller points.

According to the figures above, the classifier shows robust performance over 500 points, while the segmentation model works reasonably well over 1,000 points.