**Assignment 5**
Student name: Gaurav Parmar
(#) Q1. Classification Model
The test accuracy of the best model is: 96.54%
A few random test points are shown below. Their (correctly) predicted classes are chairs, vase, lamps (left to right).
Some samples with incorrectly predicted labels are shown next.
The actual labels are vase,vase,lamp
The predicted labels are lamp,lamp,vase
I observed that the errors in predictions typically happen between the two classes especially on images that
genuinely look like they could belong to either of the classes.
(#) Q2. Segmentation Model
The test accuracy of the best model is: 89.51%
Some exaples of good predictions are shown below. The ground truth is on the left,
the prediction is on the center, and accuracy is on the right.
94.55%
98.79%
97.03%
98.72%
94.12%
Some harder predictions are shown next. The ground truth is on the left,
the prediction is on the center, and accuracy is on the right.
58.97%
51.00%
67.54%
72.32%
72.37%
Overall, I notice that chairs that parts that blend into each other with no obvious boundaries
are understandably segmented the worst.
(#) Q3. Robustness Analysis
(##) Robustness to rotations - classification
For this test I vary the azimuth from (0, 360) and rotate the input points accordingly.
The rotations and the classification accuracy is shown below.
The values show that the classification model is not robust to the rotations and
the accuracy decreases sharply.
Rotation Amount
Classification Accuracy
360
96.85%
300
72.40%
240
27.07%
180
32.21%
120
28.96%
60
79.64%
(##) Robustness to rotations - segmentation
Similarly, next I show the performance on the segmentation model below.
The values show that the segmentaion model is not robust to the rotations and
the accuracy decreases sharply.
Rotation Amount
Segmentation Accuracy
360
89.51%
300
69.23%
240
40.94%
180
43.45%
120
33.32%
60
69.72%
(##) Robustness to number of points
I decrease the number of points used and show how the performance of
the classification and segmentation model changes accordingly.
Number of points
Classification Accuracy
Segmentation Accuracy
100
91.71%
80.01%
1000
96.12%
87.80%
5000
96.85%
89.43%
10000
96.85%
89.51%
(#) late days
I used 0 late days on this assigment.
