The network correctly predicts this is a lamp.
The accuracy on the test set is 0.9758656873032528
Random Test Point Clouds:
The network correctly predicts this is a lamp.
The network correctly predicts this is a vase.
The network correctly predicts this is a chair.
Pred Errors:
Prediction is lamp, but it is actually a chair. I think this is because the chair is folded.
Prediction is vase, but it is actually a lamp. This vase indeed looks like a chair.
Prediction is lamp, but it is actually a vase. This lamp indeed looks like a vase.
The accuracy on the test set is 0.7514677471636954
Some examples, where the left is groundtruth and the right is the prediction:
The first one has a bad accuracy which is only 0.4388. The model is fooled to think part of the body is the foot of the chair, and part of the armrest to be the cusion. The model is not predicting any armrest, which is funny. I think the model learns mostly this: the lower part is the foot, the middle part is the cusion, and the upper part is the back.
The second one also has a bad accuracy of only 0.2665, and also confirms my guess of what the model learns: bottom is foot, middle is cusion, top is back. It totally ignores armrest and footrest.
The third one has an OK accuracy: 0.7493. Again, the model does not learn to recoginize armrest at all.
The 4th one has a high accuracy: 0.8859. I think the high accuracy is due to that the chair only does not have the fancy parts such as armrest or footrest.
The 5th one has a high accuracy: 0.8795. Again, I think the high accuracy is due to that the chair only does not have the fancy parts such as armrest or footrest.
Experiment 1: rotate the input point cloud along the z-axis for 30 degrees.
The classficaction accuracy is now: 0.8069254984260231. Without rotation, the accuracy is 0.9758656873032528, so it drops a lot, which means the model is not robust to the rotation.
The segmentation accuracy is now: 0.7125705024311183. Without rotation, the accuracy is 0.7514677471636954, so it drops too, though the drop is less compared to the classification case. Still, it means the model is not robust to the rotation.
Experiment 2: use fewer points in the input. Instead of 10000 points, I now use 1000 points.
The classficaction accuracy is now: 0.9716684155299056. Without rotation, the accuracy is 0.9758656873032528. The model is suprisingly robust to decrease in the number of points.
The segmentation accuracy is now: 0.7516726094003241. Without rotation, the accuracy is 0.7514677471636954. The model is suprisingly robust to decrease in the number of points.