Point Cloud Classification and Segmentation
1. Classification Model
The test accuracy of my best model is shown below.| Accuracy | 94.22% |
|---|
I also calculate the confusion matrix for the predicted results as shown below:
| Chair | Vase | Lamp | |
|---|---|---|---|
| Chiar | 614 | 2 | 1 |
| Vase | 4 | 69 | 29 |
| Lamp | 1 | 18 | 215 |
From the confusion matrix and the qualitative results, we can see that the model mostly gets confused in predicting the vases and lamps, which makes scenes as sometimes the shape of certrain instances from these two classes can be very similar.
2. Segmentation Model
The test accuracy of my best model is shown below.
| Accuracy | 90.27% |
|---|
From the qualitative and quantitative results, we can see the model works well in general. But for some cases like the failure case 1, the model will produce non-continuous segmentation results, which is reasonable as we don't introduce locality in our model. And for the failure case 2 and 3, the model can't make a correct segmentation as these two kinds of chair are uncommon in the dataset.
3. Robustness Analysis
Robustness to Rotation
We first can create the rotation matrix by following formula, where \(\alpha, \beta, \gamma\) represents the rotation angle in z, y, x axis.
Then we can multipy the rotation matrix to the orginal points to get rotated point clouds and evelaute our model on them
The results for classification model are as follow
| Degree | -30 | -20 | -10 | 10 | 20 | 30 |
|---|---|---|---|---|---|---|
| Accuracy (X-axis) | 43.65% | 64.11% | 88.56% | 90.45% | 80.58% | 80.79% |
| Accuracy (Y-axis) | 29.59% | 45.54% | 90.44% | 93.80% | 92.05% | 81.01% |
| Accuracy (Z-axis) | 43.54% | 69.25% | 84.36% | 86.78% | 67.26% | 41.87% |
The results for segmentation model are as follow
| Degree | -30 | -20 | -10 | 10 | 20 | 30 |
|---|---|---|---|---|---|---|
| Accuracy (X-axis) | 70.85% | 78.67% | 86.70% | 88.47% | 85.13% | 80.12% |
| Accuracy (Y-axis) | 75.31% | 82.13% | 87.87% | 88.53% | 84.82% | 80.63% |
| Accuracy (Z-axis) | 57.71% | 69.77% | 85.01% | 85.97% | 76.72% | 69.22% |
From the results above, we can see that the performances of two models drop a lot with a large rotation (like 30 degree), and the segmentation model is more robust to the rotation comparatively.
Robustness to Number of Points
The results for classification model are as follow
| Number of Points | 2000 | 4000 | 6000 | 8000 | 10000 |
|---|---|---|---|---|---|
| Accuracy | 93.28% | 93.70% | 94.01% | 94.33% | 94.22% |
The results for segmentation model are as follow
| Number of Points | 2000 | 4000 | 6000 | 8000 | 10000 |
|---|---|---|---|---|---|
| Accuracy | 89.98% | 90.16% | 90.24% | 90.25% | 90.26% |
From the results above, we can see that the performances of two models are robust to the inpit number of points.