Q1¶

1. The accuracy of best model in test dataset: 0.985310

2. Sucees case

predict	Chair	Vase	Lamp
data

3. Failure case

GT class	Chair	Vase	Lamp
Predicted	Lamp	Lamp	Vase
data

• The structure and shape of lamp and vase are similar. This mislead the classfication network. The network may classify lamp as vase, or vice versa.
• The chair is easier for classification. However, there are still some challenging cases, i.e., the folded chair shown above.

Q2¶

1. The accuracy of best model in test dataset: 0.897432

2. Examples

examples	case1	bad case 1	bad case 2
gt
predicted
accuracy	0.907	0.663	0.899

• In general the result is good. There are some parts are challenging for semnatic segmentation. For example, the cushion on the chair may be confused with leg or the back of the chair. Also there are some object is not very common, these cases will lead to a failure of the semantic segmentation.

  ## Q3.1 First I will analyze the number of sampling points and the results.

1. For classification

points	10000	5000	1000	100	50
gt
accuracy	0.983	0.980	0.978	0.895	0.686

1. For segmentation

points	10000	5000	1000	100	50
gt
accuracy	0.907	0.898	0.894	0.821	0.778

Q3.2¶

Second is the rotation of the points and the results, In this step I will calculate the orientation around the x axis, from 0 to 180 degree. To achieve this, I apply a rotation matrix to the point cloud

1. classification

angle	0	20	40	60	90	180
gt
accuracy	0.983	0.912	0.602	0.339	0.261	0.698

1. segmentation

   angle	0	20	40	60	90	180
   gt
   accuracy	0.897	0.806	0.665	0.460	0.286	0.334

   • For both semantic segmentation and classification, the network can tolerate a small rotation. However, the network is not robust with a large rotation.
   • The result in segmentation with 180 degree rotation is interesting. It shows that the network maybe overfit to some condition like the height of the points. To be more specific, the upper part may be the armrest, the lower part may be the leg.