1.1. Fitting a voxel grid
1.2 Fitting a point cloud
1.2 Fitting a mesh
2.1. Image to voxel grid
Prediction and Ground truth visualized meshes of 3 random objects
 |
 |
 |
 |
 |
 |
2.2. Image to point cloud
Prediction and Ground truth visualized meshes of 3 random objects
 |
 |
 |
 |
 |
 |
2.3. Image to mesh
Prediction and Ground truth visualized meshes of 3 random objects
 |
 |
 |
 |
 |
 |
2.4. Quantitative comparisions
F1 score of voxel prediction: Avg F1 at 0.05: 60.303
F1 score of point prediction: Avg F1 at 0.05: 85.777
F1 score of mesh prediction: Avg F1 at 0.05: 73.121
The quality is lowest in voxel prediction because of low resolution of voxels used in training which will fit the actual mesh poorly. In case of mesh and point decoder point decoder doesn't need to learn the vertex relationship between points and thus the task is easier comparatively. This shows in the F1 scores as well.
2.5 Analyse effects of hyperparms variations
experiment with increasing n_points in point prediction task
With the increase in the points the density increases and thus F1 score relatively looks better. Based on the visualization shown below the predictions are not significantly better.
At 3000 points F1 score is 83.579
At 5000 points F1 score is 85.777
At 8000 points F1 score is 86.429.
At 10000 points F1 score is 87.242
|
 |
 |
|
| 5k points | 10k points | 3k points | GT |
|
 |
 |
|
| 5k points | 10k points | 3k points | GT |
experiment with more levels in mesh(ico_sphere) size in mesh prediction task
At level 3 F1 score is 78.667
At level 4 F1 score is 73.121
At level 5 F1 score is 75.177
With the decrease in number of faces and vertices in the initial mesh the surface is more smoother and thus F1 score is higher. This is also visible in the below visualization. But similar effect can be reached by increasing the weight of smoothness loss as shown in the next ablation.
|
 |
|
| 4 level Icosphere | 3 level Icosphere | GT |
experiment with --w_smooth in smoothless loss
At 0.2, F1 score is 76.929
At 0.1, F1 score is 73.121
At 0.3, F1 score is 71.460
With the increase in F1 score surface becomes more smooth and helps in overall quality of predicted mesh. But a large increase will hurt the actual chamfer loss optimization and thus F1 score decreases overall.
|
 |
 |
|
| 0.1 w_smooth | 0.2 w_smooth | 0.3 w_smooth | GT |
2.6 Interpret your model
Visualization of samples with lowest F1 score for each method can give us insights into where the model is making the most mistakes. Below I visualize such samples for all three methods.
Prediction and ground truth of lowest F1 score samples in voxel prediction out of 100 validation samples
 |
 |
 |
 |
 |
 |
Prediction and ground truth of lowest F1 score samples in point prediction out of 100 validation samples
 |
 |
 |
 |
 |
 |
Prediction and ground truth of lowest F1 score samples in mesh prediction out of 100 validation samples
 |
 |
 |
 |
 |
 |
The results suggests that the networks are overfit to the general shape of chair with a base and head. With chairs highly varied from the standard shape F1 score decreases drastically.
F1 score: Avg F1 at 0.05: 58.079
 |
 |
 |
 |
 |
 |
F1 score: Avg F1 at 0.05: 77.367
 |
 |
 |
 |
 |
 |
