16-889 HW1 by Sheng-Yu Wang (andrew ID: shengyu2)

Q1.1

Voxel reconstruction.

source	target

Q1.2

Point reconstruction.

source	target

Q1.3

Mesh reconstruction.

source	target

Q2.1

Image to voxel.

input RGB	prediction	ground truth

Q2.2

Image to points.

input RGB	prediction	ground truth

Q2.3

Image to mesh.

input RGB	prediction	ground truth

Q2.4

Quantitative comparison.

(type: Avg F1@0.05)
vox: 29.133
point: 90.747
mesh: 74.925
Voxel resolution can be too crude to provide better F1 score that matches the point prediction. While point prediction obtains the best F1 score, the mesh prediction regularizes the mesh smoothness. The mesh model doesn't obtain better F1 score, but is capable of generating a 3D representation that is smoother.

Q2.5

I analyze the w_smooth term. The default is 0.1, and I tried training it with higher value (10) and lower value (0.001). Results are shown below. Note that the lower the w_smooth regularization, the higher the F1 score is. However, if we plot out the meshes, we can find out that the model trained with low w_smooth loss generates mesh that is not smooth enough. Therefore, an adequate smooth loss (0.1) is necessary to do the tradeoff.

ws10: 32.109 ws0.001: 87.750

input RGB	w_smooth=10	w_smooth=0.1	w_smooth=0.001	ground truth

Q2.6

I plot out the spatial chamfer error map of the predicted mesh. To do this, I sample a lot of points from both the predicted mesh and the ground truth mesh. Then, a chamfer loss is assigned to each point on the predicted mesh, and we visualize this error. Red is high error, and vice versa for blue.

input RGB	w_smooth=0.1	w_smooth=0.001	ground truth