Assignment 2: Single View to 3D

1. Exploring loss functions

1.1. Fitting a voxel grid

My optimized voxel grid (left: truth, right: predicted)

1.2. Fitting a point cloud

My optimized point cloud (left: truth, right: predicted)

1.3. Fitting a mesh

My optimized mesh (left: truth, right: predicted)

2. Reconstructing 3D from single view

2.1. Image to voxel grid

Set 1 - From top to bottom (input RGB, predictd voxel grid, truth mesh)

Set 2 - From top to bottom (input RGB, predictd voxel grid, truth mesh)

Set 3 - From top to bottom (input RGB, predictd voxel grid, truth mesh)

2.2. Image to point cloud

Set 1 - From top to bottom (input RGB, predictd point cloud, truth mesh)

Set 2 - From top to bottom (input RGB, predictd point cloud, truth mesh)

Set 3 - From top to bottom (input RGB, predictd point cloud, truth mesh)

2.3. Image to mesh

Set 1 - From top to bottom (input RGB, predictd mesh, truth mesh)

Set 2 - From top to bottom (input RGB, predictd mesh, truth mesh)

Set 3 - From top to bottom (input RGB, predictd mesh, truth mesh)

2.4. Quantitative comparisions

3D Type	F1 Score
Voxel Grid	81.98%
Point Cloud	17.165%
Mesh Grid	38.66%

I can say that the point cloud and mesh did not perform as well as the voxel grid maybe due to my network structure which could not adapt wll to the point cloud and meshes. But overall, point clouds should yield a higher F1 score because there is no need to take into consideraion, the conectivity constraints compared to the other two representaions and are more flexible to reconstruct.

2.5. Analyse effects of hyperparms variations

2.6. Interpret your model

I did not get a chance to complete these last two questions and did not want to use any of my late days as of yet.