Assignment 3 of Learning for 3D Vision (16-889)

1. Differentiable Volume Rendering

1.3. Ray sampling (10 points)

1.4. Point sampling (10 points)

1.5. Volume rendering (30 points)

2. Optimizing a basic implicit volume

2.1. Random ray sampling (5 points)

plese refer the ray_utils.py

2.2. Loss and training (5 points)

Box center: (0.249, 0.25, -0.0006)

Box side lengths: (2.00, 1.50, 1.50)

2.3. Visualization

optimized volume

3. Optimizing a Neural Radiance Field (NeRF) (30 points)

Visualization

Running command : python -u main.py --config-name=nerf_lego

4. NeRF Extras (Choose at least one! More than one is extra credit)

4.1 View Dependence (10 pts)

The trade-offs between increased view dependence and generalization quality: Adding view-dependence should help in explaining the specular effect in the scene (shadows and lighting changes when viewed from different viewing angles). However, there is also a possiblity that the model can overfit to these direction and not learn consistant scene geometry leading to less generalizable model.

Running command : python -u main.py --config-name=nerf_lego_view_dep

4.3 High Resolution Imagery (10 pts)

The following model was trained with 128 sampling points and MLP decoder comprising of 10 layers with 256 as hidden dimension. We observe that the reconstructions are sharp.

Running command : python -u main.py --config-name=nerf_lego_highres

The following model was trained with 32 sampling points and MLP decoder comprising of 10 layers with 256 as hidden dimension. Hence, we observe reducing the number of samples along the rays degrades the performance quite a bit.

The following model was trained with 128 sampling points and MLP decoder comprising of 5 layers with 128 as hidden dimension. Hence, we observe reducing the complexity of the implicit decoder degrades the performance.

Number of late days 1