Assignment 3

NAME: Mayank Agarwal

ANDREW ID: mayankag

LATE DAYS USED: 0

1. Differentiable Volume Rendering

1.3. Ray sampling (10 points)

Visualization

python main.py --config-name=box

1.4. Point sampling (10 points)

Visualization

python main.py --config-name=box

1.5. Volume rendering (30 points)

# Call implicit function with sample points
implicit_output = implicit_fn(cur_ray_bundle)
density = implicit_output['density']
feature = implicit_output['feature']

Visualization

python main.py --config-name=box

2. Optimizing a basic implicit volume

Visualization

python main.py --config-name=train_box

My Result

Reference Output

Box center: (0.25, 0.25, 0.00)
Box side lengths: (2.00, 1.50, 1.50)

3. Optimizing a Neural Radiance Field (NeRF) (30 points)

Visualization

python main.py --config-name=nerf_lego

4. NeRF Extras (Choose at least one! More than one is extra credit)

4.1 View Dependence (10 pts)

python main.py --config-name=nerf_lego_viewd

The NeRF paper solely uses position encoding to calculate the density and the direction is used only for predicting the color information. If we pass the direction input, the model might overfit to the input views. This model won't generalize well to unknown views. Adding view dependence improves the reconstruction quality, however the differences are not very significant for low-res lego reconstruction. In general, view dependence produces more realistic outputs (view dependent effects such as specular effects) compared to only positional encoding results.

If we squint enough, we can see view dependent color changes on the top of the bulldozer and its front blade. Without direction information, we observe some artifacts (reflection-like) in the front view rendering in the gif. Adding view dependence improves the generalization ability to unseen views, by allowing the network to learn colors that are consistent with the viewing directions.

4.2 Hierarchical Sampling (10 pts)

NeRF employs two networks: a coarse network and a fine network. During the coarse pass, it uses the coarse network to get an estimate of geometry geometry, and during fine pass uses these geometry estimates for better point sampling for the fine network. Implement this hierarchical point-sampling strategy and discuss trade-offs (speed / quality).

4.3 High Resolution Imagery (10 pts)

python main.py --config-name=nerf_lego_highres