Late Days Used: 2

1. Differentiable Volume Rendering

python main.py --config-name=box

1.3 Ray Sampling

xy_grid	rays

These images are produced at images/xy_grid.png and images/rays.png.

1.4 Point Sampling

This image is produced at images/1-4.png.

1.5 Volume Rendering

Box	Depth

These outputs are produced at images/part_1.gif and _2.png.

2. Optimizing a basic implicit volume

python main.py --config-name=train_box

2.2 Center & Side Lengths of Box

Box Center: $(0.25, 0.25, - 0.00)$

Box Side Lengths: $(2.00, 1.50, 1.50)$

2.3 Visualization

This output is produced at images/part_2.gif.

3 NeRF

python main.py --config-name=nerf_lego

This output is produced at images/part_3.gif.

4.1 View-Dependent NeRF

python main.py --config-name=nerf_lego_view

No view dependence	View-dependent

The output is produced at images/part_3.gif.

Trade-off between increased view dependence and generalization quality:

• We observe that the specular or non-Lambertian surfaces in the scene (especially the siren at the top of the bulldozer and the surrounding area of the roof) look more realistic because they vary in their color/intensity values with viewing direction. This shows us the specular or shiny nature of the surface which seems more natural to something we might see in the real world.

• As done in the original NeRF paper, I passed the harmonic embedding of direction to the network near the very end of the network, allowing just a couple of layers to predict RGB after predicting density. Increased view dependence can be achieved by passing the viewing direction earlier in the NeRF network, so that the color predicted by the network is a more complex function of the viewing direction (as direction is processed through more layers).

• However, as we increase view dependence by adding more layers to process the direction, we run into the danger of the network memorizing exactly what the scene looks like at a particular point when viewed from a particular direction. This will make the network overfit to images (& viewpoints) which exist in our training set, but not generalize well to novel viewpoints outside our training set.

4.3 High-res NeRF

Experiments with hidden layer size: n_hidden_neurons_xyz

The three experiments below can be run as:

python main.py --config-name=nerf_lego_highres_hidden_128

python main.py --config-name=nerf_lego_highres_hidden_256

python main.py --config-name=nerf_lego_highres_hidden_512

The output for each of these is produced at images/part_3.gif (they should be run sequentially to avoid overwriting).

Hidden Layer Size	Chunk Size	Result
128	32768
256	32768
512	16384

Experiments with number of sampled points per ray: n_pts_per_ray

The three experiments below can be run as:

python main.py --config-name=nerf_lego_highres_ray_64

python main.py --config-name=nerf_lego_highres_hidden_256 (intentionally same as second above)

python main.py --config-name=nerf_lego_highres_ray_256

The output for each of these is produced at images/part_3.gif (they should be run sequentially to avoid overwriting).

n_pts_per_ray	Chunk Size	Result
64	32768
128	32768
256	16384