16-889: Assignment 3

Ben Eisner

This assignment was submitted 2 days late.

1. Differentiable Volume Rendering

1.3. Ray sampling (10 points)

xy_grid	rays

1.4. Point sampling (10 points)

Here are the points as rendered.

1.5. Volume rendering (30 points)

I implemented it. It's pretty fast on an RTX 3090.

Depth render	Spinning cube

2. Optimizing a basic implicit volume

2.1. Random ray sampling (5 points)

Implemented. Used the nice permutation trick.

2.2. Loss and training (5 points)

Here are the rounded estimates:

Box center: (0.25, 0.25, 0.00)

Box side lengths: (2.00, 1.50, 1.50)

2.3. Visualization

Here's the rendered wonky box.

3. Optimizing a Neural Radiance Field (NeRF) (30 points)

Initially, I implemented things without view dependence. This resulted in an image that was a tiny bit noisy and a tiny bit imprecise. I ended up using the nice skip architecture provided by the friendly TAs.

3090 go BRRRRR training only took 5min. Who needs instantngp anyway... Here's my shovel:

Notice that there's a bit of noise underneath the shovel. This is because the direction wasn't considered, making things harder to optimize in a global sense.

4. NeRF Extras (Choose at least one! More than one is extra credit)

4.1 View Dependence (10 pts)

This was pretty easy, I just followed the architecture from the original NeRF paper, but modified the number of layers from the original one to the one suggested by the TA's in their config file.

Here's my much cleaner shovel. Also it took fewer steps to converge.

Thankfully, my network did not overfit to the training images. In general though, if you also provide the directions, and you have a fixed (and small) number of perspectives, the network can just memorize the viewpoint-dependent color. There's no telling what the network would predict when interpolating between the viewpoints in this case. However, since the viewpoint is not used in connection with the density (how the original NeRF paper does it), then there's no danger of overfitting the density prediction based on viewpoint.

4.2 Hierarchical Sampling (10 pts)

No time.

4.3 High Resolution Imagery (10 pts)

No time.