16-889: Learning for 3D Vision (SP22) - Assignment 3

• Name: Yen-Chi Cheng (yenchich@andrew.cmu.edu)
• AndrewID: yenchich

===================================

1. Differentiable Volume Rendering

1.3. Ray sampling (10 points)

The visualization of grid and rays:

1.4. Point sampling (10 points)

The visualization of the sample points of the first camera:

1.5. Volume rendering (30 points)

The visualization of the spiral rendering:

And the visualization of the depth:

2. Optimizing a basic implicit volume

2.2. Loss and training

Box center: (0.25, 0.25, 0.00)
Box side lengths: (2.00, 1.50, 1.50)

2.3. Visualization

The visualization of the optimized volume is shown below. Left: TA-image. Right: output from my implementation.

3. Optimizing a Neural Radiance Field (NeRF) (30 points)

The visualization of the trained NeRF:

4. NeRF Extras (Choose at least one! More than one is extra credit)

4.1 View Dependence (10 pts)

NeRF with view dependence:

Trade-offs between increased view dependence and generalization quality: adding view dependece makes the model easier to memorize the pixel values and the corresponding rays. So the model might perform poorly on novel views. For instance, for a ray that directly hitting the front of the lego, model can directly copying the pixels when rendering a novel view from the back of the lego. This is not ideal as the rendering images of front and the back of the lego differ drastically.

4.2 Hierarchical Sampling (10 pts)

NeRF with hierarchical sampling:

Trade-offs between speed and quality: hierarchical sampling increases the quality since we sample more relevant points along each ray when optimizing the volume. However, this requires one more forward pass of the coarse networks and additional sampling during the inverse transform sampling step. Therefore, each forward pass is 2x slower for the exchange of better quality.

4.3 High Resolution Imagery (10 pts)

NeRF with higher resolution:

I increase the number of the harmonic function for embedding of the viewing directions. I also decrease the chunk_size to fit the model into the GPU. Other parameters such as hidden_size of the MLP, dimension of the positional embedding, ..., etc., remains the same.