16-889 16-889 Assignment 3

-1. Number of Late Days

0. Setup

Q1 Differentiable Volume Rendering

Q1-3 Ray sampling

Grid	Rays

Q1-4 Point sampling

Q1-5 Volume rendering

Spiral Rendering of Part 1	Depth Rendering of Part 1 (A)	Depth Rendering of Part 1 (B)

Here, I used two methods of rendering depth from feature and weight output. Method A produces a smooth continuous depth map, while method B produces a discrete one. Method A's depth map has wrong values at the object edge. To conquer that, method B relies on a filtering threshold, and different threshold selections will produce different depth maps.

(A) weight.argmax() * feature

(B) weight.argmax() * (feature > 0.5)

Q2 Optimizing a basic implicit volume

Q2-1 Random ray sampling

Q2-2 Loss and training

Report the center of the box, and the side lengths of the box after training, rounded to the nearest 1/100 decimal place.

Box center: (0.25, 0.25, -0.00)

Box side lengths: (2.01, 1.50, 1.50)

Q2-3 Visualization

The following two gifs look almost the same.

ta_images/part_2.gif	images/part_2.gif)

Q3 Optimizing a Neural Radiance Field (NeRF)

The following gif is rendered by my neural radiance field model after trained for 250 epochs. The color and density field don't depend on the viewing angle information.

Q4 NeRF Extras

Q4-1 View Dependence

Two extra fully connected layers are attached at the end of the previous architecture. The inputs are the previous last latent layer and the viewing angle, and the output is still the rgb color. This architecture enables view dependency, and similarly, it can render a 360-degree-view gif. To better compare it with the Q3's result, pictures in four frames (7, 8, 9 and 10) are cherrypicked and shown in the following table. A very clear contrast happens on the shovel. The shovel on the right column have darker color at frame 7 and 8, while at frame 10, there's no big difference.

Frame	Q3 (no view dependency)	Q4-1 (with view dependency)
7
8
9
10

Q4-3 High Resolution Imagery

A few changes were made to the configuration file:

xxxxxxxxxx
renderer:
  type: volume
  chunk_size: 32768 -> 8192
  white_background: False

implicit_function:
  type: nerf

  n_harmonic_functions_xyz: 6 -> 9
  n_harmonic_functions_dir: 2 -> 3
  n_hidden_neurons_xyz: 128 -> 256
  n_hidden_neurons_dir: 64 -> 128
  density_noise_std: 0.0
  n_layers_xyz: 6 -> 8
  append_xyz: [3] -> [4]

The resultant higher resolution gif can be rendered, and on contrast to the low resolution version, here the gray dots on the lego board are visible.