Skip to content

Assignment 3

Name: Nitheesh Lakshminarayana

andrewId: nlakshmi

number of late days: 0

zero

1.3. Ray sampling (10 points)

Visualization

python main.py --config-name=box
Grid Rays
Grid Rays

1.4. Point sampling (10 points)

Visualization

Sample points

1.5. Volume rendering (30 points)

Visualization

Spiral Rendering of Part 1

Spiral Rendering of Part 1

2. Optimizing a basic implicit volume

2.2. Loss and training (5 points)

Replace the loss in train

python main.py --config-name=train_box
Result
Box center: (0.25006094574928284, 0.25053277611732483, -0.00043326508603058755)
Box side lengths: (2.004148006439209, 1.5033373832702637, 1.5034971237182617)

2.3. Visualization

Spiral Rendering of Part 2

3. Optimizing a Neural Radiance Field (NeRF) (30 points)

Visualization

I've added an extra parameter in config file to control view dependence.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
implicit_function:
  type: nerf
  view_dependence: False
  n_harmonic_functions_xyz: 6
  n_harmonic_functions_dir: 2
  n_hidden_neurons_xyz: 128
  n_hidden_neurons_dir: 64
  density_noise_std: 0.0
  n_layers_xyz: 6
  append_xyz: [3]

python main.py --config-name=nerf_lego

Spiral Rendering of Part 3

4. NeRF Extras (Choose at least one! More than one is extra credit)

4.1 View Dependence (10 pts)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
implicit_function:
  type: nerf
  view_dependence: True
  n_harmonic_functions_xyz: 6
  n_harmonic_functions_dir: 2
  n_hidden_neurons_xyz: 128
  n_hidden_neurons_dir: 64
  density_noise_std: 0.0
  n_layers_xyz: 6
  append_xyz: [3]

python main.py --config-name=nerf_lego_view_dependence

Spiral Rendering of Part 3

NeRF model predicts the volume density and features as a function of position and direction. If this direction (view-dependence) is increased, ie a lot of inputs along a particular direction, the model would overfit to this view. On the contrary, increasing the view-dependence allows the model to capture more details and specularities. Therefore, the view-dependence should be balanced to allow the model to be generalizable to all directions and also capture/render all the model details.

4.3 High Resolution Imagery (10 pts)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
training:
  num_epochs: 250
  batch_size: 512
  lr: 0.0005

renderer:
  type: volume
  #chunk_size: 32768
  chunk_size: 16384
  white_background: False

implicit_function:
  type: nerf
  view_dependence: True
  n_harmonic_functions_xyz: 6
  n_harmonic_functions_dir: 2
  n_hidden_neurons_xyz: 128
  n_hidden_neurons_dir: 64
  density_noise_std: 0.0
  n_layers_xyz: 6
  append_xyz: [3]

python main.py --config-name=nerf_lego_highres

Spiral Rendering of Part 3

I experimented with various points_per_ray and increasing the model size i,e. number of layers and number of neurons per layer. The below results show the results from increasing the number of points for a fixed model size. This shows that more the number of the sample points per ray, the better is the rendered model. However, it also increases the memory and compute. Increasing the model size and number of epochs enables the model to capture a lot of details, like the lego blocks, etc, however I ran out of memory for this configuration.

Points/Ray = 32 Points/Ray = 64 Points/Ray = 128
Spiral Rendering of Part 3 Spiral Rendering of Part 3 Spiral Rendering of Part 3
High Res
training:
  batch_size: 1024
renderer:
  chunk_size: 32768
sampler:
  n_pts_per_ray: 256
implicit_function:
  type: nerf
  view_dependence: True
  n_harmonic_functions_xyz: 6
  n_harmonic_functions_dir: 2
  n_hidden_neurons_xyz: 256
  n_hidden_neurons_dir: 128
  density_noise_std: 0.0
  n_layers_xyz: 8
  append_xyz: [6]

PS - I tried to implement the original network, but constantly found myself running out of memory, despite lowering the chunk size, batch_size, network size...etc. Either I ran out of memory, or the samples were so low that the rendered image was blank or wasn't useful. So just made this video to keep it lively :)

Last update: March 19, 2022
Back to top