1. Differentiable Volume Rendering

1.3. Ray sampling (10 points)

1.4. Point sampling (10 points)

1.5. Volume rendering (30 points)

Rendered features and depth map:

2. Optimizing a basic implicit volume

2.1. Random ray sampling (5 points)

Center of the box: $(0.25, 0.25, -0.00)$

Side lengths of the box: $(2.01, 1.50, 1.50)$

• Results with more decimal points

  Center: $(0.2502155005931854, 0.25056925415992737, -0.00043147074757143855)$Box side lengths: $(2.0051181316375732, 1.503587007522583, 1.5033600330352783)$

2.3. Visualization

3. Optimizing a Neural Radiance Field (NeRF) (30 points)

Implemented Nerf without view-dependency (in code submission)

Visualization:

4. NeRF Extras (Choose at least one! More than one is extra credit)

4.1 View Dependence (10 pts)

Implemented view-dependent nerf (in code submission)

Visualization:

Adding view dependency did not change the generalization quality significantly. Based on the rendering, we can see that both achieved reasonable results for novel view synthesis, and both have captured the fine details. Here we also compare the results for higher-resolution nerfs:

High resolution xyz-only nerf:

High resolution directional nerf:

The result is consistent with the lower-resolution: for this scene, the addition of view-dependency does not degrade the generalization quality that much. However, with directional nerf the bottom wooden plate has a clearer patten than without, showcasing the ability of the view-dependency model for modeling view-changes better.

4.3 High Resolution Imagery (10 pts)

High-resolution, 64 points along the ray

High-resolution, 128 points along the ray

High-resolution, 256 points along the ray

Findings:

• Initialization:

  During training and tuning, I noticed that the model is unstable when changing the number of points on the sampling ray. Different random seed will cause the model to not converge. After investigation, I realize that the model is relatively sensitive to weight initialization. Thus, I added xavier initialization to the model for more stable performance:

  torch.nn.init.xavier_uniform_(self.linear.weight.data)
  for l in self.density_linear:
      if isinstance(l, torch.nn.Linear):
          torch.nn.init.xavier_uniform_(l.weight.data)
  for l in self.color_linear:
      if isinstance(l, torch.nn.Linear):
          torch.nn.init.xavier_uniform_(l.weight.data)

• Number of points along the ray: I tried varying the number of points along the ray (trying 64, 128, 256). During training, the 64 points along the ray model converges the fastest and produced good results early on. This is expected as the model has lower points to learn and is able to converge quickly. The final results show that though 64 points along the ray can achieve reasonable result, 256 points can capture finer details (e.g. the rendered image looks less blurred, the railing of the bulldozer is captured better).