16-889 Assignment 3

Name: Sri Nitchith Akula
Andrew ID: srinitca

1. Differentiable Volume Rendering

1.3. Ray sampling (10 points)

Run Command

python main.py --config-name=box

Visualization

XY-Grid	Rays

1.4. Point sampling (10 points)

Run Command

python main.py --config-name=box

Visualization

1.5. Volume Rendering (30 points)

Run Command

python main.py --config-name=box

Visualization

Rendered Cube	Depth Image

2. Optimizing a basic implicit volume

2.1. Random ray sampling (5 points)

Code:

def get_random_pixels_from_image(n_pixels, image_size, camera):
    xy_grid = get_pixels_from_image(image_size, camera)
    
    # TODO (2.1): Random subsampling of pixel coordinates
    xy_grid_sub = xy_grid[torch.randperm(xy_grid.size(0))[:n_pixels]]

    # Return
    return xy_grid_sub.reshape(-1, 2)[:n_pixels]

2.2. Loss and training (5 points)

Code:

loss = torch.nn.MSELoss(reduction='mean')(out['feature'], rgb_gt)

Run Command

python main.py --config-name=train_box

Results:

Box center: (0.25, 0.25,0.00)
Box side lengths: (2.00, 1.50, 1.50)

2.3. Visualization

Run Command

python main.py --config-name=train_box

Visualization

Before Training	After Training

3. Optimizing a Neural Radiance Field (NeRF) (30 points)

Run Command

python main.py --config-name=nerf_lego

Visualization

4. NeRF Extras (Choose at least one! More than one is extra credit)

4.1 View Dependence (10 pts)

Run Command

# Change use_dirs to True in nerf_lego.yaml
python main.py --config-name=nerf_lego

Visualization

Size	Without View Dependence	With View Dependence
128x128
400x400

Comments

• Having view dependence, helps in modelling speculaties at each point in all directions. It helps in rendering realistic views
• But increasing the view dependence can make the network overfit to the training image views and perform bad for novel views. Thus increasing too much of view dependence can lead to problem in generalization quality
• Example of overfitting using NeRF can be seen in this video (4th column). We can notice the ghosting artificats (white color) in some views

4.3 High Resolution Imagery (10 pts)

Run Command

python main.py --config-name=nerf_lego_highres

Visualization

Low Resolution	High Resolution

Effect of increasing n_pts_per_ray

n_pts_per_ray = 64	n_pts_per_ray = 128	n_pts_per_ray = 256

Comments

• Increasing n_pts_per_ray helps improving quality of rendered output