Assignment 3

Name: Manikandtan Chiral Kunjunni Kartha

Andrew ID: mchiralk

Late days used: 1

one late day

1. Differentiable Volume Rendering

1.3. Ray sampling (10 points)

run command

python main.py --config-name=box

Visualization

XY grid	Rays

1.4. Point sampling (10 points)

run command

python main.py --config-name=box

Visualization

Sample points

1.5. Volume rendering (30 points)

python main.py --config-name=box

Visualization

Rendered cube	Depth Image

2. Optimizing a basic implicit volume

2.1. Random ray sampling (5 points)

def get_random_pixels_from_image(n_pixels, image_size, camera):
    xy_grid = get_pixels_from_image(image_size, camera)

    # TODO (2.1): Random subsampling of pixel coordinates
    indices = torch.randint(low=0, high=xy_grid.shape[0], size=(n_pixels, 1))
    xy_grid_sub = xy_grid[indices].squeeze(1).to(camera.device)

    # Return
    return xy_grid_sub.reshape(-1, 2)[:n_pixels]

2.2. Loss and training (5 points)

Replace the loss in train

main.py

 # TODO (2.2): Calculate loss
 loss = torch.nn.MSELoss(reduction='mean')(out['feature'], rgb_gt)

run command

python main.py --config-name=train_box

Result

Box center: (0.25023114681243896, 0.25058114528656006, -0.0004499410279095173)

Box side lengths: (2.0051236152648926, 1.5036123991012573, 1.5034124851226807)

2.3. Visualization

3. Optimizing a Neural Radiance Field (NeRF) (30 points)

run command

python main.py --config-name=nerf_lego

Visualization

Spiral Rendering of Part 3

4. NeRF Extras (Choose at least one! More than one is extra credit)

4.1 View Dependence (10 pts)

# modify in config:
# implicit_function:
#   enable_view_dependence: true
python main.py --config-name=nerf_lego

without view dependence	with view dependence

observations:

view dependence models specular reflections more accurately and makes the scene more realistic
A larger view dependence weight can make the network overfit to the view in the training set and would not generalize well for novel views.
Overfitting can lead to wispy artifacts when looking from novel orientations

4.3 High Resolution Imagery (10 pts)

128 px, 128 pts per ray	400px, 128 pts per ray

varying n_pts_per_ray

higher n_pts_per_ray helps in having higher resolution output render, but increases memory usage

32 pts per ray, 32768 chunk size	128 pts per ray, 32768 chunk size	256 pts per ray, 24576 chunk size
$128px$