/home/nabakshi/anaconda3/envs/l3dgpu/bin/python

1 Differentiable volume Rendering

1.3 Ray Sampling

Code to generate outputs: python main.py --config-name=box

1.4 Point Sampling

Code to generate outputs: python main.py --config-name=box

1.5 Volume Rendering

Code to generate outputs: python main.py --config-name=box

Depth map still

Depth map GIF

Autogenerated output

2 Optimizing a basic implicit volume

Code to generate is python main.py --config-name=train_box

Box center: (0.25, 0.25, -0.00)
Box side lengths: (2.01, 1.50, 1.50)

3 NeRF

Code to generate: python main.py --config-name=nerf_lego

Remember to set the parameter in nerf_lego.implicit_function.view_dep = False

More NeRF

4.1 View Dependence

Code to generate: python main.py --config-name=nerf_lego

Remember to set the parameter in nerf_lego.implicit_function.view_dep = True

The resolution in this image is too low to appreciate the view dependence effects coming to the fore however if you pay attention you'll see that the same pixels to the from of the lego bulldozer appear darker on the back swing of the truck as compared to the GIF in #3. I was not satisfied with the result hence I attempted view dependence again after switching to high res, stay tuned.

View Dependence vs Generalisation: If the direction related information is provided too early to the network it may be highly view dependent, which is undesirable as the object remains the same, hence we introduce the directional information later in the network (for me it was in the 6th layer) in order to have an inductive bias that allows the basic geometry to largely only depend on the positional information but have a smaller effect of the direction of viewing in the final RGB values produced.

4.3 High res

Code to generate: python main.py --config-name=nerf_lego_highres

Remember to set the parameter nerf_lego_highres.implicit_function.view_dep = False

Remember to set the parameter nerf_lego_highres.sampler.n_pts_per_ray = 128

Remember to set the parameter nerf_lego_highres.training.batch_size = 1024

At 240 iterations we can see that finer details of the lego bezels have not come through although the details are much much clearer than #3.

4.3 + 4.1 High res with view dependence

Code to generate: python main.py --config-name=nerf_lego_highres

Remember to set the parameter nerf_lego_highres.implicit_function.view_dep = True

Remember to set the parameter nerf_lego_highres.sampler.n_pts_per_ray = 256

Remember to set the parameter nerf_lego_highres.training.batch_size = 512

The view dependence effects are clearly shining through and we can see the extrusions of the lego board appear differently with different views depending on the shadows in just 250 iterations.

More Training

Using the exact same config as above but training another 100 epochs results in an even more accurate rendering:

Parameters played with

• I found that in the highres version it was important to increase the nerf_lego_highres.sampler.n_pts_per_ray parameter inorder to model finer details. Correspondingly batch_size needed to be reduced in order to keep memory requirements in check.
• Surprisingly I discovered that a model trained on a lower res output can be quite an effective pretraining for higher res output. In #4.3 (high res without view dependence) was trained on top of the the #3 model (basic nerf without view dependence) and in fewer iterations it could generate good results.
• I tried to play with higher network capacities as well, changing hidden layer size to 512 instead of the 256 used across the board in the above results however I personally found it hard to train. A few issues I ran into was instability a.k.a empty outputs eventhough it started out in a promising way as well as higher time requirements to train. Hence I stuck with a constant 256 hidden layer size.