16-889 Assignment 4

Name: Sri Nitchith Akula
Andrew ID: srinitca

1. Sphere Tracing (30pts)

Run Command

python -m a4.main --config-name=torus

Implementation : Starting from the origin, we update each point in the direction of the ray based on the distance from the surface using implicit_fn. We update the points till iterations reaches max_iters. Mask is generated based on the final distance of the points w.r.t surface. I kept a threshold of 1e-3. We consider the points to be on surface if the final distance is < threshold.

We can exit the loop earlier by keeping an additional check if all points are within the threshold 1e-3 or if the distance of all points cross the far distance.

2. Optimizing a Neural SDF (30pts)

Run Command

python -m a4.main --config-name=points

Point Cloud	SDF Geometry

MLP Descrition :

• The architecture is similar to the one used for NeRF.
• We have 8 layers of Linear+RELU layers and one final Linear layer to get distance as the output. There is a skip connection connecting the input embeddings to 4th layer
• Here we do not use any activation function while computing distance. We want it to be unbounded and the output should support both +ve and -ve distance.

implicit_function:
  type: neural_surface

  n_harmonic_functions_xyz: 4

  n_layers_distance: 8
  n_hidden_neurons_distance: 256
  append_distance: [4]

  n_layers_color: 2
  n_hidden_neurons_color: 256
  append_color: []

Eikonal Loss :

• Ideally, sdf function should have ||gradient|| = 1. Eikonal loss tries to enforce this by penalizing the ||gradient|| - 1. Loss is minimal when this difference is as small as possible.

3. VolSDF (30 pts)

Set alpha, beta in volsdf.cfg and run the below command

Run Command

python -m a4.main --config-name=volsdf

alpha	beta	Geometry Output	Scene Outuput
10	0.001
10	0.05
10	0.5
10	1.0

alpha can be thought of constant density of the object while beta controls the decay (smoothening) of density as it crosses the boundary

1. How does high beta bias your learned SDF? What about low beta?

• Low beta enforces the network to learn sharp surfaces while high beta makes the surface smoothened.

2. Would an SDF be easier to train with volume rendering and low beta or high beta? Why?

• High beta should be easier to learn. If beta is high, then more points in the scene can have density and contribute to color of the pixel. So photometric loss of the rendered image will be low. In the end, you will be able to render back the color images but the surface learnt will be smoothened version.

3. Would you be more likely to learn an accurate surface with high beta or low beta? Why?

• Low beta is more likely to learn accurate surface. Low beta makes sure that the density at a point is high when it is only closer the surface. To render the image, it has to accurately learn the sdf, otherwise there will be many points with low density and image that gets rendered will be empty.

Architecture: We are using the same architecture defined in the previous question. After the 8 layers, we have 2 more layers + sigmoid to get the color outputs. beta with low values are performing well compared to high values as seen in the above figures.

4. Neural Surface Extras (CHOOSE ONE! More than one is extra credit)

4.1. Render a Large Scene with Sphere Tracing (10 pts)

Run Command

python -m a4.main --config-name=large_scene

The above scene is inspired from Tower of Hanoi and Rubik's Cube.

Tower of Hanoi :

I added new CapsuleSDF class to render the cylinder poles using implementation from this website. I created multiple TorusSDF and CapsuleSDF objects and positioned them apropirately to get Tower of Hanoi. I also added some cubes just for fun.

Rubik's Cube :

First, I added a black cube to the scene. I created new CubeFaceSDF class that creates 9 cubes with appropriate offset to form a single face of the Rubik's cube. Finally, I created 5 instances of CubeFaceSDF to render the five faces of the Rubik's cube

Total Objects : 11 (Torus) + 3 (Capsules) + 4 (Cubes) + 9 * 5 (Rubik's cube faces) = 63 objects

Coloring : Each object instance is assosicated with a color from the given color-palette. For every queried point, we assign the color of its closest object.

4.2 Fewer Training Views (10 pts)

Num Views	VolSDF	NeRF
100
20
10

Observations: We can makeout the structure of the bulldozer even from just 10 views in case of VolSDF

4.3 Alternate SDF to Density Conversions (10 pts)

Set alpha, s in volsdf_ners.cfg and run the below command

Run Command

python -m a4.main --config-name=volsdf_ners

I have implemented the SDF to density function from the NeuS paper. I experiemented with various s to increase/decrease the variance of the logistic distribution and I found out that when s > 100, loss becomes Nan. To counter this, I introduced the alpha scale factor. Below is the final formula.

density = alpha * s * torch.exp(-s * signed_distance) / (1 + torch.exp(-s * signed_distance)) ** 2

alpha	s	Geometry Output	Scene Outuput
0.1	100
0.5	100
1.0	100
10	100