Assignment 4

AndrewId: nlakshmi

Late days: 1

---

1. Sphere Tracing (30pts)

python -m a4.main --config-name=torus

2. Optimizing a Neural SDF (30pts)

I've implemented a network similar to the network defined in the VolSDF Paper. The hyperparametrs and the network depth is controlled from the appropriate config files for each experiment.

python -m a4.main --config-name=points

Input Pointclouds	Output Mesh

3. VolSDF (30 pts)

I've implemented the network as in the VolSDF paper, but with a reduced network size as shown below. The hyperparameters are controlled from this file as well.

implicit_function:
  type: neural_surface
  n_harmonic_functions_xyz: 6
  n_layers_distance: 8
  n_hidden_neurons_distance: 256
  append_distance: [4]
  n_layers_color: 4
  n_hidden_neurons_color: 256
  append_color: []

renderer:
  type: volume_sdf
  chunk_size: 32768
  white_background: False
  alpha: 10.0
  beta: 0.1

python -m a4.main --config-name=volsdf

Geometry	Color

SDF to Density: Read section 3.1 of the VolSDF Paper and implement their formula converting signed distance to density in the sdf_to_density function in a4/renderer.py. In your write-up, give an intuitive explanation of what the parameters alpha and beta are doing here. Also, answer the following questions:

Answer: An intuitive explanation for the density \(\sigma\) models a homogenous object with a constant density \(\alpha\) which smoothly decreases near the object's boundary. And \(\beta\) controls the amount of smoothing.

1. How does high beta bias your learned SDF? What about low beta?

   As \(\beta\) reduces, the density \(\sigma\) converges to a scaled indicator function of the signed distance, i,e. \(\sigma \rightarrow \alpha \mathbf{1}_{\Omega}\). This can intuitively thought of as controlling the sharpness of the rendered object density. Lower the \(\beta\), better is the rendered density (ie. higher sharpness). Else if \(\beta\) is high, the rendering will be blurry.

2. Would an SDF be easier to train with volume rendering and low beta or high beta? Why?

   It would be eaiser (ie. faster) to learn with high \(\beta\) because the loss for density would be low as \(\beta\) controls the average density around the object, leading to a faster convergence.

3. Would you be more likely to learn an accurate surface with high beta or low beta? Why?

   It would be more likely to learn a better surface with low \(\beta\), because, as explained above, as \(\beta\) reucdes, the density \(\sigma\) become a scaled value of the distance. What this means is the object boundaried would be much clear. As a result, the surfaces would be more accurate.

\(\beta\)	Geometry	Color	Time
1			67m33.011s
0.1			67m53.609s
0.05			68m10.476s
0.005			69m25.347s

4. Neural Surface Extras (CHOOSE ONE! More than one is extra credit)

4.1. Render a Large Scene with Sphere Tracing (10 pts)

In Q1, you rendered a (lonely) Torus, but to the power of Sphere Tracing lies in the fact that it can render complex scenes efficiently. To observe this, try defining a ‘scene’ with many (> 20) primitives (e.g. Sphere, Torus, or another SDF from this website at different locations). See Lecture 2 for equations of what the ‘composed’ SDF of primitives is. You can then define a new class in implicit.py that instantiates a complex scene with many primitives, and modify the code for Q1 to render this scene instead of a simple torus.

I've implemnted a custom class KlnSDF in implicit.py that creates 25 torus, spatially placed around the origin. The below results show the SDF outputs of this scene.

python -m a4.main --config-name=kln

4.2 Fewer Training Views (10 pts)

In Q3, we relied on 100 training views for a single scene. A benefit of using Surface representations, however, is that the geometry is better regularized and can in principle be inferred from fewer views. Experiment with using fewer training views (say 20) -- you can do this by changing train_idx in data laoder to use a smaller random subset of indices). You should also compare the VolSDF solution to a NeRF solution learned using similar views.

The sparsity of input data is controlled from the config file as shown below.

data:
  image_size: [128, 128]
  dataset_name: lego
  sparse: True
  sparse_samples: 20
  sparse_type: random # or uniform

python -m a4.main --config-name=volsdf

As seen from the below results, VolSDF if able to generate pretty good model even with just 20 inputs.

Num Samples	VolSDF	NeRF
100
20

Observations

• The above resulst are from uniform down sampling. However, if randomly sampled and there aren't good number of input views covering the object, then in such cases, NeRF fails to generate any outputs.
• It is difficult to visualize the improvement in these small image and small networks. The advantges are clearly visible from the paper outputs from larger images and networks.

4.3 Alternate SDF to Density Conversions (10 pts)

In Q3, we used the equations from VolSDF Paper to convert SDF to density. You should try and compare alternate ways of doing this e.g. the ‘naive’ solution from the NeuS paper, or any other ways that you might want to propose!

For this, I used the same network architecutre, but a different sdf_to_density function as defined in the NeuS paper. The hyperparameters are controlled from a differnet config file neus.yaml.

renderer:
  type: neus_sdf
  chunk_size: 32768
  white_background: False
  s: 50.0

x = signed_distance
e = torch.exp(-s * x)
psi = s * e / torch.square(1 + e)

python -m a4.main --config-name=neus

s	Geometry	Color
1
20
50

---

Last update: April 3, 2022