1. Sphere Tracing (30pts)

Figure layout:

Attached below is the sphere tracing output obtained from our implementation. This figure could be reproduced by running

python -m a4.main --config-name=torus

Implementation Description:

For sphere tracing, we initialize the points at the given origin locations. Thereafter, considering the extreme limits provided in the config, we design the logic such that the ray is marched along the space until it hits the surface. While the ray is being marched, implicit function is continuosly evaluated at the location to check whether the points are outside or on the surface of the torus. The sphere tracing logic is as follows:

0. Initialize points at the origins.

1. Start a for loop until maximum iterations

2. Evaluate the implicit function at this point.

3. Generate masks. Provide True flag to all the points that are on the surface. Provide False flag to all the points that are outside (the threshold is set at 1e-5).

4. Normalize and generate the points considering the extreme locations of self.far and self.near.

5. Set a limit. If the rays does not find any intersection till self.far, break the loop.

6. Return the points and masks.

2. Optimizing a Neural SDF (30pts)

Figure layout:

Left figure is the pointcloud input.
Right figure is our optimized neural SDF output.

Implementation Description (MLP):

Our MLP was exactly the same as the NeRF architecture implementation except for the following changes:

1. Since SDFs could have negative values, we removed the ReLU nonlinearities from our MLP architecture.

2. Since we need a distance as the final output, and since it need to be within a normalized range, we changed the final output layer to output a single scalar and removed the sigmoid layer.

Implementation Description (Eikonal Losses):

For the Eikonal losses within the a4/losses.py , we simply take the norm of the gradients along the dim=1 and force it to be close to 1. The loss looks something like this:

eikonal_loss = torch.mean(torch.abs(torch.norm(gradients, dim=1) - 1))

Moreover, the pointcloud SDF loss on the distances in main.py takes the absolute values of the distances and minimizes this value. This loss is implemented as

loss = torch.abs(distances[:,0]).mean()

Hyperparameter description:

For the result shown above, following hyperparameters were used:

n_harmonic_functions_xyz: 4

n_layers_distance: 8

n_hidden_neurons_distance: 256

append_distance: [4]

num_epochs: 5000

3. VolSDF (30 pts)

In your write-up, give an intuitive explanation of what the parameters alpha and beta are doing here?

We implement the following equations from the VolSDF paper to generate the density values from the sdf.

Although alpha and beta are learnable parameters in the original VolSDF paper, we use the constant values that we provide from the relevant config files. As stated in the first expression above, alpha is a scaling factor for the generated density. On the other hand, beta adjusts the scale of the Laplace distributions. One thing to note is that as the beta values decreases and goes closer to 0, the density function converges to the indicator function of the occupied space.

1. How does high beta bias your learned SDF? What about low beta?

Higher beta would increase the calculated density, which would lead to occupancy looking denser and higher. On the other hand, lower beta would decrease the apparent density, resulting in reduced occupancy magnitude in the renderings. With a lower beta, there would be some artifacts in the occupancy and gaps would be visible in the renderings. More precisely, we would see non-continuous surface renderings with a low beta values.

2. Would an SDF be easier to train with volume rendering and low beta or high beta? Why?

Higher beta would be easier to train. The reason being: with a lower beta, since the denominator of the density function would approach 0, the exponential might result in NaN values and hence the model might fail to converge and learn. Another reason being: since the gradient of capital Psi with respect to the signed distances would be very small, there might be some issues with backpropagation due to very small gradients, resulting in failing to converge.

3. Would you be more likely to learn an accurate surface with high beta or low beta? Why?

An accurate surface would be learned more likely with a lower beta. Since lower beta generates more continuous and occupied surfaces (since the density values it produces are more closer to 0 or 1) -- which is crucial to understanding the surfaces, lower beta is a better candidate to learn accurate surfaces.

Our best results are as follows. I found beta = 0.05 to work the best. I observed that this value of beta is an optimum value since it is not too low, nor too high.

4. Neural Surface Extras (CHOOSE ONE! More than one is extra credit)

4.2 Fewer Training Views (10 pts)

With 20 views

We randomly chose 20 views and ran VolSDF as well as NeRF using the same epochs and similar trainig parameters. The comparison between the VolSDF solution (left figure) and NeRF solution (right figure) is attached below. Turns out NeRF solution is pretty good and comparable to the solution by VolSDF.

With 5 views

Similar experiment was carried out with 5 Views. I observed similar performance from SDF and NeRF for this setup as well. One reason we are observing this behavior may have to do with the the fact the 5 views might not have captured enough variability in the surface representations. For the following figures, the figure on the left is the VolSDF output and the figure on the right is the NeRF output.

4.3 Alternate SDF to Density Conversions (10 pts)

For the alternate SDF to Density conversion, we implement the following naive expression from the NeuS paper, as the function def NeuS_density in renderer.py.

Here, we take s = 100 and x represents the signed distance. All the parameters were kept the same as we used in Q3 above. The best results are attached below.