16889 - Assignment 4

Jinkun Cao (jinkunc@andrew.cmu.edu)

1. Sphere Tracing

A short description of my implementation:

given the position of origins, and the directions, we could statrt from the origin positions for each ray then march forward step by step. In each step, the forward length is just the normalized length defined in directions though there might be other choices. Given a maximum steps and a tolerance of distance tolerance, if on one step, the output distance from the implicit function regarding a ray is smaller than the tolerance, we consider it as an intersection and won't continue forwarding the ray. If a ray can't reach an intersection till the maximum step, we simply stop it and set the corresponding position in mask as False.

The output of my implementation is shown as below.

part_1

2. Optimizing a Neural SDF

A short description of my implementation:

MLP

For MLP implementation, I basically follow the network I used in Assignment 3 but adopts the parameters defined in the config file provided this time. I have a HarmonicEmbedding to convert the input points into an embedding, which is later forward to a MLP as an instance of the provided class MLPWIthInputSkips, whose hyperparameters are defined in the configs/points.yaml, such as hidden dimensions. Then, I have a simple linear layer to predict the distance from the features from MLP.

Eikonal Loss

I simply follow the eikonal regularization provided in the page 33 of the lecture slide#14 as shown below:

||∇f||−1

The output from my implementaion is as shown below

part_2

3. VolSDF

I tried to tune the value of both alpha and beta. The visualizations are shown as below

alpha = 10.0 beta=0.05 (default)

alpha = 10.0 beta=0.03

alpha = 10.0 beta=0.5

alpha = 100.0 beta=0.05

When the beta is too small, given the default training parameters, the network will fall in NaN issue.

In the density generation from the VolSDF paper, alpha controls the overall scale of the output density and beta controls the sensitivity of the sdf change. To be precise, a low value of beta means high sensitivity of density regarding the change of distance.

  1. I find when the value is bigger, the learned color renderring is blurred and geometry is denser but the detail is softer. With low beta, it shows the opposed pattern with sharper details.
  2. During training I find a high value of beta can make the training easier but it converges to a not that optimial position. I think this is because in the form provided in the VolSDF paper, a high value tends to output density less sensitive to the distance change. So the workload for optimization is more smooth, making the loss curve smooth as well. But essentially, the training should depend on the cooperations of the full stack hyperparameters. Given the analysis of the roles of alpha and beta, if we'd like to make a good model, we should choose a relatively small value of beta.
  3. From the analysis and intuition discussed above, I will be more likely to choose a low beta, so the density can be more fine-grainedly related to the distance, resulting in more details for accurate surface.

Given the results above, I would see the combinatin ofalpha = 10.0 beta=0.03 makes the best result I get.

4. Neural Surface Extras

4.2 Fewer Training Views

I tried to use only 20 views uniformly selected from the original 100 views for training. I use the default parameters of alpha = 10.0 beta=0.05 for this experiment. Compared with the result from the same parameters but using 100 views above, the result is obviously more blurred.

Also, by using the NeRF implementation from the previous assignment 3 with the same views for training, I get the results as below

I have to see NeRF performs much better than Neural SDF at least regarding my implementation. There are many variables that we can't control the same for the two groups. So this is just to provide a rough sense at the setting I worked on.

4.3 Alternate SDF to Density Conversions

Referring to the provided NeuS paper, I implement an alternative version of density conversion from SDF, which is controled by an S-term as a parameter. I tried different values of S-term and show the corresponding visualizations as below

S = 1.0

S = 10.0

S = 100.0

It is clear that, at least in the range of S-term I tried, a higher S value makes better performance. But considering the best result is still inferior than the previous ones we get from VolSDF or NeRF, there are still good potential we can continue to improve the implementation.