16889 Assignment 4: Neural Surfaces.

Presented: Dijing Zhang

1. Sphere Tracing

My implementation of sphere tracing is based on the algorithm of slides, by interatively updating points and distance and then maintain a mask with distance below a threshold.

Pseduocode:

For loop in max_iters:
*   distance get from implicit function
*   update variable t adds distance
*   update points by adding t * distance to origins
Below threshold, mask[i] = True
Above threshold, mask[i] = False

2. Optimizing a Neural SDF

Model Architecture

Model(
  (implicit_fn): NeuralSurface(
    (harmonic_embedding_xyz): HarmonicEmbedding()
    (module): MLPWithInputSkips(
      (mlp): ModuleList(
        (0): Sequential(
          (0): Linear(in_features=27, out_features=256, bias=True)
          (1): ReLU(inplace=True)
        )
        (1): Sequential(
          (0): Linear(in_features=256, out_features=256, bias=True)
          (1): ReLU(inplace=True)
        )
        (2): Sequential(
          (0): Linear(in_features=256, out_features=256, bias=True)
          (1): ReLU(inplace=True)
        )
        (3): Sequential(
          (0): Linear(in_features=256, out_features=256, bias=True)
          (1): ReLU(inplace=True)
        )
        (4): Sequential(
          (0): Linear(in_features=256, out_features=256, bias=True)
          (1): ReLU(inplace=True)
        )
        (5): Sequential(
          (0): Linear(in_features=256, out_features=256, bias=True)
          (1): ReLU(inplace=True)
        )
      )
    )
    (distance_layer): Sequential(
      (0): Linear(in_features=256, out_features=256, bias=True)
      (1): ReLU(inplace=True)
      (2): Linear(in_features=256, out_features=1, bias=True)
    )
  )
  (sampler): StratifiedRaysampler()
  (renderer): SphereTracingRenderer()
)

Eikonal Loss:

It encourages the gradients to be of unit 2-norm, which makes sure that the distances don't go to zero (or close to zero) everywhere.

We can simply compute it by $(||gradient||_2 - 1)^2$

3. VolSDF

Give an intuitive explanation of what the parameters alpha and beta are doing here.

Alpha is a constant controls the magnitude of density and beta controls the smoothness of surface.

1. How does high beta bias your learned SDF? What about low beta?

High beta will make boundary blurreid by smoothing the transition and low beta will encourage the density to converge to a scaled indicator function of Ω, which means CDF gets larger outside the object and approach 1/2 inside the object.

2. Would an SDF be easier to train with volume rendering and low beta or high beta? Why?

SDF will be much easier to be trained with high beta since network would be penalized less with blurred boundary

3. Would you be more likely to learn an accurate surface with high beta or low beta? Why?

Low beta. The variance of the boundary will be much smaller.

A high value is going to make the SDF "blurrier" by smoothing the transition between object boundaries. A low means that the density converges to a "scaled indicator function" of , which basically means that the exponent gets large for distances outside the object, and approaches for anything inside the object.

Comment on settings

I set the alpha=10 and beta=0.05 and then hidden neurons fro distance and color are both 256, which is doubled. So I can get a powerful model. The default values for alpha and beta work well so I didn't change. Model architecutre is as follows.

Model Architecture

Model(
  (implicit_fn): NeuralSurface(
    (harmonic_embedding_xyz): HarmonicEmbedding()
    (module): MLPWithInputSkips(
      (mlp): ModuleList(
        (0): Sequential(
          (0): Linear(in_features=39, out_features=256, bias=True)
          (1): ReLU(inplace=True)
        )
        (1): Sequential(
          (0): Linear(in_features=256, out_features=256, bias=True)
          (1): ReLU(inplace=True)
        )
        (2): Sequential(
          (0): Linear(in_features=256, out_features=256, bias=True)
          (1): ReLU(inplace=True)
        )
        (3): Sequential(
          (0): Linear(in_features=256, out_features=256, bias=True)
          (1): ReLU(inplace=True)
        )
        (4): Sequential(
          (0): Linear(in_features=256, out_features=256, bias=True)
          (1): ReLU(inplace=True)
        )
        (5): Sequential(
          (0): Linear(in_features=256, out_features=256, bias=True)
          (1): ReLU(inplace=True)
        )
      )
    )
    (distance_layer): Sequential(
      (0): Linear(in_features=256, out_features=256, bias=True)
      (1): ReLU(inplace=True)
      (2): Linear(in_features=256, out_features=1, bias=True)
    )
    (color_layer): Sequential(
      (0): Linear(in_features=256, out_features=256, bias=True)
      (1): ReLU(inplace=True)
      (2): Linear(in_features=256, out_features=3, bias=True)
      (3): Sigmoid()
    )
  )
  (sampler): StratifiedRaysampler()
  (renderer): VolumeSDFRenderer()
)

Surface

Geometry

4. Neural Surface Extras

4.2 Fewer Training Views

Below are the results trained by NeRF and VolSDF about 100 epoches. As we can see here, the result is pretty good except there are some details blurred. This is becasue we provide fewer views and details are lost.

NeRF

VolSDF

4.3 Alternate SDF to Density Conversions

Alternative Conversion from NeuS

The equation for density:

$ φ_{s}(x) = se^{−sx}/(1 + e^{−sx})^2 $, where $x$ is the SDF

$s$ is a hyperparameter, which named scale in configuration files. Also, for reuse the main code, set $is_NeuS$ to indicate whether to use alternative conversion.

Ablation Study on scale

The value of scale will signficantly effect the result. Here are the result when each is trained for 40 epoches.

Scale=1

Scale=3

Scale=5