16-889: Learning for 3D Vision

HW 4

Tanay Sharma(tanays)



Late Days Used : 1

1. Sphere Tracing

Implementation:
  1. Start with the origins as initial points
  2. Pass the points to the implicit function to get nearest distances to the surface for each point
  3. Update the points using the formula : origins + dist * directions
  4. Update points to these new points and repeat from 2 till max iterations
  5. Finally, create a mask based on distances that are zero (surface) after max iterations

2. Optimizing a Neural SDF

Input Point Cloud
Prediction

MLP Details:
  1. Harmoninc Embedding of input points with output dimensions 4
  2. 6 Layer MLP with ReLU activation + 1 output layer (no activation)
  3. Hyperparamerers : [epochs = 10000, eikonal_weight=0.04, hidden_neurons=200, n_layers=6]
Eikonal Loss:

The norm of the gradients should be equal to 1 and hence, loss should penalize deviation from 1. It can be implemented as :

Loss = || norm(gradients) - 1 ||^2

3. VoISDF

Geometry Prediction
Prediction
Intuition:

With reference to the paper, alpha can be intuitively understood as modelling a constant density while beta is a smoothness parameter that decreases the density smoothly near the boundaries.
  1. How does high beta bias your learned SDF? What about low beta?
    2. High beta increases the variance in the densities which can lead to blurrier/smoother results. Low beta reduces the variance and eventually converts the density to a scaled indicator function when it approaches 0.
  2. Would an SDF be easier to train with volume rendering and low beta or high beta? Why?
    3. It would be easier to train with slightly higher beta as it may help in generalization and smoother gradients while low beta can lead to overfitting. We need to balance as there is a tradeoff.
  3. Would you be more likely to learn an accurate surface with high beta or low beta? Why?
    4. Low beta can help to get a better surface as there will be very low variance near the boundaries and hence can have sharper results.


    Regarding the hyperparameters, I increased the number of neurons to 200 to increase the representational capacity of the network. I reduced number of epochs to 50 to avoid overfitting. I also changed the lr_scheduling rate to every 20 steps for better convergence in 50 epochs. The default alpha and beta values worked for me well so I kept it the same.

4.2 Fewer Training Views

VoISDF on fewer training views (20)




Comparison of VoISDF with NeRF
VoISDF
NeRF

The output of NeRF seems to be sharper than the VoISDF with less number of views. NeRF seems to be better for details while VoISDF generalizes well with less number of views.