No colaboration
Zero late days used
No colaboration
Below is the rendered result correctly showing the torus.
For my sphere tracing implementation I introduce two additional arrays. The first is a boolean mask, the size of the number of samples, which is initialized to all zeros. The second is an array of indices, counting up from zero to the number of samples. In each iteration I compute the SDF for each point. I check the points which are too far and remove them from the set of points and remove the corresponding indices. Then I check for points that are within a threshold distance of the surface. I use the array of indices to mark the corresponding locations in the mask as true. Then, I conduct analagous operations to before where I remove these points from the array of points and remove the corresponding indices. Then I update each point in the direction of the array by the value of the SDF. This process repeats until no points remain.
I the same feature extraction network that was used for the NeRF assignment, where a positional embedding was processed with fully connected layers and ReLU acitivations. Halfway through, the positional embedding was concatenated with the features. After the backbone, which was later shared with the color prediction task, I used two layers to predict the SDF. The SDF values are unbounded so I did not apply a nonlinearity after the last layer.
When I used the default parameters I found that it appeared that my mesh was becoming hollow, where the interior region was only a thing layer near the sampled points. There were a few holes in this surface where the structure was not captured at all. Therefore, I chose to enforce the Eikonal loss not only at randomly sampled points but also at the surface. This fixed my issue because now the network was peanalized for reversing the direction of the gradient in that direction. TODO see if I upated the Eikonal loss term.
This produced a result I was quite happy with. I believe that artifacts on the Rabbit's ear a caused by the marching cubes algorithm and not a the underlying representation.
To predict the color, I used two layers after the backbone that predicted a three-channel output with Sigmoid to enforce that all values were in the range \((0, 1)\).
beta term controls the falloff of the density function. The alpha term controls the amplitude of the volume density. The density asymptotically approaches this value for low SDF values.beta value spreads out the density function around the zero crossing. Because this density is integrated by the volume rendering proceedure, an exessively high beta will make the predicted volume opaque before the actual surface. To compensate, the SDF will have to be predicted as a lower value than it actually should be. Conversely, a very low beta value will effectively cause a step function in the density function. Therefore, the SDF will be predicted as a higher value so that some of the density occurs before the actual surface of the object. Therefore, the contribution to the volume rendering will be centered around the true surface.beta value spreads out the density function around the zero crossing of the SDF. Since gradients are only propagated from poritions of the volume that are used in rendering, this allows more regions of the SDF to be updated, assuming the alpha value is sufficiently-low to avoid only rendering from the first few observations along the ray.beta value begins to approximate a step function in the density function. With a sufficiently-high alpha value, this will be like actually rendering a single surface since the integrated volume will become opaque right at the zero crossing of the SDF. This allows the SDF surface to become precisely fit to the true data.Below is the result using beta=0.125 and 100 epochs, with all other paramters left the same. If the beta value was lower, then the object seemed to colapse to a singular point. With a higher value, the training seemed to diverge.
I did this by creating an aggregate scene and then taking the minimum SDF value over all of the objects in the scene. The result of 25 spheres in a grid can be seen below.
part 1
I initially tried running NeRF with 20 images and the result were still good as seen below. This is likely because there is enough data to fit this more expressive representation.
part 1
Then I tried with 5 images and the SDF approach (left) slightly outperformed NeRF (right), as expected. Specifically, NeRF is very blurry while the SDF method approximates a surface well.
I implemented two different functions. The first was the naive approach suggested in the NeUS paper. This led to results which were very blurry and had geometry which was too large. I suspect this was because I didn't normalize the distribution so it predicted it farther out so the mean contribution would be at the surface.
The second was inspired by the truncated signed distance function (TSDF) which is commonly used in reconstuction tasks. In this fomulation, \(\sigma(s) = \alpha (\min(\max((\frac{-s}{\beta}), -1), 1) + 1)/2\). This is simply a ramp which starts at a value of 0 for an SDF value of \(\beta\) and ends at \(\alpha\) for an SDF value of \(-\beta\). This formulation produced surprisingly good results, which were actually better than the initial experiments with \(\beta=0.25\). I think this could simply be getting lucky since the two functions are fairly similar, or that this simply created a shorter falloff due to the different meaning of the \(\beta\) term.
