|
| Torus with Sphere Tracing |
To implement Sphere Tracing, I use keep track of offsets returned by the implicit function. I initialize the initial offset to the near variable. I iterate over the number of steps and at each step I compute the 3D point based on origin, direction and offset. I also compute the signed distance from the impicit function and update the offset. I limit max value of offset to far variable. Forground mask is computed for all 3D points whose signed distance is less than a threshold (1e-4).
 |
 |
| Input Points | Learnt implicit surface |
For this implementation, I made the following changes to hyper-parameters: * I increased the input dimension to 256. * I increased the number of layers for main feature extractor to 8. * I append the original xyz at layer 4
For exact specification, please refer to 'points.yaml'
Parameter alpha: This parameter controls the magnitude of density for an object. Higher value of alpha indicates that density value will be large as the ray intersects a surface.
Parameter beta: This parameter controls the magnitude of change in density. Lower value of beta indicates that density will change more sharply near the surface and higher beta will indicate that change in density is more gentle.
For these experiments, I use alpha as 100 and beta as 0.01.
Ideally, an opaque object would have very low beta and correspondingly a high alpha. The VolSDF paper empirically suggests alpha value to be inverse of beta.
A high value of beta can make the learnt SDF to be smoother around the boundary and may be less sensitive to distance-change. Hence the result can be blurry. A low value of beta will make the SDF to learn sharper boundaries as lower beta is closer to true 0-1 indicator.
I suspect it will be easier to learn the SDF with slightly higher beta. I believe this is because during the initial stages distances etc. can be noisy and a higher value will indicate gradients for a larger range of distance close to boundary. However, there needs to be trade-off as very high or very low values can both impact learning of SDF.
You are more likely to learn an accurate surface with a lower beta. This is because the lower the value is, the more sharp is the change in density as the ray intersects a surface. This in turn ensures that colors are not aggregated over a larger domain and are a property of the surface. This also suggests that just before the surface density is low and hence the surface is more precise.
 |
 |
 |
| Alpha 100, Beta 1. | Alpha 100, Beta 0.1 | Alpha 100, Beta 0.01 |
 |
 |
 |
| Alpha 1, Beta 0.01. | Alpha 10, Beta 0.01 | Alpha 100, Beta 0.01 |
Here I perform a visualization, wherein I plot 100 points between [-1., 1] and plot the sdf_to_density() function and analyse the impact of alpha and beta values.
 |
 |
| Geometry | Learnt Color |
For this implementation, I made the following changes to hyper-parameters: * I increased the input dimension to 256. * I increased the number of layers for main feature extractor to 8. * I append the original xyz at layers 2, 4 * I used the Kaiming initialization for the linear layers (as activation was ReLU)
 |
| Visualization composed of around 25 primitives |
In this case, I visualize 6 torus, 4 cubes and 15 spheres. I render them using Sphere Tracing Algorithm. To re-perform the visualization load the config file many.yaml.
| 5 Views |  |
 |
 |
| 10 Views |  |
 |
 |
| 20 Views |  |
 |
 |
| 50 Views |  |
 |
 |
| Number of Views | SDF Geometry | SDF Color | NeRF |
It is interesting that with only 5 views, learning geometry enables us to model the lego atleast crudely, while NeRF fails terribly. However from 20 and 50 views, we can see that NeRF rendering has better colors. To enable this behavior set the train_img_size behavior in volsdf.yaml to 5 (or any value between [1, 100]).
In this case, I use the NeUS based conversion from SDF to density. The parameter s is set to 100. To enable this behavior set the convert_method behavior in volsdf.yaml to NeUS (or VolSDF).
 |
 |
| Geometry | Learnt Color |
It is important to note that a lot of structures are missing even when the rendering looks okay.