Assignment 3
Adnan Ahmad
Late days used :
2
1.3
1.4
1.5
2.1
I’ve
implemented the get_random_pixels_from_image function
in codebase
2.2
Box Center : (0.2502,
0.2506, -0.0004)
Box side lengths :
(2.005, 1.5035, 1.5033)
2.3
3 NeRF
4.1 View
Dependence
Adding view dependence lets Nerf account for view specific effects like reflections. While its not clearly visible in this example it was very evident in the ship model visualized in the original Nerf paper. It showed reflections that were visible in water. However, adding view dependence also increases the chance of overfitting on the ground truth data where secularities are prominent. Thus, generalizability tends to decrease as view dependence increases.
4.3 HighRes NeRF
n_pts_per_ray = 20 n_pts_per_ray = 60 n_pts_per_ray = 128
For this experiment, I varied the n_pts_per_ray parameter. I started with a value of 20 and gradually increase it to 128. I visualized results from 3 values – 20, 60, 128. From the above results it can be concluded that increasing the number of points sampled per ray increases the quality of the reconstruction. This is to be expected as more points allow for greater detail to be processed during the rendering process in effect making the output look more ‘high-res’ with increased sharpness.
NeRF Model Used
class NeuralRadianceField(torch.nn.Module):
def __init__(
self,
cfg,
):
super().__init__()
self.harmonic_embedding_xyz = HarmonicEmbedding(3, cfg.n_harmonic_functions_xyz)
self.harmonic_embedding_dir = HarmonicEmbedding(3, cfg.n_harmonic_functions_dir)
embedding_dim_xyz = self.harmonic_embedding_xyz.output_dim
embedding_dim_dir = self.harmonic_embedding_dir.output_dim
# embedding_dim_xyz
= 3
self.top5like = torch.nn.Sequential(
torch.nn.Linear(embedding_dim_xyz,128),
torch.nn.ReLU(),
torch.nn.Linear(128,128),
torch.nn.ReLU(),
torch.nn.Linear(128,128)
)
self.feat_layer = torch.nn.Sequential(
torch.nn.Linear(128+embedding_dim_xyz,128),
torch.nn.ReLU(),
torch.nn.Linear(128,128),
)
self.density_layer = torch.nn.Linear(128,1) ## apply relu
self.last_layers = torch.nn.Sequential(
torch.nn.Linear(128+embedding_dim_dir,128),
torch.nn.ReLU(),
torch.nn.Linear(128,3),
torch.nn.Sigmoid()
)
def forward(self, ray_bundle):
direc = ray_bundle.directions.unsqueeze(1).repeat(1,ray_bundle.sample_points.shape[1],1)
x = self.harmonic_embedding_xyz(ray_bundle.sample_points.reshape(-1,3))
d = self.harmonic_embedding_dir(direc.reshape(-1,3))
# x = ray_bundle.sample_points.reshape(-1,3)
z = self.top5like(x)
z = torch.cat((z,x),dim=1)
z = self.feat_layer(z)
density = F.relu(self.density_layer(z))
z = torch.cat((z,d),dim=1) ### Adding view dependence
feats = self.last_layers(z)
out = {
'density': density,
'feature': feats
}
return out