16-889 Assignment 1: Rendering Basics with PyTorch3D

1. Practicing with Cameras

1.1 360-degree Renders (5 points)

1.2 Re-creating the Dolly Zoom (10 points)

2. Practicing with Meshes

2.1 Constructing a Tetrahedron (5 points)

Vertices: 4 vertices

np.array([[-1, -1, -1],[1, 1, -1],[-1, 1, 1],[1, -1, 1],])

Faces: 4 faces

np.array([[0, 1, 2], [0, 2, 3], [0, 1, 3], [1, 2, 3]])

For better visualization, I rotate the vertices such that the base have the same y axis

2.2 Constructing a Cube (5 points)

Vertices: 8 vertices

np.array(    [[-1, -1, -1],[-1, -1, 1],[1, -1, 1],[1, -1, -1],
						 [-1, 1, -1],[-1, 1, 1],[1, 1, 1],[1, 1, -1],])

Faces: 12 faces

np.array([[0, 1, 2],[2, 3, 0],[0, 1, 5],[5, 4, 0],[0, 4, 3],[3, 4, 7],
          [1, 5, 2],[2, 5, 6],[3, 2, 6],[3, 6, 7],[4, 5, 6],[4, 6, 7]])

3. Re-texturing a mesh (10 points)

Two colors:

tensor([0.0000, 0.4472, 0.8944]) tensor([1.0000, 0.5528, 0.1056])

4. Camera Transformations (20 points)

R_relative = [[ 0.  1.  0.]
						  [-1.  0.  0.]
							[ 0.  0.  1.]]
T_relative = [0, 0, 0]

R_relative = [[1. 0. 0.]
							[0. 1. 0.]
						  [0. 0. 1.]]
T_relative = [0.5, -0.5, 0]

R_relative = [[1. 0. 0.]
							[0. 1. 0.]
						  [0. 0. 1.]]
T_relative = [0, 0, 2]

R_relative = [[ 0.  0.  1.]
							[ 0.  1.  0.]
							[-1.  0.  0.]]
T_relative = [-3, 0, 3]

5. Rendering Generic 3D Representations

5.1 Rendering Point Clouds from RGB-D Images (10 points)

Point Cloud 1

Point Cloud 2

Point Cloud 1 + 2

5.2 Parametric Functions (10 points)

5.3 Implicit Surfaces (15 points)

Rendered as mesh

Rendered as point cloud

Discussion:

Rendering as a mesh:

Speed: 1.72s

Rendering the same torus (same number of vertices), we can see that rendering mesh takes longer than point cloud. This makes sense as rendering meshes are more involved than point clouds due to its data structure and rasterizing process. However, for more complex 3D objects though, meshes can be potentially more memory efficient since one can optimize faces and vertices to reduce the number of vertices used. With the same number of vertices, meshes would also be of high quality since the rendered object will be closed and more refined.

Rendering as point cloud:

Speed: 0.71s

Rendering point cloud is easier since it only involves drawing independent colored points. However, its memory usage will be higher once the number of points grew, especially for more complex 3D objects. The quality is also worse since one will need a lot of points to represent similar objects (e.g. boxes) than meshes.

6. Do Something Fun (10 points)

Rendering of the SMPL human body following this tutorial using the Densepose body mesh texture mapping and human motion sequences from the H3.6M dataset.

(Extra Credit) 7. Sampling Points on Meshes (10 points)

Samples = 10

Samples = 100

Samples = 1000

Samples = 10000

If rendering vertices directly: