Assignment #1 - Colorizing the Prokudin-Gorskii Photo Collection

Emily Kim

This assignment seeks to align the RGB channels from the Prokudin-Gorskii glass plates.

Single-scale Alignment

This method seeks to align the RGB channels using exhaustive iterations of offsetting the image by a desired range. An example input image looks like the following image below. In order to obtain each of the RGB channels, divide each of the input images in to thirds by the height, and each of the images can be used as R, G, B channels respectively. The R and G channels are offset to align with the B channel.

cathedral image
Fig 1. Example input image
The goal is to minimize the offset between the three images, by calculating the x, y offsets that minimize the following scores.
  1. Sum of Squared Differences (Norm): Calculate the difference between the two images and sum the squared differences to calculate the score.
  2. Normalized crossed-correlation: Negative dot product between the two images
I also used the Sobel filter to calculate the gradients in directions x and y to more accurately calculate the scores.

Image Pyramid

To align the images more efficiently, the images were rescaled to be of smaller dimensions to make updates from the coarsest scale. For higher dimensionality, this should be able to speed up the process as while checking at larger offsets. The coarest level image is 1/8 of the actual image, and for each level the images are rescaled two times the previous scale.

image_pyramid
Fig 2. This figure illustrates what an image pyramid is like

Resulting image

The following are the results from the single scale alignment method from a small jpg file.

File Name SSD_single NCC_single
cathedral.jpg norm_cathedral ncc_cathedral

Image pyramid results

For the larger tif files, I used image pyramid method. In addition to using the two different scoring schemes, I also used the Sobel filter before the adjustment to compare the performance. Note that emir.tif actually performs better with edge (Sobel) detection method whereas for for self-portrait, non-edge detection method (non-Sobel) aligns better.

File Name SDD NCC SDD + Sobel NCC + Sobel
emir.tif norm_emir NCC_emir norm+sobel_emir norm+sobel_emir
harversters.tif norm_harvesters NCC_harvesters norm+sobel_harvesters norm+sobel_harvesters
icon.tif norm_icon NCC_icon norm+sobel_icon norm+sobel_icon
lady.tif norm_lady NCC_lady norm+sobel_lady norm+sobel_lady
self_portrait.tif norm_self_portrait NCC_self_portrait norm+sobel_self_portrait norm+sobel_self_portrait
three_generations.tif norm_three_generations NCC_three_generations norm+sobel_three_generations norm+sobel_three_generations
train.tif norm_train NCC_train norm+sobel_train norm+sobel_train
turkmen.tif norm_turkmen NCC_turkmen norm+sobel_turkmen norm+sobel_turkmen
village.tif norm_village NCC_village norm+sobel_village norm+sobel_village

Cropping

I also added an automatic cropping feature to my code to crop the images. This feature also uses the Sobel filter to do the edge detection. In this function, RGB filtered values are summed and thresholded to find the row and column values for cropping at right, left, top and bottom edges.

File Name Before After Cropping
emir.tif emir_NCC_sobel cropped_emir
cathedral.jpg cathedral_NCC_sobel cropped_cathedral
harvesters.tif harvesters_NCC_sobel cropped_harvesters
icon.tif icon_NCC_sobel cropped_icon
lady.tif lady_NCC_sobel cropped_lady
self_portrait.tif self_portrait_NCC_sobel cropped_self_portrait
three_generations.tif three_generations_NCC_sobel cropped_three_generations
train.tif train_NCC_sobel cropped_train
turkmen.tif turkmen_NCC_sobel cropped_turkmen
village.tif village_NCC_sobel cropped_village