The goal of this assignment is to take the digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce a color image with as few visual artifacts as possible. In order to do this, we extract the three color channel images, place them on top of each other, and align them so that they form a single RGB color image. We implemented two approaches that achieves our goal. In approach one, we exhaustively search over a window of possible displacements of [-15,15] pixels, score each one using SSD distance, and take the displacement with the best score. We run this algorithm on cathedral.jpg, and obtained the following result (for all of the images we displayed, ar offset means the offset between red channel and blue channel, while ag offset means the offset between green channel and blue channel):
In approach two, we build a image pyramid to speed up the alignment process. I set the pyramid layer to 4, the initial search range to [-20, 20], and the search range of following layers to [-4, 4]. This greatly speed up the alignment, as it only takes about 10 second per image. The results are shown below:
On most of the images, our algorithm works decently. However, on emir.tif and train.tif, our algorithm could definitely improve. I think the reason might be that when we are building our image pyramid, one of the important hyperparameter is how many times of down-scaling we should do. If this number is too small, we won't get much speed up; if it's too big, our low resolution alignment may be incorrect since the image is so blurred. It might be that I have down-scaled the image too many times. Another potential reason is that SSD might not be a good enough metric for measuring the alignment score across different channels, as they do not actually have the same brightness values. We might need some better metric to handle that.
As for a few examples of my own choosing, downloaded from the Prokudin-Gorskii collection, I presented two examples: one that aligned well and one that doesn't align that well:
