By: Maneesh Bilalpur
Sergei Mikhailovich Prokudin-Gorskii (1863-1944) [Сергей Михайлович Прокудин-Горский, to his Russian friends] was a man well ahead of his time. Convinced, as early as 1907, that color photography was the wave of the future, he won Tzar’s special permission to travel across the vast Russian Empire and take color photographs of everything he saw including the only color portrait of Leo Tolstoy. And he really photographed everything: people, buildings, landscapes, railroads, bridges… thousands of color pictures! His idea was simple: record three exposures of every scene onto a glass plate using a red, a green, and a blue filter. Never mind that there was no way to print color photographs until much later – he envisioned special projectors to be installed in “multimedia” classrooms all across Russia where the children would be able to learn about their vast country. Alas, his plans never materialized: he left Russia in 1918, right after the revolution, never to return again. Luckily, his RGB glass plate negatives, capturing the last years of the Russian Empire, survived and were purchased in 1948 by the Library of Congress. The LoC has recently digitized the negatives and made them available on-line.
In this work we align the plates of the camera used by Sergei. We present some sample images and try image processing methods to evaluate how good they perform in the template matching task. One of the input images is displayed below.
A simple solution to the problem is to simply overlay the R, G, B plates to form an RGB image. The following is an example of such case.
Exhaustive search is expensive for obvious reasons. To remedy this, we use a patch based method. The method works well if the patch corresponds to highly informative portion of the image. Choosing a uniform portion of the image as patch would results in poor outcomes. We also restrict the search range to a limited area over the entire image as most images have a small misalignment when compared to the size of the image. The results of patch based method could be seen below
Gradient alignment uses magnitude of image gradients to match the template and reference images. The gradients are calculated using the Sobel filters. The following image shows gradient images for the red plate for the cathedral image. The gradient images are relatively expensive over patch alignment so we use the gradients over patch for alignment.
The methods fails for self_portrait.jpg(simple overlay shown left) images. We find that the gradient based method(shown right) does better over the patch method(shown middle) on such clusters of finegrain and similar faetures. However, the noticable misalignment in the gradient based methods when compared to rest of the input images is a concern. We believe that this could be largely due to the clustered similar features(stones and grass) in the image. These could be solved using feature based matching such as SIFT feature matching
A common element of pitfall with all methods described include the border issue. One possible solution could be padding the offset using a replication of adjacent pixels in the image.
For all experiments the search range is limited to (-75, 25), (-75, 25) for green and blue channels respectively. All experiments were run on the dual core Google Colab environment.