Given the blue, red, and green images, I use multiscale image pyramid matching to align green to blue, then red to blue. Finally, I stack with the original blue channel image to produce the colorized output.
I define an image pyramid class (in image_pyramid.py) which with a default downscale factor of 2 for each new level. Depending on whether the input is a .jpg or .tif file, I set the pyramid size to 1 (single scale) or 3 (multi scale). I create both an image and template pyramid, so that both the image and template are downsampled. I start matching at level n-1 (the coarsest resolution) which outputs a ROI (region of interest). I then recursively call the image pyramid matching on the next level, bounding the search area to the outputted ROI each time. This makes searching much more efficient across large images.
I decided to use Normalized Cross Correlation over SSD to find the corresponding image patch that matches the template, since I found it to be more robust to differences in brightness/lighting intensities.
Cathedral.jpg [G to B: (5,2), R to B: (12, 3)]
Emir.tif [G to B: (47, 23), R to B: (79, 43)]
Harvesters.tif [G to B: (61, 18), R to B: (124, 14)]
Icon.tif [G to B: (44, 18), R to B: (90,23)]
Lady.tif [G to B: (56, 8), R to B: (119, 12)]
Self_portrait.tif [G to B: (80, 30), R to B: (176, 37)]
Three_generations.tif [G to B: (54, 16), R to B: (109, 13)]
Train.tif [G to B: (47, 7), R to B: (87, 33)]
Turkmen.tif [G to B: (57, 22), R to B: (117, 29)]
Village.tif [G to B: (66, 14), R to B: (137, 23)]