Assignment 2 : Learning Based Image Synthesis

Gradient Domain Fusion

By: Sanil Pande

1. Description

This project involves using the gradient domain fusion method as compared to a naive method of directly blending the images. Human perception is more sensitive to changes in intensities of images as compared to the actual intensities of the pixels in the image. To blend two images, the source and the target, we can make the gradients in the source image similar to the gradients in the target image, and also constrain the edge pixels to be the same value as the target image. This ensures that there are no sudden changes in the boundary, and that the source image retains its structure when blended onto the target image.

We can formulate our objective as a least squares problem. Given the pixel intensities of the source image “s” and of the target image “t”, we want to solve for new intensity values “v” within the source region “S”:

Here, each “i” is a pixel in the source region “S”, and each “j” is a 4-neighbor of “i”. Each summation guides the gradient values to match those of the source region. In the first summation, the gradient is over two variable pixels; in the second, one pixel is variable and one is in the fixed target region.

The method presented above is called “Poisson blending”.

2. Toy Problem

The results of this method on a toy problem of blending an image onto itself is shown below.

Left: Naive Blending         Right: Poisson Blending

3. Poisson Blending

The results of Poisson Blending are shown in the figures below. To get this to work, I modified the algorithm mentioned above to work with 3 channel images. Also, the constraint that the modified source values at the edge should be the same as the ones at the target is implemented only in the horizontal pass.

Source Image                                        Target Image

Results:

The results of the naive blending vs the Poisson Blending are shown below. We can see that the Poisson method works much better.

4. Other Examples

Target                         Source                         Mask

Failure Case:

The following blending is one of the failure cases I experienced. I think that it is because the texture of the grass overpowers the texture in the source image. Also, this might show that the gradient domain fusion method fails when the source image is from a completely different background source than the target image.

Target                         Mask                         Source

Result: