Max Grebinskiy's Home Page for 16-726 Project 2

Project Overview

For this project, we explore the use of Poisson blending in order to blend an object from a source image into a target image, regardless of the original intensities of each individual image. In order to handle potentially noticeable seams between the source object and target image, we aim to preserve the gradient of the source object when finding the values of the target pixels without altering the background.

Toy Problem

In this toy problem, we take a simplified approach to our main algorithm of Poisson blending. Rather than looking at each nearby pixel in our neighborhood, we will instead only look at the pixel to our right and our bottom. This still maintains the horizontal and vertical gradients for each pixel in our source image. In order to tether the intensity of the source image to the target image, we will also match the top left corner of each image to be the same color. By using this simplified version of Poisson blending, we are able to recover the original image of Woody and Buzz Lightyear.
Toy Problem Result

More specifically, we aim to minimize the following three types of equations:
Minimizing Formulas

where x,y range over all possible pixel locations in the image. To solve this, we represent the system of equations to minimize as a sparse matrix using scipy's csr_matrix. We then use scipy's sparse least squares solver, scipy.sparse.linalg.lsqr, in order to produce an output for each pixel that minimizes the total sum of each constraint.

Poisson Blend

In this section, we follow a similar approach to the toy problem, except we add constraints for each adjacent pixel in the xy-direction. In essence, this forms 4 constraints per pixel in our source object. There is, however, a small adjustment; specifically, for pixels in our source object that are adjacent to our target image. As we do not want to adjust any pixels in our background, we do not treat any background pixels as variables, but instead as fixed values. This leads to us using the following constraint formula:
Poisson Equation

Results with Poisson Blending

Here are some results using Poisson Blending. This first example was inspired by seeing the results of the recent volcanic eruptions on Iceland while I was flying over the land. The Poisson blend seemed to work fairly well on the perimeter of the dragon, but seems to mess up on the inner portions of the dragon (particularly between the fire breathing and dragon tail).

This second example is King Kong invading a plaza, with people seeming unphased by it.

This third example is trying out some new sunglasses on my friend, David. For this one, I had to be particular about cropping out the sunglasses, as the skin tone did not blend as well as I would have hoped.

This next example is a situation in which Poisson Blending failed. You can notice that the border around my friend is extremely blurry, and most of his figure becomes transparent in the Poisson blend. A possible reason as to why this may have failed is due to the lack of contrast in the gradient; the overall gradient of my friend in the source object generally remains fixed (as the entire object is dark), and so his gradient may get overpowered by the gradients of the background image. This is especially evident in the mixed-blend version, which I will show below. Further, even though I attempted to match the overall pixel intensities by picking a target image that is relatively darker / in a nighttime setting, the pixel values in the image with my friend are still substantially darker compared to the background.

Bells & Whistles

One extra functionality I implemented was the mixed blend. This is similar to Poisson blending, except it uses the stronger gradient of the two in the source/target. In general, it produced the same results as Poisson blending, but in the ice lake example, it produces significantly worse results, as he practically phases out of existence. Gil In Ice Lake with Mixed Blend