Gradient Domain Fusion

Sudeep Dasari

Andrew ID: sdasari

---

Overview and Approach

Blending images requires copying pixels from a source image on top of a target image, and then adjusting the final creation so it looks "natural" to the human eye. Thje final result can look very impressive - like the hiking penguins shown in class.

Gradient domain processing accomplishes blending by adjusting the gradients of a target image to match that of the source image. The key observation is that human eyes are sensitive to contrast, so naive foreground masking results in awkward seams due to rapid change in pixel values. Instead Poisson Blending matches gradients while keeping boarder pixel values roughly the same, thus preserving image content without creating an ugly seam. This can be formulated using the following optimization problem:

This optimization can be solved efficiently using a least squares solver. Since we're operating on pixels the final matrices can be very large. Thankfully they are also sparse (each pixel has only 8 neighbors), so a sparse least-square solver can be used to greatly speed up computation. My code is able to solve such problems involving matrices with tens of thousands of rows/columns in seconds.

Sanity Check: Toy Problem

First, I run a very simple sanity check to verify my code. I use x/y gradient constraints and a source pixel in top left corner to reconstruct a toy image. To be clear, the solver should be able to exactly recover the original image, since the system of equations (gradient constraints, and top left pixel) has a single unique solution. The final result is shown below - it's a match!

Poisson Blending

I now attempt to run my full Poisson Blending code on a mix of my own images and provided demo images. First, I show the bear blending into the swimming pool. When using poisson blending the image values are a great match, but it still looks a little off due to the physical behavior of the water! Alas, blending can only correct visual attributes: correcting physics is a very different ballgame.

Now let's look at some of my own examples. I start by blending a Wampa from Star Wars into a picture of a Dog Sled in Alaska. I sure hope they get away in time! Note that Poisson blending makes a big difference here - thanks to copying gradients there is no massive seam caused by contrast difference.

Next I recreate the Annoying Orange meme by blending a picture of a mouth onto an orange. Note that the Poisson Blended result looks a lot like the meme (they must've done this too), whereas the other looks like a low effort copy paste job. Regardless, the final result is annoying.

Finally, I try to copy a picture of a supercar onto a New Mexican backroad. This is a notable failure case for Poisson Blending! The edge of the car looks overly blended into the road, since the algorithm tries to match the natural asphalt color. Here is a case where simple (copying) is actually better.

---

Website template graciously stolen from here