16-726 HW2 Gradient Domain Fusion¶
Overview¶
This project aims to use gradient domain fusion processing techniques for image blending. Specifically, we will extract a patch from a source image and try to blend it into the target image using Poisson Blending. We will start with a toy example that aims to reconstruct an image. This is to ensure that the implementation for gradient calculation is correct. Then we will move to the actual implementation of Poisson Blending¶
Toy Problem¶
import matplotlib.pyplot as plt
import cv2
img = cv2.imread('./toy_result.png')
plt.figure(figsize=(10,10))
plt.imshow(img)
Favorite Result¶
Method¶
We first use a helper function to select a region in the source image and a position in the target image to blend the source region into. The helper function will generate a mask and a new source image that are aligned with the target image. Then we will try to find new pixel values for the source region by minimizing the Poisson Blending constraints below
where "s" is the pixel intensity of the source
image and "t" is the pixel intensity of the target image. Each "i" is a pixel in the source region "S", and each "j" is a 4-neighbor of "i". Once we have the pixel values for the new source region, we can paste them into the target image and get a blended image.
source and target image¶
plt.figure(figsize=(10,10))
plt.subplot(131)
img = cv2.cvtColor(cv2.imread('./demo_data/source_02_newsource.jpg'),cv2.COLOR_BGR2RGB)
plt.imshow(img)
plt.title('source')
plt.subplot(132)
img = cv2.cvtColor(cv2.imread('./demo_data/target_02_mask.jpg'),cv2.COLOR_BGR2RGB)
plt.imshow(img)
plt.title('mask')
plt.subplot(133)
img = cv2.cvtColor(cv2.imread('./demo_data/target_02.jpg'),cv2.COLOR_BGR2RGB)
plt.imshow(img)
plt.title('target')
Blending Result¶
img = cv2.cvtColor(cv2.imread('./gbot_blend.png'), cv2.COLOR_BGR2RGB)
plt.figure(figsize=(10,10))
plt.imshow(img)
More Results - 1¶
In this example, we can see that there are some artifacts on the right side of the blended face: there is a dark area. The reason is that the source face is a little bit larger than the target's original face, which makes the border of the source face have some overlap with the hair in the target image. Since Poisson Blending will only preserve the gradient inside the mas and the pixels in the target image that are outside the mask will not change, the color of hair will blend into the face.
source and target¶
plt.figure(figsize=(10,10))
plt.subplot(131)
img = cv2.cvtColor(cv2.imread('./demo_data/trump_newsource.jpg'),cv2.COLOR_BGR2RGB)
plt.imshow(img)
plt.title('source')
plt.subplot(132)
img = cv2.cvtColor(cv2.imread('./demo_data/mona_mask.jpg'),cv2.COLOR_BGR2RGB)
plt.imshow(img)
plt.title('mask')
plt.subplot(133)
img = cv2.cvtColor(cv2.imread('./demo_data/mona.jpg'),cv2.COLOR_BGR2RGB)
plt.imshow(img)
plt.title('target')
Blending Result¶
img = cv2.cvtColor(cv2.imread('./trump_blend.png'), cv2.COLOR_BGR2RGB)
plt.figure(figsize=(10,10))
plt.imshow(img)
More Results - 2¶
This example works decently well, though the color of penguin changed to yellow. This is because Possion Blending tries to preserve the gradient instead of pixel intensity.
source and target¶
plt.figure(figsize=(10,10))
plt.subplot(131)
img = cv2.cvtColor(cv2.imread('./demo_data/penguin_newsource.jpg'),cv2.COLOR_BGR2RGB)
plt.imshow(img)
plt.title('source')
plt.subplot(132)
img = cv2.cvtColor(cv2.imread('./demo_data/desert_mask.jpg'),cv2.COLOR_BGR2RGB)
plt.imshow(img)
plt.title('mask')
plt.subplot(133)
img = cv2.cvtColor(cv2.imread('./demo_data/desert.jpg'),cv2.COLOR_BGR2RGB)
plt.imshow(img)
plt.title('target')
Blending Result¶
img = cv2.cvtColor(cv2.imread('./penguin_blend.png'), cv2.COLOR_BGR2RGB)
plt.figure(figsize=(10,10))
plt.imshow(img)
