Gradient Domain Fusion

In assignment 2, we aim to explore the wide range of applications of gradient-domain fusion, a simple technique. Specifically, we will focus on "Poisson blending" to seamlessly blend an object or texture from a source image into a target image. While copying and pasting pixels directly may seem easy, it often results in noticeable seams, even when the backgrounds match. Poisson blending utilizes the gradients of both the source and target images to create a seamless blend. However, it is essential to note that this technique may cause a change in colour since it ignores overall intensity. Our focus is on the smoothness of the final image rather than the texture or intensity.

Toy Problem

The reconstruction algorithm is based on solving a system of linear equations that relates the differences between adjacent pixels in the image to the values of the pixels themselves. The goal is to estimate the values of the pixels that best fit the given set of equations. First step is identifying the height and width of the input image and creating a mapping from the indices of the pixels in the image to the indices of the variables in the system of linear equations. Next, it initializes two matrices: A_mat which represents the coefficients of the linear equations and b_vec which represents the right-hand side of the equations. The matrices are populated using the differences between adjacent pixels in the image. Finally, the we solve the system of linear equations using the lsqr function from the scipy.linalg module and returns the reconstructed image. Overall, the implemention of a simple algorithm for reconstructing an image based on a system of linear equations.

toy_problem reconstructed
x y_gradient Laplacian

Poisson Belnding

The basic idea behind Poisson blending is to take an image and replace a region of it with another image, while ensuring that the boundary between the two regions is smooth and seamless. This is accomplished by solving a Poisson equation in which the image values inside the region to be replaced are set as the boundary conditions, and the values of the replacement image are used as the source terms. The algorithm for Poisson blending involves several steps. First, the source and destination images are preprocessed to compute their Laplacian matrices. Next, the Laplacian of the source image is modified to match the Laplacian of the destination image in the region to be replaced. This modified Laplacian is then used to solve the Poisson equation and generate the blended image. The resulting blended image is then post-processed to remove any artifacts and ensure that the color and brightness are consistent across the boundary.

Processed 1 Processed 2 Processed 3
Processed 6
Processed 6

Failure cases

While Poisson blending is a powerful technique for seamlessly blending images, following are certain scenarios where it failed to produce good results: (1) If mask regision is not accurate or one of the images has regions that are completely occluded by the other image. (2) When the colors / brightness values in the two images are significantly different. (3) When the textures in the two images are significantly different

Processed 6
Processed 6
Processed 6

Reference