16-726 Project 2, Jason Xu

Overview

This project aims to blend a portion of one image onto another image by using gradients. By solving for the values of the blend source using least squares optimization, the source can be blended onto the target as seamlessly as possible. 

Approach

My code solves for the optimal value for blending by:

• Finding all valid pixels in the blending mask (having value of 1)
• Creating sparse matrices A and b to allocate space for all equations based on the number of valid pixels
• For each pixel:
• Find the gradient in the up, down, left, right direction using 4 equations
• Replace image value v_j with target intensity t_j if the target pixel is not in the mask
• Solve the least squares problem using numpy's solving abilities, and copy result into original image

Results

The implementation blended images well, making the mask border virtually invisible. Two results that I liked are:

• blending a plane into the sky of a cityscape
• blending a ship in the desert from the movie "Dune" into a grassland

An example that does not work well is the following:

The color looks wildly off due to the island picture being taken in daylight conditions, and the seascape being taken in sunset conditions. The color temperature of the illumination in the blend source and target are different, resulting in the color in this case. Mitigations could include pre-processing the blend source through editing or style transferring.