---

Overview

In this assignment, we use gradient-domain processing techniques for blending and color-to-grayscale (Bells and Whistles) applications. More specifically, for the primary part of this assignment, we focus on Poisson blending, which aims to seamlessly blend an object or texture from a source image into a target image.

Toy Problem

In this problem, we try to reconstruct the toy image by minimizing three kinds of objectives described below.

• Vertical gradient error: \( ((v(x+1, y)-v(x, y))-(s(x+1, y)-s(x, y)))^{2} \).
• Horizontal gradient error: \( ((v(x, y+1)-v(x, y))-(s(x, y+1)-s(x, y)))^{2} \).
• Top left corner error: \( (v(1,1)-s(1,1))^{2} \).

Below shows the result of the reconstruction.

---

Poisson Blending

In this part, our goal is to seamlessly blend the masked source image to the target image using a gradient-based technique named Poisson blending. For this technique, we copy and past the pixel intensity values from the target image to the output blent image, if the pixel is not in the mask, and we minimize a gradient error with respect to the pixel that are inside the mask and the pixels that are in its four-neighbor. More specifically, we want to minimize the following objective: $$ \boldsymbol{v}=\operatorname{argmin}_{\boldsymbol{v}} \sum_{i \in S, j \in N_{i} \cap S}\left(\left(v_{i}-v_{j}\right)-\left(s_{i}-s_{j}\right)\right)^{2}+\sum_{i \in S, j \in N_{i} \cap \neg S}\left(\left(v_{i}-t_{j}\right)-\left(s_{i}-s_{j}\right)\right)^{2}, $$ where the first term correspond to the case where the neighboring pixel (indexed by \(j\)) is also inside the mask, and the second term correspond to the case where the neighboring pixel in not inside the mask. Results of the Poisson blending on the provided images and some other images are listed below.

---

---

---

From the results above we can see when the area between the border of the mask has similar color with the area of the object, Poisson blending often change the color of the object to the color of area along the mask in the background which is a failure case (lion in forest example). When the area between the border of the mask has different color with the area of the object, the Poisson blending tend to work well (Monalisa and lizard example).

---

Bells & Whistles: Mixed Gradients

In this section, we explore a mixed gradient method with the following objective: \( \boldsymbol{v}=\operatorname{argmin}_{\boldsymbol{v}} \sum_{i \in S, j \in N_{i} \cap S}\left(\left(v_{i}-v_{j}\right)-d_{i j}\right)^{2}+\sum_{i \in S, j \in N_{i} \cap \neg S}\left(\left(v_{i}-t_{j}\right)-d_{i j}\right)^{2} \) where \( d_{ij}\) represents the value of the gradient from the source or the target image with the larger magnitude. Results are shown below.

From the results above, we can clearly see that the mixed gradient method can show the texture behind the object, which means that the mask area will not only covers the front ground texture and gradient but those of the background as well.

---

Bells & Whistles: Color2Gray

In this part, our goad is to convert the color image to grayscale image while keep the numbers (patterns) on color image. A naive open-cv color2gray method (cvtColor) will lose the pattern. Here we explore a method where we convert the color image into HSV color space, and use the gradient method by minimize the error between the gradient of output image and the mean gradient of S channel and V channel. More specifically, we aim to compute the \( \boldsymbol{v}=\operatorname{argmin}_{\boldsymbol{v}} \sum_{i \in S, j \in N_{i} \cap S}\left(\left(v_{i}-v_{j}\right)-d_{i j}\right)^{2} \), where \( d_{ij}\) represent the mean gradient of S channel and V channel. Results are shown below.