GAN Photo Editing

Sanil Pande

(spande)

Part 1: Inverting a Pre-trained Generator

The following images are obtained from 250, 500, 750 and 1000 iterations respectively with different weights for the perceptual loss as mentioned.

The image to be inverted is:

0.002 (Vanilla) from Z-latent

0.02 (Vanilla) from Z-latent

0.2 (Vanilla) from Z-latent

We can see that weight in the range of 0.002 works best.

0.002 (StyleGAN) from W+

0.002 (StyleGAN) from W

0.002 (StyleGAN) from Z

Analysis:

We can see that for the vanilla GAN, the weight does not make much of a difference, but that it also has a very narrow range of outputs.

For the StyleGan, the weight of 0.002 turns out to be the most stable for most of the cases, which is why I have only added results from that particular weight.

We can see that the images generated from a single latent W vector gives stable results, as does generating images from Z.

In terms of speed, the vanilla GAN is the fastest to optimize for. The StyleGan is slower, and the W+ is slower as compared to optimizing W.

Part 2: Interpolating between Cats

The quality of the images between the cat images is good and they seem realistic for most part. However, there are some images which do not look realistic in the middle, since we are depending on the generator for generating images that lie on the real manifold and the generator is not perfect. Another consideration is that we are essentially interpolating between images generated from the inversion, so if the projections of the images to be interpolated between are not realistic, their interpolation suffers in quality as well.

Part 3: Scribble to Images

Here are some example outputs of the given sketches.

Target Image

StyleGan with W+

StyleGan with W

StyleGan with Z

Vanilla with Z

Some other examples of images that are generated with good quality are as follows:

Analysis:

The above images are from the 500th iteration of the StyleGans from the W+, W and Z latent vectors respectively. This leads me to the following conclusions:

• The optimization is very unstable, and seems to converge till the 500th iteration but diverges after that in all the cases
• Vanilla Gan is much more stable, but the manifold of images that it is able to generate is very constrained, and is not able to generalize to images that differ slightly. For example in the final case above, where the cat has yellow eyes, we can see that the StyleGan is at least able to mimic the brown fur and yellowish eyes. But the image that is generated from the vanilla GAN does not reflect these at all, it is only able to slightly mimic the pose that the input sketch has.