Nerual Style Transfer

Introduction

In this assignment, you will implement neural style transfer which resembles specific content in a certain artistic style. For example, generate cat images in Ukiyo-e style. The algorithm takes in a content image, a style image, and another input image. The input image is optimized to match the previous two target images in content and style distance space.

Part 1: Content Reconstruction

</tr></table>

1. We can see that reconstruction from initial convolution layers result in output closer to the content image, while reconstruction from later convolution layers result in more texture from the original noise image.

Reconstruction from conv_1 content	Reconstruction from conv_3 content	Reconstruction from conv_5 content

Original botanic garden	Reconstruction of botanic garden
Original dog	Reconstruction of dog

Here are some of the reconstruction examples. The reconstruction has slighly different color in some patches, but overall the original image and the reconstructed one are pretty similar. My favorite one is the dog picture.

Part 2: Texture Synthesis

</tr></table>

1. Texture generated from earlier layers reflect coarser shape of texture and texture generated from later layers reflect finer details of texture.

Texture from conv_1, conv2, conv3	Texture from conv_4, conv_5	Texture from conv_1, conv_3, conv_5

Texture from The Scream	Texture from Frida Kahlo's portrait

The textures here are generated from all conv layers. This means that the low level details and high level shape of the texture are all included.

Part 3: Style Transfer

1. The optimization runs for 300 steps with parameters style_weight=1000000, content_weight=1. The content layer is conv_4 and style layer is all the conv layers.

   	Original dancing image	Original dog image
   Picasso style	Picasso style dancing	Picasso style dog
   Starry Night style	Starry Night style dancing	Starry Night style dog

2. Here are some results of style transfer.
3. Here are some results from random noise. Compared to the previous results, the ones generated from random noise is significantly less clear and has many artifacts that doesn't belong in the content image.

   	Generated from Picasso style, dancing content	Generated from Picasso style, dog content

   Frida Kahlo style Falling Waters
   Jackson Pollock painting	Jackson Pollock style botanic garden

4. Some of my favourite outputs.