Skip to content
A Jupyter notebook project that implements Neural Style Transfer using Swift for TensorFlow.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore Remove checkpoints and re-ran notebook. May 2, 2019
CheckpointReader.swift
LICENSE
README.md
StyleGraphic.jpg
StyleTransfer.ipynb
baybridge.jpg
dots.jpg
einstein.jpg Finally solved clipping issue. Apr 28, 2019
goldengate.jpg Changed around the last bit of the notebook May 2, 2019
kadinsky.jpg
lines.jpg
painted_ladies.jpg Finally solved clipping issue. Apr 28, 2019
painted_ladies_256.jpg
puppy.jpg
strokes.jpg Hacking on getting activation maps out of network. Apr 21, 2019
turtle.jpg Finally solved clipping issue. Apr 28, 2019
vangogh_rhone.jpg Changed around the last bit of the notebook May 2, 2019
vangogh_starry_night.jpg Finally solved clipping issue. Apr 28, 2019
vangogh_starry_night_256.jpg

README.md

Neural Style Transfer using Swift for TensorFlow

This is a Jupyter notebook project that implements Neural Style Transfer using Swift for TensorFlow.

Alt text

Attribution from left to right:

What is Neural Style Transfer?

Simply put, Neural Style Transfer [Gatys et al.] is a process by which we take the style of one image, the content of another image and generate a new image that exhibits the same stylistic features as the style image while preserving the high-level structure of the content image.

The neural part, as you might expect, is where the real magic happens. If you've read about or taken a course on Convolutional Neural Networks, you've probably encountered the wonderful paper Visualizing and Understanding Convolutional Networks by Zeiler and Fergus. This paper keenly illustrates the fact that each kernel in a conv net acts as a sort of feature extractor. The features start out primitive and local but as the net goes deeper, the features become more abstract and global. Neural Style Transfer exploits this fact, extracting style information from the activations of various layers. We also extract content information from the higher layers of the network. We use this information to compute what is referred to in the literature as the "Perceptual Loss".

Given an input image, our goal is to minimize Perceptual Loss with respect to that image. In order to compute the Perceptual Loss, we need the following:

  • A style image
  • A content image
  • A styled image. This is the output. We can initialize it a number of ways (which we'll explore below).
  • A way of extracting style information from layer activations.
  • A pretrained conv net (we'll use VGG-19 [Karen Simonyan, Andrew Zisserman]).

We'll use the Adam optimizer [Diederik P. Kingma, Jimmy Ba] to optimize our "styled" image.

You can’t perform that action at this time.