Skip to content
code for "Fast Patch-based Style Transfer of Arbitrary Style".
Lua Shell
Branch: master
Clone or download
Latest commit c1a393a Nov 15, 2018
Type Name Latest commit message Commit time
Failed to load latest commit information.
helpers Initial commit Nov 25, 2016
images new examples Nov 28, 2016
lib fix typo Nov 8, 2017
models Initial commit Nov 25, 2016
videos Initial commit Nov 25, 2016
LICENSE Create LICENSE Nov 15, 2018 Update Dec 29, 2017
style-swap.lua Update style-swap.lua Apr 1, 2017
train-vgg-decoder.lua Update train-vgg-decoder.lua Mar 16, 2017

Fast Patch-based Style Transfer of Arbitrary Style


Code is written in Torch. CUDA and CPU modes are available.


(3x3 Patches) Content - w/ Starry Night - w/ Small Worlds I
with AvgPooling - using Inverse Network - using Inverse Network
(w/ Composition X) Original - 5x5 Patch - 9x9 Patch - 15x15 Patch
(w/ La Muse) Original - 3x3 Patch - 5x5 Patch - 9x9 Patch

Download Pretrained VGG-19

git clone
cd style-swap/models
cd ..


Stylizing a single image:

th style-swap.lua --content images/content/bike.jpg --style images/style/starry_night.jpg

More options:

th style-swap.lua --help

eg. increase --patchSize for more abstract stylization

th style-swap.lua --content images/content/brad_pitt.jpg --style images/style/la_muse.jpg --patchSize 7 --patchStride 3

eg. use --contentBatch to stylize all images in a directory.

th style-swap.lua --contentBatch images/content --style images/style/starry_night.jpg

Training an inverse network

Install nninit module:

luarocks install nninit


th train-vgg-decoder.lua --contentDir /path/to/dir --styleDir /path/to/dir

More options:

th train-vgg-decoder.lua --help

For training the network in our paper, we used images from MS COCO and the Painter by Numbers competition hosted by Kaggle. A trained network can be downloaded here.


Frame-by-frame stylization can be done using the -contentBatch option.

An example script using ffmpeg to extract frames, stylize, and re-encode a video.

mkdir video_frames
ffmpeg -i /path/to/video -qscale:v 2 video_frames/video_%04d.jpg
th style-swap --contentBatch video_frames --style /path/to/style/file --save stylized_frames
ffmpeg -i stylized_frames/video_%04d_stylized.jpg -c:v libx264 -pix_fmt yuv420p stylized_video.mp4

Examples of stylized videos are placed in the videos folder. (Original video by TimeLapseHD.)

Reducing GPU Memory Usage

A few ways to reduce memory usage for style-swap.lua:

  • Decrease --maxStyleSize and --maxContentSize. The latter changes the size of the resulting image.
  • Increase --patchStride. This extracts less patches to use for style swap. Best to use a larger --patchSize to ensure the patches still overlap.
  • Last resort: use CPU-only mode by specifying --cpu.
You can’t perform that action at this time.