Skip to content
This is the Pytorch implementation of "Learning Linear Transformations for Fast Image and Video Style Transfer" (CVPR 2019).
Python Cuda C Shell
Branch: master
Clone or download
Latest commit 188e8e5 Jun 19, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data release Apr 2, 2019
doc release Dec 5, 2018
libs release Apr 2, 2019
.DS_Store release Dec 5, 2018
LICENSE Create LICENSE Dec 5, 2018
README.md Update README.md Jun 18, 2019
TestArtistic.py release Apr 2, 2019
TestPhotoReal.py release Apr 2, 2019
TestVideo.py release Apr 2, 2019
Train.py release Apr 2, 2019
TrainSPN.py
real-time-demo.py release Dec 5, 2018

README.md

Learning Linear Transformations for Fast Image and Video Style Transfer

[Paper] [Project Page]

Prerequisites

All code tested on Ubuntu 16.04, pytorch 0.4.1, and opencv 3.4.2

Style Transfer

  • Clone from github: git clone https://github.com/sunshineatnoon/LinearStyleTransfer
  • Download pre-trained models from google drive.
  • Uncompress to root folder :
cd LinearStyleTransfer
unzip models.zip
rm models.zip

Artistic style transfer

python TestArtistic.py

or conduct style transfer on relu_31 features

python TestArtistic.py --vgg_dir models/vgg_r31.pth --decoder_dir models/dec_r31.pth --matrixPath models/r31.pth --layer r31

Photo-realistic style transfer

For photo-realistic style transfer, we need first compile the pytorch_spn repository.

cd libs/pytorch_spn
sh make.sh
cd ../..

Then:

python TestPhotoReal.py

Note: images with _filtered.png as postfix are images filtered by the SPN after style transfer, images with _smooth.png as postfix are images post process by a smooth filter.

Video style transfer

python TestVideo.py

Real-time video demo

python real-time-demo.py --vgg_dir models/vgg_r31.pth --decoder_dir models/dec_r31.pth --matrixPath models/r31.pth --layer r31

Model Training

Data Preparation

  • MSCOCO
wget http://msvocds.blob.core.windows.net/coco2014/train2014.zip
  • WikiArt
    • Either manually download from kaggle.
    • Or install kaggle-cli and download by running:
    kg download -u <username> -p <password> -c painter-by-numbers -f train.zip
    

Training

Train a style transfer model

To train a model that transfers relu4_1 features, run:

python Train.py --vgg_dir models/vgg_r41.pth --decoder_dir models/dec_r41.pth --layer r41 --contentPath PATH_TO_MSCOCO --stylePath PATH_TO_WikiArt --outf OUTPUT_DIR

or train a model that transfers relu3_1 features:

python Train.py --vgg_dir models/vgg_r31.pth --decoder_dir models/dec_r31.pth --layer r31 --contentPath PATH_TO_MSCOCO --stylePath PATH_TO_WikiArt --outf OUTPUT_DIR

Key hyper-parameters:

  • style_layers: which features to compute style loss.
  • style_weight: larger style weight leads to heavier style in transferred images.

Intermediate results and weight will be stored in OUTPUT_DIR

Train a SPN model to cancel distortions for photo-realistic style transfer

Run:

python TrainSPN.py --contentPath PATH_TO_MSCOCO

Acknowledgement

Citation

@inproceedings{li2018learning,
    author = {Li, Xueting and Liu, Sifei and Kautz, Jan and Yang, Ming-Hsuan},
    title = {Learning Linear Transformations for Fast Arbitrary Style Transfer},
    booktitle = {IEEE Conference on Computer Vision and Pattern Recognition},
    year = {2019}
}
You can’t perform that action at this time.