Skip to content

This is the Pytorch implementation of "Learning Linear Transformations for Fast Image and Video Style Transfer" (CVPR 2019).

License

Notifications You must be signed in to change notification settings

sunshineatnoon/LinearStyleTransfer

Repository files navigation

Learning Linear Transformations for Fast Image and Video Style Transfer

[Paper] [Project Page]

Prerequisites

All code tested on Ubuntu 16.04, pytorch 0.4.1, and opencv 3.4.2

Style Transfer

  • Clone from github: git clone https://github.com/sunshineatnoon/LinearStyleTransfer
  • Download pre-trained models from google drive.
  • Uncompress to root folder :
cd LinearStyleTransfer
unzip models.zip
rm models.zip

Artistic style transfer

python TestArtistic.py

or conduct style transfer on relu_31 features

python TestArtistic.py --vgg_dir models/vgg_r31.pth --decoder_dir models/dec_r31.pth --matrixPath models/r31.pth --layer r31

Photo-realistic style transfer

For photo-realistic style transfer, we need first compile the pytorch_spn repository.

cd libs/pytorch_spn
sh make.sh
cd ../..

Then:

python TestPhotoReal.py

Note: images with _filtered.png as postfix are images filtered by the SPN after style transfer, images with _smooth.png as postfix are images post process by a smooth filter.

Video style transfer

python TestVideo.py

Real-time video demo

python real-time-demo.py --vgg_dir models/vgg_r31.pth --decoder_dir models/dec_r31.pth --matrixPath models/r31.pth --layer r31

Model Training

Data Preparation

  • MSCOCO
wget http://msvocds.blob.core.windows.net/coco2014/train2014.zip
  • WikiArt
    • Either manually download from kaggle.
    • Or install kaggle-cli and download by running:
    kg download -u <username> -p <password> -c painter-by-numbers -f train.zip
    

Training

Train a style transfer model

To train a model that transfers relu4_1 features, run:

python Train.py --vgg_dir models/vgg_r41.pth --decoder_dir models/dec_r41.pth --layer r41 --contentPath PATH_TO_MSCOCO --stylePath PATH_TO_WikiArt --outf OUTPUT_DIR

or train a model that transfers relu3_1 features:

python Train.py --vgg_dir models/vgg_r31.pth --decoder_dir models/dec_r31.pth --layer r31 --contentPath PATH_TO_MSCOCO --stylePath PATH_TO_WikiArt --outf OUTPUT_DIR

Key hyper-parameters:

  • style_layers: which features to compute style loss.
  • style_weight: larger style weight leads to heavier style in transferred images.

Intermediate results and weight will be stored in OUTPUT_DIR

Train a SPN model to cancel distortions for photo-realistic style transfer

Run:

python TrainSPN.py --contentPath PATH_TO_MSCOCO

Acknowledgement

Citation

@inproceedings{li2018learning,
    author = {Li, Xueting and Liu, Sifei and Kautz, Jan and Yang, Ming-Hsuan},
    title = {Learning Linear Transformations for Fast Arbitrary Style Transfer},
    booktitle = {IEEE Conference on Computer Vision and Pattern Recognition},
    year = {2019}
}

About

This is the Pytorch implementation of "Learning Linear Transformations for Fast Image and Video Style Transfer" (CVPR 2019).

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published