Few Shot Patch Based Training for Image Translation using PyTorch Lightning

---

Trained using NVIDIA GTX 1050 Ti in seven minutes. Yep, this is me.

About

This is my personal implementation of following the paper using PyTorch Lightning.

Interactive Video Stylization Using Few-Shot Patch-Based Training
O. Texler, D. Futschik, M. Kučera, O. Jamriška, Š. Sochorová, M. Chai, S. Tulyakov, and D. Sýkora
[WebPage], [Paper], [BiBTeX]

I wrote it as an exercise to learn PyTorch. I tried many different variants of the models but the original one is the one that works the best.

You can find more information on the official github page https://github.com/OndrejTexler/Few-Shot-Patch-Based-Training

and on the Lightning docs https://pytorch-lightning.readthedocs.io/en/latest/

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

Installing

Tested with Python 3.9.6, pytorch 1.9.0 on Ubuntu 20.04 using conda

conda create -n FSPBT -y python==3.9.6
conda activate FSPBT
conda install -y -c pytorch -c conda-forge pytorch-gpu==1.9.0 torchvision==0.10.0 cudatoolkit==11.2.2 pytorch-lightning==1.4.2

Demo data and pretrained models

To download the demo data along with pretrained models (on Linux)

./download_data.sh

Alternatively you can download it from https://drive.google.com/file/d/1WI71nYP-z0mfDpuUW36s3sswpwRwwfrN/view and extract in the "data" folder

Training

You can just start

conda activate FSPBT
python train.py

settings for data_path are inside the file itself

Files are expected to be in folders

data_path/
          input
          target
          mask (optional)

View logs

The trainer uses default Lightning logger (Tensorboard)

conda activate FSPBT
tensorboard --logdir lightning_logs/ 

Evaluation

You can just start

conda activate FSPBT
python eval.py

Files will be produced in folder "data_path/output", but you can change it in eval.py

Authors

• Midas Gordiades (Lorenzo Breschi) - PyTorch Lightning implementation

Acknowledgements

All credits go to the original authors

Interactive Video Stylization Using Few-Shot Patch-Based Training
O. Texler, D. Futschik, M. Kučera, O. Jamriška, Š. Sochorová, M. Chai, S. Tulyakov, and D. Sýkora
[WebPage], [Paper], [BiBTeX]

Citing

If you find Interactive Video Stylization Using Few-Shot Patch-Based Training useful for your research or work, please use the following BibTeX entry.

@Article{Texler20-SIG,
    author    = "Ond\v{r}ej Texler and David Futschik and Michal Ku\v{c}era and Ond\v{r}ej Jamri\v{s}ka and \v{S}\'{a}rka Sochorov\'{a} and Menglei Chai and Sergey Tulyakov and Daniel S\'{y}kora",
    title     = "Interactive Video Stylization Using Few-Shot Patch-Based Training",
    journal   = "ACM Transactions on Graphics",
    volume    = "39",
    number    = "4",
    pages     = "73",
    year      = "2020",
}