This repository is for STVUN introduced in the following paper
Jaeyeon Kang, Younghyun Jo, Seoung Wug Oh, Peter Vajda, and Seon Joo Kim. "STVUN: Deep Space-Time Video Upsampling Networks", ECCV 2020. PDF, Video
Python>=3.6.8, Pytorch=1.2, CUDA version>= 10.2
-
Clone this github repo
git clone https://github.com/JaeYeonKang/STVUN-Pytorch cd STVUN-PYtorch
-
Compile the correlation package
cd networks/correlation_package_pytorch ./build.sh
-
Place your test dataset in './test' folder (e.g. ./test/Vid4)
-
Download our pretrained models from link. Then, place the weights in ./pretrained_model
-
Run demo
python demo.py --data_dir $DATA_DIR$ \ --save_dir $SAVE_DIR$ --pre_train $PRETRAINED_MODEL$ \ --time_step $TIME_STEP$
- DATA_DIR : path to test dataset
- SAVE_DIR : path to save results
- PRETRAINED_MODEL : path to pretrained model
- TIME_STEP : number of intermediate frames to generate
For example,
python demo.py --data_dir ./test/Vid4 \ --save_dir ./results --pre_train ./pretrained_model/STVUN.pth \ --time_step 3
We collect Space-Time Video Test(STVT) dataset that consists of 12 dynamic scenes with both various motions and spatial details for the joint upsampling evaluation. Each scene has at least 50 frames. You can download our STVT dataset from link
If you use any part of this code in your research, please cite our paper
@article{kang2020deep,
title={Deep Space-Time Video Upsampling Networks},
author={Kang, Jaeyeon and Jo, Younghyun and Oh, Seoung Wug and Vajda, Peter and Kim, Seon Joo},
journal={arXiv preprint arXiv:2004.02432},
year={2020}
}