Video Frame Interpolation via Residue Refinement (RRIN)

Paper

Haopeng Li, Yuan Yuan, Qi Wang

IEEE Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020

Table of Contents

1. Introduction
2. Requirements and Dependencies
3. Installation
4. Pre-trained Model
5. Testing Demo
6. Downloading Results

Introduction

We propose Video Frame Interpolation via Residue Refinement (RRIN) that leverages residue refinement and adaptive weight to synthesize in-between frames.

Residue refinement is used for optical flow and image generation for higher accuracy and better visual appearance, while the adaptive weight map combines the forward and backward warped frames to reduce the artifacts.

All sub-modules in our method are implemented by U-Net with various depths.

Experiments on public datasets demonstrate the effectiveness and superiority of our method over the state-of-the-art approaches.

Requirements and Dependencies

• Python = 3.6.8 in Anaconda3 = 4.7.5
• CUDA = 9.2 & cuDNN = 7.0
• PyTorch = 1.0

Installation

Download the repository:

$ git clone https://github.com/HopLee6/RRIN.git

Pre-trained Model

We provide the pre-trained model of "RRIN" at OneDrive, which achieves the same results as reported in the paper. Download the pre-trained model to /RRIN.

Testing Demo

Test the model using frames in /RRIN/data:

$ python demo.py

and get the interpolated frame /RRIN/data/im_interp.png.

Downloading Results

Our RRIN model achieves the state-of-the-art performance on Vimeo90K, and comparable performance on UCF101. Download our interpolated results:

• Vimeo90K

• UCF101

Contact

Haopeng Li

License and Citation

The use of this code is RESTRICTED to non-commercial research and educational purposes.

@INPROCEEDINGS{RRIN, 
author={Haopeng, Li and Yuan, Yuan and Qi, Wang}, 
booktitle={ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, 
title={Video Frame Interpolation Via Residue Refinement}, 
year={2020}, 
pages={2613-2617}
}