This repo is under construction.
Mingdeng Cao,
Sidi Yang,
Yujiu Yang,
Yinqiang Zheng
Paper | Checkpoints | Results on BS-RSC | Results on Fastec-RS | Results on Carla-RS
We propose to rectify Rolling Shutter (RS) distorted images by directly estimating the intermediate distortion flow from the underlying Global Shutter (GS) image to the RS image. This method differs from previous methods that calculate undistortion flow and apply forward warping to obtain the GS image. Instead, the proposed method directly estimates the non-linear distortion flow and employs backward warping to acquire the corrected image. More specifially, the frame-wise RS features are firstly obtained by a multi-scale encoder. After that, a global correlation-based attention mechanism is proposed to to jointly obtain the initial distortion flow and GS features. Then, the coarse-to-fine decoder refines and upscales the resolu-tion of the flow and GS features simultaneously. The final GS image is obtained by a multi-flow predicting strategy.
Clone the repo and install corresponding packages:
git clone https://github.com/ljzycmd/DFRSC.git
cd DFRSC
pip install -r requirements.txt
-
Download pretrained checkpoints
Model Dataset #Num Frames Link DFRSC-3F BS-RSC 3 Github DFRSC-3F Fastec-RS 3 Github DFRSC-3F Carla-RS 3 Github -
Prepare data
You can download the following datasets to calculate metrics:
-
Usage
You should modify the pretrained path in the corresponding config file.
python test.py -opt ./configs/test_DFRSC_3F_BSRSC.yml
-
Prepare dataset
Download the BS-RSC, Fastec-RS and Carla-RS and specify the dataset root in the training configs (
.yaml
file). -
Start training
bash run_train.sh
You my modify the config file path in the
run_train.sh
to train the model with different datasets.
The code is implemented upon the open-soured image restoration framework BasicSR, we thank the authors for relasing such an awesome framework.
If the proposed model is useful for your research, please consider citing our paper
@InProceedings{Cao_2024_CVPR,
author = {Cao, Mingdeng and Yang, Sidi and Yang, Yujiu and Zheng, Yinqiang},
title = {Rolling Shutter Correction with Intermediate Distortion Flow Estimation},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2024}
}
If you have any questions about our project, please feel free to contact me at mingdengcao [AT] gmail.com
.