Learning Degradation Representations for Image Deblurring

This is an official implementation of “Learning Degradation Representations for Image Deblurring” with PyTorch, accepted by ECCV 2022.

Abstract: In various learning-based image restoration tasks, such as image denoising and image super-resolution, the degradation representations were widely used to model the degradation process and handle complicated degradation patterns. However, they are less explored in learning-based image deblurring as blur kernel estimation cannot perform well in real-world challenging cases. We argue that it is particularly necessary for image deblurring to model degradation representations since blurry patterns typically show much larger variations than noisy patterns or high-frequency textures. In this paper, we propose a framework to learn spatially adaptive degradation representations of blurry images. A novel joint image reblurring and deblurring learning process is presented to improve the expressiveness of degradation representations. To make learned degradation representations effective in reblurring and deblurring, we propose a Multi-Scale Degradation Injection Network (MSDI-Net) to integrate them into the neural networks. With the integration, MSDI-Net can handle various and complicated blurry patterns adaptively. Experiments on the GoPro and RealBlur datasets demonstrate that our proposed deblurring framework with the learned degradation representations outperforms state-of-the-art methods with appealing improvements.

Keywords: Image Deblurring, Degradation Representations

Performance

Get Started

Installation

python 3.8.5
pytorch 1.8.0
cuda 11.3

git clone https://github.com/dasongli1/Learning_degradation.git
cd Learning_degradation
pip install -r requirements.txt
python setup.py develop --no_cuda_ext

Image Deblurring

• download the train set in ./datasets/GoPro/train and test set in ./datasets/GoPro/test (refer to MPRNet)

  • it should be like:

    ./datasets/
    ./datasets/GoPro/
    ./datasets/GoPro/train/
    ./datasets/GoPro/train/input/
    ./datasets/GoPro/train/target/
    ./datasets/GoPro/test/
    ./datasets/GoPro/test/input/
    ./datasets/GoPro/test/target/

  • python gopro.py

Training version 1:

• Download the pre-trained model of net_Encoder and prior_upsampling to ./checkpoints/

• training script:

python -m torch.distributed.launch --nproc_per_node=8 basicsr/train.py -opt MSDINet-Train.yml --launcher pytorch

Training version 2's Learning degradation in joint training of reblurring and deblurring:

• previous version of learning degradation requires A100 gpus (80G) to train. We provide version 2 (less than 16G).
• training script:

python -m torch.distributed.launch --nproc_per_node=8 basicsr/train1.py -opt MSDINet2e-Train.yml --launcher pytorch

Training version 2:

• Download the pre-trained model of net_Encoder2, prior_upsampling2 and net_Encoder to ./checkpoints/
• training script:

python -m torch.distributed.launch --nproc_per_node=8 basicsr/train.py -opt MSDINet2-Train.yml --launcher pytorch

Testing:

• eval: We provide the pre-trained model for evaluation.
• Please download the model pretrained model to ./checkpoints/msdi_net.pth
• python basicsr/test.py -opt MSDINet-Test.yml

Citation

If our work is useful for your research, please consider citing:

@InProceedings{li2022learning,
    author = {Li, Dasong and Zhang, Yi and Cheung, Ka Chun and Wang, Xiaogang and Qin, Hongwei and Li, Hongsheng},
    title = {Learning Degradation Representations for Image Deblurring},
    booktitle = {ECCV},
    year = {2022}
}

Acknowledgement

In this project, we use parts of codes in:

• Basicsr
• HINet
• SPADE