Designing a Practical Degradation Model for Deep Blind Image Super-Resolution

Kai Zhang, Jingyun Liang, Luc Van Gool, Radu Timofte
Computer Vision Lab, ETH Zurich, Switzerland

[Paper]

✨ Some visual examples: oldphoto2; butterfly; comic; oldphoto3; oldphoto6; comic_01; comic_03; comic_04

---

• Testing code
• Main idea
• Comparison
• More visual results on RealSRSet dataset
• Visual results on DPED dataset
• Citation
• Acknowledgments

Testing code

• main_test_bsrgan.py
• model_zoo (Download the following models from Google drive or 腾讯微云).

  • Proposed:

    • BSRNet.pth 🌱
    • BSRGAN.pth 🌱

  • Compared methods:

    • RRDB.pth ---> original link
    • ESRGAN.pth ---> original link
    • FSSR_DPED.pth ---> original link
    • FSSR_DPED.pth ---> original link
    • RealSR_DPED.pth ---> original link
    • RealSR_JPEG.pth ---> original link

  • If you need BSRGAN model for scale factor 2, please send an email to me.

Main idea

Design a new degradation model to synthesize LR images for training:

• 1) Make the blur, downsampling and noise more practical
  • Blur: two convolutions with isotropic and anisotropic Gaussian kernels from both the HR space and LR space
  • Downsampling: nearest, bilinear, bicubic, down-up-sampling
  • Noise: Gaussian noise, JPEG compression noise, processed camera sensor noise
• 2) Degradation shuffle: instead of using the commonly-used blur/downsampling/noise-addition pipeline, we perform randomly shuffled degradations to synthesize LR images

Some notes on the proposed degradation model:

• The degradation model is mainly designed to synthesize degraded LR images. Its most direct application is to train a deep blind super-resolver with paired LR/HR images. In particular, the degradation model can be performed on a large dataset of HR images to produce unlimited perfectly aligned training images, which typically do not suffer from the limited data issue of laboriously collected paired data and the misalignment issue of unpaired training data.

• The degradation model tends to be unsuited to model a degraded LR image as it involves too many degradation parameters and also adopts a random shuffle strategy.

• The degradation model can produce some degradation cases that rarely happen in real-world scenarios, while this can still be expected to improve the generalization ability of the trained deep blind super-resolver.

• A DNN with large capacity has the ability to handle different degradations via a single model. This has been validated multiple times. For example, DnCNN is able to handle SISR with different scale factors, JPEG compression deblocking with different quality factors and denoising for a wide range of noise levels, while still having a performance comparable to VDSR for SISR. It is worth noting that even when the super-resolver reduces the performance for unrealistic bicubic downsampling, it is still a preferred choice for real SISR.

• One can conveniently modify the degradation model by changing the degradation parameter settings and adding more reasonable degradation types to improve the practicability for a certain application.

Comparison

These no-reference IQA metrics, i.e., NIQE, NRQM and PI, do not always match perceptual visual quality [1] and the IQA metric should be updated with new SISR methods [2]. We further argue that the IQA metric for SISR should also be updated with new image degradation types, which we leave for future work.

[1] "NTIRE 2020 challenge on real-world image super-resolution: Methods and results." CVPRW, 2020.
[2] "PIPAL: a large-scale image quality assessment dataset for perceptual image restoration." ECCV, 2020.

More visual results on RealSRSet dataset

Left: real images | Right: super-resolved images with scale factor 4

Visual results on DPED dataset

Without using any prior information of DPED dataset for training, our BSRGAN still performs well.

Citation

@inproceedings{zhang2021designing,
  title={Designing a Practical Degradation Model for Deep Blind Image Super-Resolution},
  author={Zhang, Kai and Liang, Jingyun and Van Gool, Luc and Timofte, Radu},
  booktitle={arxiv},
  year={2021}
}

Acknowledgments

This work was partly supported by the ETH Zurich Fund (OK), a Huawei Technologies Oy (Finland) project, and an Amazon AWS grant.

License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.