PyTorch codes for "From Degrade to Upgrade: Learning a Self-Supervised Degradation-Guided Adaptive Network for Blind Remote Sensing Image Super-Resolution", Information Fusion, 2023.
Authors: Yi Xiao, Qiangqiang Yuan*, Kui Jiang, Jiang He, Yuan Wang, and Liangpei Zhang
Wuhan University and Huawei Technology
Over the past few years, single image super-resolution (SR) has become a hotspot in the remote sensing area, and numerous methods have made remarkable progress in this fundamental task. However, they usually rely on the assumption that images suffer from a fixed known degradation process, e.g., bicubic downsampling. To save us from performance drop when real-world distribution deviates from the naive assumption, blind image super-resolution for multiple and unknown degradations has been explored. Nevertheless, the lack of a real-world dataset and the challenge of reasonable degradation estimation hinder us from moving forward. In this paper, a self-supervised degradation-guided adaptive network is proposed to mitigate the domain gap between simulation and reality. Firstly, the complicated degradations are characterized by robust representations in embedding space, which promote adaptability to the downstream SR network with degradation priors. Specifically, we incorporated contrastive learning to blind remote sensing image SR, which guides the reconstruction process by encouraging the positive representations (relevant information) while punishing the negatives. Besides, an effective dual-wise feature modulation network is proposed for feature adaptation. With the guide of degradation representations, we conduct modulation on feature and channel dimensions to transform the low-resolution features into the desired domain that is suitable for reconstructing high-resolution images. Extensive experiments on three mainstream datasets have demonstrated our superiority against state-of-the-art methods. Our source code can be found at https://github.com/XY-boy/DRSR
git clone https://github.com/XY-boy/DRSR.git
- Python 3.8
- PyTorch >= 1.9
- Ubuntu 18.04, cuda-11.1
Step I. Please download the following remote sensing datasets:
| Data Type | AID | DOTA-v2.0 | Jilin-1 |
|---|---|---|---|
| Training | Download | None | None |
| Testing | Download | Download | Download |
Step II. Prepare the test sets under different degradation settings:
-
For "Isotropic Blur" degradations: Use the degradation function
generate_mod_LR_bic.pyin IKC by changing the kernel width σ∈[0.2,4.0] at linesig=2.0. Or using our functiongenerate_mod_LR_bic_iso.pyby setting the kernel width list at linesig_list. -
For "Anisotropic Blur + Noise" degradations: Use our modified function
generate_mod_LR_bic_aniso.pyby changing the noise level at linenoise_stable, then change the shape of anisotropic Gaussian blur kernel by setting λ1, λ2, and θ at linedagradation_list.
Set the training option at option/train.py. Then run the main file:
python main.py
Note: The setting of isotropic Gaussian blur and anisotropic Gaussian blur are useless during model training.
- Download the pre-trained models from checkpoint. We provide 4 weights for the evaluation of remote sensing and natural images!
d2u-aniso.pth/d2u-iso.pth ---------- trained on remote sensing images (AID)
DRSR_Blur.pth/DRSR_Noisy.pth ------- trained on natural images (DIV2K)
- For "Isotropic Blur" degradations: Change the
--sigand other testing options atoption/test.py. Then run the test file:
python eval_iso.py
- For "Anisotropic Blur + Noise" degradations: Change the
noise,lr_folder,model_name, andsave_results_dirateval_aniso.py. Then run the test file:
python eval_aniso.py
More Results can be found in our paper PDF!
Our work mainly borrows from DASR and SimCLR. Thanks to these excellent works!
If you have any questions or suggestions, feel free to contact me. 😊
Email: xiao_yi@whu.edu.cn; xy574475@gmail.com
If you find our work helpful in your research, kindly consider citing it. We appreciate your support！😊
@article{xiao2023d2u,
title={From degrade to upgrade: Learning a self-supervised degradation guided adaptive network for blind remote sensing image super-resolution},
author={Xiao, Yi and Yuan, Qiangqiang and Jiang, Kui and He, Jiang and Wang, Yuan and Zhang, Liangpei},
journal={Information Fusion},
volume={96},
pages={297--311},
year={2023},
publisher={Elsevier}
}