This is the official repository of the following paper:
Local Implicit Normalizing Flow for Arbitrary-Scale Image Super-Resolution (LINF)
Jie-En Yao*, Li-Yuan Tsao*, Yi-Chen Lo, Roy Tseng, Chia-Che Chang, Chun-Yi Lee
If you are interested in our work, you can access ElsaLab for more and feel free to contact us.
pip install torch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 -f https://download.pytorch.org/whl/torch_stable.html
pip install -r requirements.txt
- SR benchmark datasets (Set5, Set14, BSD100, Urban100) You can find the download link in the repo jbhuang0604/SelfExSR
- DIV2K
You should download
DIV2K/DIV2K_train_HR
andDIV2K/DIV2K_valid_HR
with their corresponding downsampled versions for training and validation. - Flickr2K
Model | Download |
---|---|
EDSR-baseline-LINF | Google drive |
RDN-LINF | Google Drive |
SwinIR-LINF | Google Drive |
RRDB-LINF (3x3 patch) | Google drive |
- You should modify the
root_path
in the config files to the path of the datasets. - For stage 1, you can use the config files located in
configs/train-div2k
. For stage 2, the config files are located inconfigs/fine-tune
. - If you want to resume training, please specify the the path of your checkpoint in the
resume
argument in the config file. - The checkpoints will be automatically saved in
./save/<EXP_NAME>
.
To launch your experiments, you can use the following command:
python train.py --config <CONFIG_PATH> --gpu <GPU_ID(s)> --name <EXP_NAME> --patch <PATCH_SIZE>
# EDSR
python train.py --config configs/train-div2k/train_edsr-flow.yaml --gpu 0 --name edsr
# RDN
python train.py --config configs/train-div2k/train_rdn-flow.yaml --gpu 0 --name rdn
# SwinIR
python train.py --config configs/train-div2k/train_swinir-flow.yaml --gpu 0 --name swinir
# RRDB patch DF2K
python train.py --config configs/train-div2k/train_rrdb-flow-DF2K.yaml --gpu 0 --patch 3 --name rrdb
# EDSR
python train.py --config configs/fine-tune/fine-tune_edsr-flow.yaml --gpu 0 --name edsr_finetune
# RDN
python train.py --config configs/fine-tune/fine-tune_rdn-flow.yaml --gpu 0 --name rdn_finetune
# SwinIR
python train.py --config configs/fine-tune/fine-tune_swinir-flow.yaml --gpu 0 --name swinir_finetune
# RRDB patch DF2K
python train.py --config configs/fine-tune/fine-tune_rrdb-flow-DF2K.yaml --gpu 0 --patch 3 --name rrdb_finetune
- You should modify the
root_path
in the config files to the path of the datasets. - You can store visualization results using the arguments
--sample <NUM_SAMPLES>
and--name <VIS_RESULTS_NAME>
. The generated images will be automatically saved in./sample/<VIS_RESULTS_NAME>
. - Other arguments
--patch
: Whether the model is a patch-based model (patch size > 1).
--detail
: Print the results of SSIM and LPIPS.
--randomness
: Run five experiments and report the mean results.
--temperature
: Set the sampling temperature
To launch your evaluation, you can use the following command:
- For benchmark datasets (EDSR-baseline-LINF, RDN-LINF, SwinIR-LINF)
# benchmark deterministic
sh scripts/test-benchmark-ours-t0.sh <MODEL_PATH> <GPU_ID>
# benchmark random sample
sh scripts/test-benchmark-ours-t.sh <MODEL_PATH> <GPU_ID> <TEMPERATURE>
- For DIV2K dataset (RRDB-LINF 3x3 patch)
# RRDB patch div2k deterministic
python test.py --config configs/test/test-fast-div2k-4.yaml --model <MODEL_PATH> --gpu <GPU_ID> --detail --temperature 0.0 --patch
# RRDB patch div2k random sample
python test.py --config configs/test/test-fast-div2k-4.yaml --model <MODEL_PATH> --gpu <GPU_ID> --detail --randomness --temperature <TEMPERATURE> --patch
- For the evaluation of generative SR, we didn't use the same border-shaving method as in the arbitrary-scale SR evaluation since generative SR works didn't evaluate with that. You can modify the line 142 of
utils.py
fromshave = scale + 6
toshave = scale
in order to get the same scores reported in our paper (with only a negligible difference on PSNR).
If you find our work helpful for your research, we would greatly appreciate your assistance in sharing it with the community and citing it using the following BibTex. Thank you for supporting our research.
@inproceedings{yao2023local,
title = {Local Implicit Normalizing Flow for Arbitrary-Scale Image Super-Resolution},
author = {Jie-En Yao and Li-Yuan Tsao and Yi-Chen Lo and Roy Tseng and Chia-Che Chang and Chun-Yi Lee},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2023},
}
Our code was built on LIIF and LTE. We would like to express our gratitude to the authors for generously sharing their code and contributing to the community.