KeDuSR (AAAI 2024)

This repository contains official implementation of our AAAI 2024 paper "KeDuSR: Real-World Dual-Lens Super-Resolution via Kernel-Free Matching", by Huanjing Yue, Zifan Cui, Kun Li, and Jingyu Yang.

paper | dataset | results | pre-trained model

You can download the above-mentioned resources from Baidu Netdisk, or Google Drive.

Introduction

We propose a kernel-free matching strategy by matching between LR-center and LR-corner, which avoids the resolution gap between the LR and Ref in Dual-Lens Reference-Based SR task.

Dataset

We constructed the first well-aligned Dual-Lens SR dataset, DuSR-Real.

We further apply the same processing approach on the CameraFusion and RealMCVSR, to generate well-aligned real LR-HR pairs and the pairs that cannot be well aligned are removed. The reorganized datasets are named CameraFusion-Real and RealMCVSR-Real, respectively.

Copyright

The dataset is available for the academic purpose only. Any researcher who uses the dataset should obey the licence as below:

All of the Dataset are copyright by Intelligent Imaging and Reconstruction Laboratory, Tianjin University and published under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License. This means that you must attribute the work in the manner specified by the authors, you may not use this work for commercial purposes and if you alter, transform, or build upon this work, you may distribute the resulting work only under the same license.

Dependencies

• Python 3.10.8 and Pytorch 2.0.0, other versions are also acceptable.
• tqdm, lpips, numpy, opencv-contrib-python, tensorboardX.
• NVIDIA 3090 24G.

Quick Start

Download the dataset and the pre-trained models, place them as follows:

KeDuSR
├── dataset
│   ├── DuSR-Real
│   ├── RealMCVSR-Real
│   └── CameraFusion-Real
└── pre-trained
    ├── xxx.pth
    ├── xxx.pth
    └── ...

For quick test on DuSR-Real, run the scipts:

# only reconstruction loss (DuSR-Real)
python test.py --dataset_root dataset --dataset_name DuSR-Real --resume pre-trained/DuSR-Real_220000.pth

# hybrid loss (DuSR-Real)
python test.py --dataset_root dataset --dataset_name DuSR-Real --resume pre-trained/DuSR-Real_GAN_180000.pth

# Calculate LPIPS
python LPIPS_metric.py --SR_dir DuSR-Real_220000 --HR_dir dataset/DuSR-Real/test/HR

Training

To train the KeDuSR model on DuSR-Real, run the scipts:

# only reconstruction loss (DuSR-Real)
python train.py --dataset_root dataset --dataset_name DuSR-Real --use_tb_logger --loss_Charbonnier

# hybrid loss (DuSR-Real)
python train.py --dataset_root dataset --dataset_name DuSR-Real --use_tb_logger --loss_Charbonnier --loss_perceptual --loss_adv

#Resume Training
python train.py --dataset_root dataset --dataset_name DuSR-Real --use_tb_logger --loss_Charbonnier --resume weights/20240117_232231/snapshot/net_xxx.pth --resume_optim weights/20240117_232231/snapshot/optimizer_G_xxx.pth --resume_scheduler weights/20240117_232231/snapshot/scheduler_xxx.pth

Citation

If you find this work useful for your research, please cite:

@article{yue2023kedusr,
  title={KeDuSR: Real-World Dual-Lens Super-Resolution via Kernel-Free Matching},
  author={Yue, Huanjing and Cui, Zifan and Li, Kun and Yang, Jingyu},
  journal={arXiv preprint arXiv:2312.17050},
  year={2023}
}

Acknowledgement

We thank the authors of TTSR, DCSR, MASA and for sharing their codes.