LF-DMnet: Real-World Light Field Image Super-Resolution via Degradation Modulation

This is the PyTorch implementation of the method in our paper "Real-World Light Field Image Super-Resolution via Degradation Modulation". [project], [paper].

News and Updates:

• 2024-04-01: Our paper is published on IEEE TNNLS. [pdf]
• 2024-03-14: Our paper is accepted by IEEE TNNLS.
• 2022-06-21: Codes and models are released. Welcome to try our codes and report the bugs/mistakes you meet.
• 2022-06-17: Website is online, on which we provided comparative videos and an interactive demo.
• 2022-05-25: Repository is created.

Demo Videos:

We show the SR results of our LF-DMnet on real LFs captured by Lytro Illum cameras. More examples are available here. Note that, these videos have been compressed, and the results shown below are inferior to the original outputs of our LF-DMnet.

ISO-Chart.mp4

General-11.mp4

Preparation:

1. Requirement:

• PyTorch 1.3.0, torchvision 0.4.1. The code is tested with python=3.7, cuda=9.0.
• Matlab for training/validation data generation.

2. Datasets:

• We used the HCInew, HCIold and STFgantry datasets for training and validation. Please first download the aforementioned datasets via Baidu Drive (key:7nzy) or OneDrive, and place these datasets to the folder ../Datasets/.
• We used the EPFL, INRIA and STFlytro datasets (which are developed by using Lytro cameras) to test the practical value of our method.

3. Generating training/validation data:

• Run GenerateDataForTraining.m to generate training data. The generated data will be saved in ../Data/Train_MDSR_5x5/.
• Please download the validation data via OneDrive and place these data to the folder ../Data/Validation_MDSR_5x5/.

Train:

• Set the hyper-parameters in parse_args() if needed. We have provided our default settings in the realeased codes.
• Run train.py to perform network training.
• Checkpoint will be saved to ./log/.

Validation (synthetic degradation):

• Run validation.py to perform validation on each dataset.
• The metric scores will be printed on the screen.

Test on your own LFs:

• Place the input LFs into ./input (see the attached examples).
• Run test.py to perform SR.
• The super-resolved LF images will be automatically saved to ./output.

Citiation

If you find this work helpful, please consider citing:

@Article{LF-DMnet,
    author    = {Wang, Yingqian and Liang, Zhengyu and Wang, Longguang and Yang, Jungang and An, Wei and Guo, Yulan},
    title     = {Real-World Light Field Image Super-Resolution Via Degradation Modulation},
    journal   = {IEEE Transactions on Neural Networks and Learning Systems},
    year      = {2024},   
}

Contact

Welcome to raise issues or email to wangyingqian16@nudt.edu.cn for any question regarding this work.

statistics