Light Field Super-Resolution with Zero-Shot Learning

Official implementation of the following paper

Zhen Cheng, Zhiwei Xiong, Chang Chen, Dong Liu, and Zheng-Jun Zha, Light Field Super-Resolution with Zero-Shot Learning. In CVPR 2021. (Oral)

Paper | Supplementary Material | Bibtex

Dependencies

• Python 3 (Recommend to use Anaconda)
• Pytorch 1.0.0
• Numpy
• Scipy
• TensorboardX
• MATLAB (For data preparation)

Usage

1. Preparation

• We make experiments on the luminance channel which could generated by MATLAB function rgb2ycbcr.
• Download our processed data from BaiduYun (Access code: cvpr) or GoogleDrive and put them into the folder data.
• Download our pre-trained model from BaiduYun (Access code: cvpr) or GoogleDrive and put them into the folder pretrain.
• If you want the ground truth HR light field images with RGB channels, please download them from BaiduYun (Access code: cvpr) or GoogleDrive.

2. Zero-shot learning from scratch

Take scene Spear_Fence_2 in dataset EPFL with scaling factor 2 as an example.

python Main_LFZSSR.py --dataset="EPFL" --start=2 --end=3 --scale=2 --record

You can refer to the script Main_LFZSSR.py to know the meaning of each parameter.

3. Error-guided finetuning

Our error-guided finetuning needs a pre-trained model for initialization and error map generation, please download our pre-trained models.

Take scene Spear_Fence_2 in dataset EPFL with scaling factor 2 and source dataset HFUT as an example.

python Main_error_guided_finetuning.py --dataset="EPFL" --start=2 --end=3 --scale=2 --source="HFUT" --record

You can refer to the script Main_error_guided_finetuning.py to know the meaning of each parameter.

4. Hyper-parameters

We set the hyper-parameters during training and testing after tuning on our testing data. If you want to use our algorithm on your own data, please refer to Hyper-parameters for detailed descriptions of each hyper-parameter.

Citation

If you find this work helpful, please consider citing our paper.

@InProceedings{Cheng_2021_CVPR,
    author    = {Cheng, Zhen and Xiong, Zhiwei and Chen, Chang and Liu, Dong and Zha, Zheng-Jun},
    title     = {Light Field Super-Resolution With Zero-Shot Learning},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {10010-10019}
}

Related Projects

Light field depth estimation, LFDEN

ZSSR

Contact

If you have any problem about the released code, please do not hesitate to contact me with email (mywander@mail.ustc.edu.cn).