DNF: Decouple and Feedback Network for Seeing in the Dark

This repository contains the official implementation of the following paper:

DNF: Decouple and Feedback Network for Seeing in the Dark
Xin Jin^*, Ling-Hao Han^*, Zhen Li, Zhi Chai, Chunle Guo, Chongyi Li
(* denotes equal contribution.)
In CVPR 2023

[Paper] [Google Drive] [Homepage (TBD)] [Video (TBD)]

News

Future work can be found in todo.md.

• May, 2023: Our code is publicly available.
• Mar, 2023: Excited to announce that our paper was selected as CVPR 2023 Highlight✨ (10% of accepted papers, 2.5% of submissions)!
• Feb, 2023: Our paper "DNF: Decouple and Feedback Network for Seeing in the Dark" has been accepted by CVPR 2023.
• Apr, 2022: A single-stage version of our network has won the third place in NTIRE 2022 Night Photography Challenge.

Dependencies and Installation

1. Clone Repo

   git clone https://github.com/Srameo/DNF.git CVPR23-DNF

2. Create Conda Environment and Install Dependencies

   conda create -n dnf python=3.7.11
   conda activate dnf
   pip install -r requirements.txt -f https://download.pytorch.org/whl/cu111/torch_stable.html

3. Download pretrained models from Pretrained Models, and put them in the pretrained folder.

Pretrained Models

Trained on	🔗 Download Links	Config file	CFA Pattern	Framework
SID Sony	[Google Drive][Baidu Cloud]	[configs/cvpr/sony/baseline]	Bayer (RGGB)	DNF
SID Fuji	[Google Drive][Baidu Cloud]	[configs/cvpr/fuji/baseline]	X-Trans	DNF
MCR	[Google Drive][Baidu Cloud]	[configs/cvpr/mcr/baseline]	Bayer (RGGB)	DNF

Quick Demo

Try DNF on your own RAW images (with RGGB Bayer pattern) !

1. Download the pretrained DNF (trained on SID Sony subset) into [PATH].
2. Remember the directory of your own images as [DIR].

   If you would like to speed up, you could process the RAW image with .ARW postfix into numpy array follow the Convert Your Own RAW Images to Numpy for Acceleration section in demo.md.
   And add -a option in command.

3. Try DNF on your images!

   bash demos/images_process.sh -p [PATH] -d [DIR] -r [RATIO]
   # [RATIO] denotes the additional digital gain you would like to add on your images.
   # If your data lies in '.npy' format, you should add '-a' arguments.
   bash demos/images_process.sh -p [PATH] -d [DIR] -r [RATIO] -a  # for data in numpy format
   
   # Let's see a simple example. 
   bash demos/images_process.sh -p pretrained/dnf_sony.pth -d dataset/sid/Sony/short_pack -r 100
   # The above command would try our pretrained DNF on the SID Sony subset with additional digital gain 100.

4. Check your results in runs/CVPR_DEMO/image_demo/results/inference!

Try DNF on your own RAW video clips (with RGGB Bayer pattern) !

1. Download the pretrained DNF (trained on SID Sony subset) into [PATH].
2. Preprocess your RAW video clip and save each frame into [DIR] with .npy format.

   You could follow the steps in Convert Your Own Video section of demo.md.

3. Try DNF on your video clip!

   bash demos/video_process.sh -d [DIR] -p [PATH] -r [RATIO] -s [SAVE_PATH] -f [FILE_NAME]
   # [RATIO] denotes the additional digital gain you would like to add on your images, Default: 50.
   # [SAVE_PATH] and [FILE_NAME] determine where to save the result.
   
   # Let's see a simple example. 
   bash demos/video_process.sh \
     -d dataset/campus/short_pack \
     -p pretrained/dnf_sony.pth \
     -r 50 \
     -s runs/videos -f campus
   # The above command would result in a 24 fps video postprocessed by our DNF with additional digital gain 50. 

4. Check result in [SAVE_PATH]/[FILE_NAME].mp4.

Try with data provided by us

Please refer to demo.md to learn how to download the provided data and how to inference.

Training and Evaluation

Please refer to benchmark.md to learn how to benchmark DNF, how to train a new model from scratch.

Attention! Due to the presence of three misaligned images in the SID Sony dataset (with scene IDs 10034, 10045, and 10172), our testing results in the article are based on excluding these images from the dataset. The txt file used for testing can be found and downloaded from the Google Drive (Sony_new_test_list.txt).

Citation

If you find our repo useful for your research, please consider citing our paper:

@inproceedings{jincvpr23dnf,
    title={DNF: Decouple and Feedback Network for Seeing in the Dark},
    author={Jin, Xin and Han, Linghao and Li, Zhen and Chai, Zhi and Guo, Chunle and Li, Chongyi},
    journal={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    year={2023}
}

License

This code is licensed under the Creative Commons Attribution-NonCommercial 4.0 International for non-commercial use only. Please note that any commercial use of this code requires formal permission prior to use.

Contact

For technical questions, please contact xjin[AT]mail.nankai.edu.cn and lhhan[AT]mail.nankai.edu.cn.

For commercial licensing, please contact cmm[AT]nankai.edu.cn

Acknowledgement

This repository borrows heavily from BasicSR and Learning-to-See-in-the-Dark.