Low-Light Image and Video Enhancement: A Comprehensive Survey and Beyond

ArXiv

Title

Low-Light Image and Video Enhancement: A Comprehensive Survey and Beyond
Shen Zheng, Yiling Ma, Jinqian Pan, Changjie Lu, Gaurav Gupta

Updates

• 2024/1/1: We update the arXiv version with important revisions.
• 2023/4/16: The enhanced images and the metric scripts have been uploaded.
• 2023/3/24: The Night Wenzhou Dataset has been uploaded.
• 2023/2/8: The arXiv has been updated. The current version contains 21 pages, 9 tables, and 25 figures!

Highlights

• Present a comprehensive survey of low-light image and videeo enhancement (LLIE).
• Propose SICE_Grad and SICE_Mix image datasets to represent complex mixed over-/under-exposed scenes.
• Introduce Night Wenzhou video dataset that features fast-moving aerial scenes and streetscapes with varied illuminations and degradations.

Timeline

Taxonomy

Category

Traditional Learning	Deep Learning

Our Datasets

SICE_Grad and SICE_Mix [Download]	Night Wenzhou [Download]

Models (in chronological order)

• PIE (TIP 2015) [paper] [Python]

  • A Probabilistic Method for Image Enhancement With Simultaneous Illumination and Reflectance Estimation

• LIME (TIP 2016) [paper] [Python]

  • LIME: Low-Light Image Enhancement via Illumination Map Estimation

• LLNet (PR 2017) [paper] [Theano]

  • LLNet: A Deep Autoencoder Approach to Natural Low-light Image Enhancement

• MBLLEN (BMVC 2017) [paper] [Keras]

  • MBLLEN: Low-light Image/Video Enhancement Using CNNs

• LightenNet (PRL 2018) [paper] [MATLAB]

  • LightenNet: A Convolutional Neural Network for weakly illuminated image enhancement

• Retinex-Net (BMVC 2018) [paper] [TensorFlow]

  • Deep Retinex Decomposition for Low-Light Enhancement

• SID (CVPR 2018) [paper] [TensorFlow]

  • Learning to See in the Dark

• DeepUPE (CVPR 2019) [paper] [TensorFlow]

  • Underexposed Photo Enhancement using Deep Illumination Estimation

• EEMEFN (AAAI 2019) [paper] [TensorFlow]

  • EEMEFN: Low-Light Image Enhancement via Edge-Enhanced Multi-Exposure Fusion Network

• ExCNet (ACMMM 2019) [paper] [Tensorflow ]

  • Zero-shot restoration of back-lit images using deep internal learning

• KinD (ACMMM 2019) [paper] [TensorFlow]

  • Kindling the darkness: A practical low-light image enhancer

• Zero-DCE (CVPR 2020) [paper] [PyTorch]

  • Zero-reference deep curve estimation for low-light image enhancement

• DRBN (CVPR 2020) [paper] [PyTorch]

  • From Fidelity to Perceptual Quality: A Semi-Supervised Approach for Low-Light Image Enhancement

• Xu et al. (CVPR 2020) [paper] [PyTorch]

  • Learning to Restore Low-Light Images via Decomposition-and-Enhancement

• DLN (TIP 2020) [paper] [PyTorch]

  • Lightening network for low-light image enhancement

• DeepLPF (CVPR 2020) [paper] [PyTorch]

  • Deep Local Parametric Filters for Image Enhancement

• EnlightenGAN (TIP 2021) [paper] [PyTorch]

  • EnlightenGAN: Deep Light Enhancement without Paired Supervision

• KinD++ (IJCV 2021) [paper] [TensorFlow]

  • Beyond Brightening Low-light Images

• Zero-DCE++ (TPAMI 2021) [paper] [PyTorch]

  • Learning to enhance low-light image via zero-reference deep curve estimation

• Zhang et al. (CVPR 2021) [paper] [PyTorch]

  • Learning Temporal Consistency for Low Light Video Enhancement from Single Images

• RUAS (CVPR 2021) [paper] [PyTorch]

  • Retinex-inspired Unrolling with Cooperative Prior Architecture Search for Low-light Image Enhancement

• UTVNet (ICCV 2021) [paper] [PyTorch]

  • UTVNet: Adaptive Unfolding Total Variation Network for Low-Light Image Enhancement

• SDSD (ICCV 2021) [paper] [PyTorch]

  • Seeing Dynamic Scene in the Dark: A High-Quality Video Dataset with Mechatronic Alignment

• RetinexDIP (TCSVT 2021) [paper] [PyTorch]

  • RetinexDIP: A Unified Deep Framework for Low-light Image Enhancement

• SGZ (WACV 2022) [paper] [PyTorch]

  • Semantic-Guided Zero-Shot Learning for Low-Light Image/Video Enhancement

• LLFlow (AAAI 2022) [paper] [PyTorch]

  • Low-Light Image Enhancement with Normalizing Flow

• SNR-Aware (CVPR 2022) [paper] [PyTorch]

  • SNR-Aware Low-light Image Enhancement

• SCI (CVPR 2022) [paper] [PyTorch]

  • Toward Fast, Flexible, and Robust Low-Light Image Enhancement

• URetinex-Net (CVPR 2022) [paper] [PyTorch]

  • URetinex-Net: Retinex-based Deep Unfolding Network for Low-light Image Enhancement

• Dong et al. (CVPR 2022) [paper] [PyTorch]

  • Abandoning the Bayer-Filter to See in the Dark

• MAXIM (CVPR 2022) [paper] [Jax]

  • MAXIM: Multi-Axis MLP for Image Processing

• BIPNet (CVPR 2022) [paper] [PyTorch]

  • Burst Image Restoration and Enhancement

• LCDPNet (ECCV 2022) [paper] [PyTorch]

  • Local Color Distributions Prior for Image Enhancement

• IAT (BMVC 2022) [paper] [PyTorch]

  • You Only Need 90K Parameters to Adapt Light: A Light Weight Transformer for Image Enhancement and Exposure Correction

Benchmark Datasets

• NPE, LIME, MEF, DICM, VV [link]

• LOL [link]

• VE-LOL [link]

• ACDC [link]

• DCS [link]

• DarkFace [link]

• ExDark [link]

• SICE [link]

Enhanced Images

• Enhanced images for all baselines are here

Metrics

Full-Reference

• PSNR

• SSIM

• LPIPS

Non-Reference

• UNIQUE

• BRISQUE

• SPAQ

Subjective

• User Study

Efficiency

• FLOPs

• #Params

• Run Time

Surveys

• IEEE [paper]

  • An experiment-based review of low-light image enhancement methods

• IJCV 2021 [paper]

  • Benchmarking low-light image enhancement and beyond

• TPAMI 2021 [paper]

  • Low-Light Image and Video Enhancement Using Deep Learning: A Survey

Related Repositories

• [Low-Light Image and Video Enhancement Using Deep Learning: A Survey]

• [Awesome Low Light Image Enhancement]

• [Awesome-Low-Light-Enhancement]

BibTeX

If you find this repository helpful, please cite our paper.

@article{zheng2022low,
  title={Low-Light Image and Video Enhancement: A Comprehensive Survey and Beyond},
  author={Zheng, Shen and Ma, Yiling and Pan, Jinqian and Lu, Changjie and Gupta, Gaurav},
  journal={arXiv preprint arXiv:2212.10772},
  year={2022}
}