[PG 2023] WaveNet: Wave-Aware Image Enhancement
Jiachen Dang, Zehao Li, Yong Zhong, Lishun Wang
- 11.09, 2023: Training codes and logs have been released. Feel free to use them. 🚀
- 10 12, 2023: Codes and pre-trained models are released! ⭐
- 09 15, 2023: Our paper has been accepted by PG 2023. Code and Models will be released. 🎉
Abstract: * As a low-level vision task, image enhancement is widely used in various computer vision applications. Recently, multiple methods combined with CNNs, MLP, Transformer, and the Fourier transform have achieved promising results on image enhancement tasks. However, these methods cannot achieve a balance between accuracy and computational cost. In this paper, we formulate the enhancement into a signal modulation problem and propose the WaveNet architecture, which performs well in various parameters and improves the feature expression using wave-like feature representation. Specifically, to better capture wave-like feature representations, we propose to represent a pixel as a sampled value of a signal function with three wave functions (Cosine Wave (CW), Sine Wave (SW), and Gating Wave (GW)) inspired by the Fourier transform. The amplitude and phase are required to generate the wave-like features. The amplitude term includes the original contents of features, and the phase term modulates the relationship between various inputs and fixed weights. To dynamically obtain the phase and the amplitude, we build the Wave Transform Block (WTB) that adaptively generates the waves and modulates the wave superposition mode. Based on the WTB, we establish an effective architecture WaveNet for image enhancement. Extensive experiments on six real-world datasets show that our model achieves better quantitative and qualitative results than state-of-the-art methods.
Pipeline
See INSTALL.md for the installation of dependencies required to run WaveNet.
Test on the local environment:
To test the pre-trained models for enhancing your images, run
cd basicsr
python demo.py --input_dir images_folder_path --result_dir save_images_here --weights path_to_models
**All pre-trained models can be found in the folder "checkpoints/$<$dataset$>$"
Training for Image Enhancement is provided in TRAINING.md. Here is a summary table containing hyperlinks for easy navigation:
| Model | LOL | log | FiveK | log | SID | log |
|---|---|---|---|
| WaveNet-B | weights | log | weights | log | weights | log |
| WaveNet-S | weights | log | weights | log | weights | log |
| WaveNet-T | weights | log | weights | log | weights | log |
For the preparation of dataset, see datasets/README.md.
- To test the PSNR, SSIM and LPIPS of image enhancement, see evaluation.py and run
cd basicsr
python evaluation.py -dirA images_folder_path -dirB images_folder_path -type image_data_type --use_gpu use_gpu_or_not
Quantitative Evaluation on LOL/FiveK/VE-LOL/SID/SICE/DARK FACE datasets.
Qualitative Evaluation on LOL/FiveK/VE-LOL/SID/SICE/DARK FACE datasets.
Efficiency (Tested on a single RTX 3090, Pytorch 1.11)
@inproceedings {10.2312:pg.20231267,
booktitle = {Pacific Graphics Short Papers and Posters},
editor = {Chaine, Raphaëlle and Deng, Zhigang and Kim, Min H.},
title = {{WaveNet: Wave-Aware Image Enhancement}},
author = {Dang, Jiachen and Li, Zehao and Zhong, Yong and Wang, Lishun},
year = {2023},
publisher = {The Eurographics Association},
ISBN = {978-3-03868-234-9},
DOI = {10.2312/pg.20231267}
}
Should you have any questions, please contact dj.chen112@gmail.com
Acknowledgment: This code is based on the BasicSR toolbox.