Harmonizer: High-Resolution Image/Video Harmonization

Harmonizer: Learning to Perform White-Box Image and Video Harmonization (ECCV 2022)

Paper | Demo | Code | License | Citation | Contact

Harmonizer Result Video | Enhancer Result Video

Harmonizer is a lightweight (20MB) model enabled image/video harmonization up to 8K resolution.

With GPUs, Harmonizer has real-time performance at Full HD resolution.

Enhancer is a model applied the Harmonizer architecture for image/video color enhancement.

---

Demo

In our demos, the Harmonizer model is trained on the iHarmony4 dataset, while the Enhancer model is trained on the FiveK + HDRPlus datasets.

Online Demo

Try our online demos for fun without code!

Image Harmonization	Image Enhancement
Online Demo	Online Demo

Offline Demo

We provide offline demos for image/video harmonization/enhancement.

Image Harmonization	Video Harmonization	Image Enhancement	Video Enhancement
Offline Demo	Offline Demo	Offline Demo	Offline Demo

Code

Training

The training code is released in the folder ./src/train.
Refer to README.md for more details about training.

Validation

We provide PyTorch validation code to reproduce the iHarmony4 results reported in our paper, please:

1. Download the Harmonizer model pre-trained on the iHarmony4 dataset from this link and put it in the folder ./pretrained.

2. Download the four subsets of iHarmony4 from this repository and put them in the folder ./dataset/harmonization/iHarmony4.

3. Install python requirements. In the root path of this repository, run:

   pip install -r src/requirements.txt
   

4. For validation, in the root path of this repository, run:

   python -m src.val_harmonizer \
          --pretrained ./pretrained/harmonizer \
          --datasets HCOCO HFlickr HAdobe5k Hday2night \
          --metric-size 256
   

   • You can change --datasets to validate a specific subset.
   • You can remove --metric-size to calculate the metrics without resizing the outputs.
   • The metric values may slightly different from our paper due to the dependency versions.

License

This project is released under the Creative Commons Attribution NonCommercial ShareAlike 4.0 license.

Citation

If this work helps your research, please consider to cite:

@InProceedings{Harmonizer,
  author = {Zhanghan Ke and Chunyi Sun and Lei Zhu and Ke Xu and Rynson W.H. Lau},
  title = {Harmonizer: Learning to Perform White-Box Image and Video Harmonization},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year = {2022},
}

Contact

This repository is maintained by Zhanghan Ke (@ZHKKKe).
For questions, please contact kezhanghan@outlook.com.