Misalignment-Robust Frequency Distribution Loss for Image Transformation

The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

Zhangkai Ni¹, Juncheng Wu¹, Zian Wang¹, Wenhan Yang², Hanli Wang¹, Lin Ma³

¹Tongji University, ²Peng Cheng Laboratory, ³Meituan

This repository provides the official PyTorch implementation for the paper “Misalignment-Robust Frequency Distribution Loss for Image Transformation”, CVPR-2024. Paper

About FDL

This paper aims to address a common challenge in deep learning-based image transformation methods, such as image enhancement and super-resolution, which heavily rely on precisely aligned paired datasets with pixel-level alignments. However, creating precisely aligned paired images presents significant challenges and hinders the advancement of methods trained on such data. To overcome this challenge, this paper introduces a novel and simple Frequency Distribution Loss (FDL) for computing distribution distance within the frequency domain. Specifically, we transform image features into the frequency domain using Discrete Fourier Transformation (DFT). Subsequently, frequency components (amplitude and phase) are processed separately to form the FDL loss function. Our method is empirically proven effective as a training constraint due to the thoughtful utilization of global information in the frequency domain. Extensive experimental evaluations, focusing on image enhancement and super-resolution tasks, demonstrate that FDL outperforms existing misalignment-robust loss functions. Furthermore, we explore the potential of our FDL for image style transfer that relies solely on completely misaligned data.

TL;DR: We propose a novel Frequency Distribution Loss (FDL) for image transformation models trained with misaligned data, opening up new avenues for addressing the broad issue of misalignment in image transformation tasks.

Quick Start

Installation:

pip install fdl-pytorch

Requirements:

• Python>=3.6
• Pytorch>=1.0

Usage:

from FDL_pytorch import FDL_loss
fdl_loss = FDL_loss()
# X: (N,C,H,W) 
# Y: (N,C,H,W) 
loss_value = fdl_loss(X, Y)
loss_value.backward()

Citation

If you find our work useful, please cite it as

@article{ni2024misalignment,
  title={Misalignment-Robust Frequency Distribution Loss for Image Transformation},
  author={Ni, Zhangkai and Wu, Juncheng and Wang, Zian and Yang, Wenhan and Wang, Hanli and Ma, Lin},
  journal={arXiv preprint arXiv:2402.18192},
  year={2024}
}

Contact

Thanks for your attention! If you have any suggestion or question, feel free to leave a message here or contact Dr. Zhangkai Ni (eezkni@gmail.com).

License

MIT License