PromptRR is a diffusion-based model that uses frequency information as new visual prompts for single-image reflection removal.
📖 For more visual results, see our project page
Current single image reflection removal (SIRR) methods using deep learning tend to miss key low-frequency (LF) and high-frequency (HF) differences in images, affecting their effectiveness in removing reflections. To address this problem, this paper proposes a novel prompt-guided reflection removal (PromptRR) framework that uses frequency information as new visual prompts for better performance. Specifically, the proposed framework decouples the reflection removal process into the prompt generation and subsequent prompt-guided restoration. For the prompt generation, we first propose a prompt pre-training strategy to train a frequency prompt encoder that encodes the ground-truth image into LF and HF prompts. Then, we adopt diffusion models (DMs) as prompt generators to generate the LF and HF prompts estimated by the pre-trained frequency prompt encoder. For the prompt-guided restoration, we integrate specially generated prompts into the PromptFormer network, employing a novel Transformer-based prompt block to effectively steer the model toward enhanced reflection removal. The results on commonly used benchmarks show that our method outperforms state-of-the-art approaches.
- [2024.2.06] This repo is created and the code will come soon!
---
If our PromptRR helps your research or work, please consider citing:
@article{wang2024promptrr,
title={PromptRR: diffusion models as prompt generators for single image reflection Removal},
author={Wang, Tao and Lu Wanglong and Zhang, Kaihao and and Luo, Wenhan and and Kim, Tae-Kyun and Lu Tong and Li, Hongdong and Ming-Hsuan Yang},
journal={arXiv preprint arXiv:2402.02374},
year={2024}
}
If you have any questions, please reach us at taowangzj@gmail.com or wanglongl@mun.ca.