This repository is an official implementation of the BiADT (accepted to ICCV 2023). Code will be avaliable after the internal approval. [paper link]
[2023/8]Code will be released soon!
[2023/7]BiADT is accepted by ICCV 2023
Abstract: We propose a Bidirectional Alignment for domain adaptive Detection with Transformers (BiADT) to improve cross domain object detection performance. Existing adversarial learning based methods use gradient reverse layer (GRL) to reduce the domain gap between the source and target domains in feature representations. Since different image parts and objects may exhibit various degrees of domain-specific characteristics, directly applying GRL on a global image or object representation may not be suitable. Our proposed BiADT explicitly estimates token-wise domain-invariant and domain-specific features in the image and object token sequences. BiADT has a novel deformable attention and self-attention, aimed at bi-directional domain alignment and mutual information minimization. These two objectives reduce the domain gap in domain-invariant representations, and simultaneously increase the distinc- tiveness of domain-specific features. Our experiments show that BiADT achieves very competitive performance to SOTA consistently on Cityscapes-to-FoggyCityscapes, Sim10K-to-Citiscapes and Cityscapes-to-BDD100K, out- performing the strong baseline, AQT, by 2.0, 2.1, and 2.4 in mAP50, respectively.
We provide the model weights for C2FC domain shift. The model weights for BIADT w/o and w/ AQT can be found here: c2fc_biadt.pth (AP=49.6) and c2fc_aqt_biadt.pth(AP=50.1).
For the inference, see test.sh and test_aqt.sh. For training, see train.sh.
Our work is based on DAB-DETR (DN-DETR version) and AQT. We sincerely appreciate their great work!
-
AQT: Adversarial Query Transformers for Domain Adaptive Object Detection.
Wei-Jie Huang, Yu-Lin Lu, Shih-Yao Lin, Yusheng Xie, and Yen-Yu Lin.
IJCAI-ECAI 2022
[paper] [code]. -
DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR.
Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, Lei Zhang.
International Conference on Learning Representations (ICLR) 2022.
[Paper] [Code]. -
DN-DETR: Accelerate DETR Training by Introducing Query DeNoising.
Feng Li, Hao Zhang*, Shilong Liu, Jian Guo, Lionel M.Ni, and Lei Zhang.
IEEE/CVF conference on computer vision and pattern recognition (CVPR oral) 2022.
[Paper] [Code]
BiADT is released under the Apache 2.0 license. Please see the LICENSE file for more information.
If you find our work helpful for your research, please consider citing the following BibTeX entry.
@article{he2023bidirectional,
title={Bidirectional alignment for domain adaptive detection with transformers},
author={He, Liqiang and Wang, Wei and Chen, Albert and Sun, Min and Kuo, Cheng-Hao and Todorovic, Sinisa},
year={2023}
}