This is the official codebase for Align and Distill: Unifying and Improving Domain Adaptive Object Detection.
[Project Page] [Arxiv] [PDF]
Align and Distill (ALDI) is a state-of-the-art framework for domain adaptive object detection (DAOD), built on top of the Detectron2 object detection library. ALDI is:
Accurate: ALDI's default settings achieve state-of-the-art results on DAOD benchmarks including Cityscapes → Foggy Cityscapes, Sim10k → Cityscapes, and CFC Kenai → Channel.
Fast to train: Thanks to efficient dataloading and optimized burn-in settings, ALDI trains upwards of 20x faster than other DAOD methods.
Easy to use: Training DAOD models on your own data requires just a couple steps; see setting up your own data and training ALDI.
Extensible: The framework is lightweight, reusing default components from Detectron2 wherever possible. See code documentation for an overview of the code structure and design decisions.
See installation instructions.
We provide burn-in checkpoints and final models for DAOD benchmarks (Cityscapes → Foggy Cityscapes, Sim10k → Cityscapes, and CFC Kenai → Channel) in the model zoo.
For compatibility with existing config files, download models to the models/ directory in this repo.
You can download the required model weights for any config file we provide using python tools/download_model_for_config.py --config-file path/to/config.yaml
Follow these instructions to set up data and reproduce benchmark results on Cityscapes → Foggy Cityscapes, Sim10k → Cityscapes, and CFC Kenai → Channel.
To use ALDI on your own data, see instructions for custom datasets.
The main branch contains all you need to run ALDI, and is a good starting point for most.
Additional code and configuration files to reproduce all experiments in our paper can be found on the extras branch.
Justin Kay, Timm Haucke, Suzanne Stathatos, Siqi Deng, Erik Young, Pietro Perona, Sara Beery, and Grant Van Horn.
Object detectors often perform poorly on data that differs from their training set. Domain adaptive object detection (DAOD) methods have recently demonstrated strong results on addressing this challenge. Unfortunately, we identify systemic benchmarking pitfalls that call past results into question and hamper further progress: (a) Overestimation of performance due to underpowered baselines, (b) Inconsistent implementation practices preventing transparent comparisons of methods, and (c) Lack of generality due to outdated backbones and lack of diversity in benchmarks. We address these problems by introducing: (1) A unified benchmarking and implementation framework, Align and Distill (ALDI), enabling comparison of DAOD methods and supporting future development, (2) A fair and modern training and evaluation protocol for DAOD that addresses benchmarking pitfalls, (3) A new DAOD benchmark dataset, CFC-DAOD, enabling evaluation on diverse real-world data, and (4) A new method, ALDI++, that achieves state-of-the-art results by a large margin. ALDI++ outperforms the previous state-of-the-art by +3.5 AP50 on Cityscapes → Foggy Cityscapes, +5.7 AP50 on Sim10k → Cityscapes (where ours is the only method to outperform a fair baseline), and +2.0 AP50 on CFC Kenai → Channel. Our framework, dataset, and state-of-the-art method offer a critical reset for DAOD and provide a strong foundation for future research.
If you find our work useful in your research please consider citing our paper:
@misc{kay2024align,
title={Align and Distill: Unifying and Improving Domain Adaptive Object Detection},
author={Justin Kay and Timm Haucke and Suzanne Stathatos and Siqi Deng and Erik Young and Pietro Perona and Sara Beery and Grant Van Horn},
year={2024},
eprint={2403.12029},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
