OA-DG: Object-Aware Domain Generalization

OA-DG is an effective method for single-domain object detection generalization (S-DGOD). It consists of two components: OA-Mix for data augmentation and OA-Loss for reducing domain gaps.

Object-Aware Domain Generalization for Object Detection, Wooju Lee^* , Dasol Hong^* , Hyungtae Lim^†, and Hyun Myung^†, AAAI 2024 (arXiv:2312.12133)

✨Features

• OA-DG consists of OA-Mix for data augmentation and OA-Loss for reducing the domain gap.

• OA-Mix increases image diversity while preserving important semantic feature with multi-level transformations and object-aware mixing.

  👀 View some example images

  

• OA-Loss reduces the domain gap by training semantic relations of foreground and background instances from multi-domain.

• Extensive experiments on standard benchmarks (Cityscapes-C and Diverse Weather Dataset) show that OA-DG outperforms SOTA methods on unseen target domains.

• OA-DG can be generally applied to improve robustness regardless of the augmentation set and object detector architectures.

🚣 Getting Started

Follow these steps to set up the project on your local machine for training and testing.

Prerequisites

Ensure you have the following prerequisites installed on your local system.

1. Install mmdetection: There are several installation guides. Follow one of the below:

   Our code is forked from mmdetection 2.28.x version.

   a. Customize Installation (recommended)

   # Install MMCV using MIM.
   
   $ pip install -U openmim
   $ mim install mmcv-full
   
   # Clone this repository
   $ git clone https://github.com/WoojuLee24/OA-DG.git
   
   # Go into the repository
   $ cd OA-DG
   
   # Install mmdetection
   $ pip install -v -e .

   b. Refer to the mmdetection's installation instructions.

   c. Use Dockerfile from mmdetection to setup the environment.

2. Install other dependencies

   # For image processing operations.
   $ pip install Pillow
   # For spectral-residual saliency algorithm in OA-Mix.
   $ pip install opencv-python
   $ pip install opencv-contrib-python

3. Prepare the following datasets:

   • Cityscapes: A dataset that contains urban street scenes from 50 cities with detailed annotations.
   • Diverse Weather Dataset: This dataset includes various weather conditions for robust testing and development of models, essential for applications in autonomous driving.

🏃 How To Run

Training

python3 -u tools/train.py $(CONFIG).py --work-dir $(WORK_DIR)

Example: OA-DG trained on Cityscapes dataset

python3 -u tools/train.py configs/oadg/cityscapes/oadg.py --work-dir data/oadg/cityscapes/oadg

Evaluation

• Cityscapes-C

   python3 -u tools/analysis_tools/test_robustness.py \
     $(CONFIG).py $(CKPT_FILE).pth --out $(OUT_PATH).pkl \
     --corruptions benchmark --eval bbox

  Example: OA-DG evaluated on Cityscapes-C dataset

   python3 -u tools/analysis_tools/test_robustness.py \
     configs/oadg/cityscapes/oadg.py \
     data/oadg/cityscapes/oadg.pth \ 
     --out data/oadg/cityscapes/oadg.pkl \
     --corruptions benchmark --eval bbox

• Diverse Weather Dataset (DWD) (TODO)

   python3 -u tools/test_dwd.py \
     $(CONFIG).py $(CKPT_FILE).pth --out $(OUT_PATH).pkl \
     --eval mAP

  Example: OA-DG evaluated on DWD dataset

   python3 -u tools/analysis_tools/test_robustness.py \
     configs/oadg/cityscapes/oadg.py \
     data/oadg/cityscapes/oadg.pth \ 
     --out data/oadg/cityscapes/oadg.pkl \
     --corruptions benchmark --eval bbox

Demo

• You can also run demo/image_demo.py, which is the script provided by open-mmlab/mmdetection

  Example: Object detection results in demo image using OA-DG

  python3 demo/image_demo.py demo/demo.jpg \
     configs/oadg/cityscapes/oadg.py \
     --weights data/oadg/cityscapes/oadg.pth

📢 License

Please see the LICENSE.md file.

📫 Contact Information

If you have any questions, please do not hesitate to contact us:

• Wooju Lee ✉️ dnwn24@kaist.ac.kr
• Dasol Hong ✉️ ds.hong@kaist.ac.kr

📎 Citation

@misc{lee2023objectaware,
      title={Object-Aware Domain Generalization for Object Detection}, 
      author={Wooju Lee and Dasol Hong and Hyungtae Lim and Hyun Myung},
      year={2023},
      eprint={2312.12133},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}