WUDA

WUDA：Unsupervised Domain Adaptation Based on Weak Source Domain Labels

Citation

Please cite this paper in your publications if it helps your research:

@INPROCEEDINGS{10094958,
  author={Liu, Shengjie and Zhu, Chuang and Li, Yuan and Tang, Wenqi},
  booktitle={ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, 
  title={WUDA: Unsupervised Domain Adaptation Based on Weak Source Domain Labels}, 
  year={2023},
  volume={},
  number={},
  pages={1-5},
  doi={10.1109/ICASSP49357.2023.10094958}}

Introduction

The acquisition of semantic labels is often time-consuming, many scenarios only have weak labels (e.g. bounding boxes). When weak supervision and cross-domain problems coexist, this paper defines a new task: unsupervised domain adaptation based on weak source domain labels (WUDA).

Frameworks

WSSS-UDA

We first propose the framework WSSS-UDA: in order to take advantage of the fine box labels in the source domain, first perform box-supervised segmentation in the source domain, then the problem is transformed into a UDA task.

Step 1: Box Supervised Segmentic Segmentation

Clone this repo.

git clone https://github.com/bupt-ai-cz/WUDA.git

Perform GrabCut on the source domain images to obtain pseudo-labels that can be used for supervised semantic segmentation training.

cd grabcut
python grabcut.py --txt_path path/to/bboxes --image_path path/to/images --save_path path/to/pseudo_labels

Train a semantic segmentation model using the pseudo-labels obtained from GrabCut. This stage corresponds to the warm-up phase of the unsupervised domain adaptation algorithm. We use HIAST in our work.

Setp 2: Unsupervised Domain Adaptation For Semantic Segmentation

This step corresponds to the adversarial training or self-training (or both) stage in unsupervised domain adaptation for semantic segmentation algorithms. Please refer to the second stage (adversarial training) and the third stage (self-training) of HIAST for more details.

We also benchmark other UDA algorithms within framework WSSS-UDA: CBST, CPSL.

TDOD-WSSS

we also implement the cross-domain process on the object detection task and propose the framework TDOD-WSSS (Figure 4(b)): First, use the bounding box labels of the source domain to train the object detection model, then predict bounding boxes on the target domain. Finally, implement box-supervised segmentation in the target domain.

Step 1: Target Domain Object Detection

In this step, it is required to train an object detection model using the source domain data and provide predictions on the target domain images.

We benchmark Yolov5 within this framework.

Step 2: Box Supervised Segmentic Segmentation

Please refer to Step 2 in the framework WSSS-UDA.

Main Results

Representation Shift and Constructed Dataset

Representation Shift

Calculate Representation Shift

Firstly, train a DeepLabv3 model using the source domain dataset.

cd representation shift
python train.py --image_root path/to/image --mask_root path/to/label

Secondly, calculate the representations of the source domain dataset and target domain dataset with the trained model.

python cal_representation.py --dataset_path path/to/dataset --model_path path/to/model --save_path path/to/representation --H 1024 --W 2048

Thirdly, calculate the representation shift of the two datasets.

python cal_representation_shift.py --representationA_path path/to/representationA --representationB_path path/to/representationB

Constructed Dataset

We construct a series of datasets with different domain shifts and further analyze the impact of multiple domain shifts on the two frameworks.