Paper - SS-SFDA: Semi Supervised Source Free Domain Adaptation for Road Segmentation for Road Segmentation in Hazardous Environments

Project Page - https://gamma.umd.edu/researchdirections/autonomousdriving/weathersafe/

Watch the entire video here

Please cite our paper if you find it useful.

@article{kothandaraman2020ss,
  title={SS-SFDA: Self-Supervised Source-Free Domain Adaptation for Road Segmentation in Hazardous Environments},
  author={Kothandaraman, Divya and Chandra, Rohan and Manocha, Dinesh},
  journal={arXiv preprint arXiv:2012.08939},
  year={2020}
}

Table of Contents

• Paper - SS-SFDA: Semi Supervised Source Free Domain Adaptation for Road Segmentation for Road Segmentation in Hazardous Environments
• Repo Details and Contents
  • Code structure
  • Testing a pretrained model
  • Training your own model
  • Datasets
  • Dependencies
• Our network
• Acknowledgements

Repo Details and Contents

Python version: 3.7

Code structure

Dataloaders

Dataset	Dataloader	List of images
Clear weather CityScapes	dataset/cityscapes.py	dataset/cityscapes_list (train_images, val_images, train_labels, val_images)
Synthetic Fog	dataset/cityscapes_fog.py	dataset/cityscapes_list (train_rain_fog, val_rain_fog)
Synthetic Rain	dataset/cityscapes_rain.py	dataset/cityscapes_list (train_rain_fog, val_rain_fog)
Synthetic Rain	dataset/cityscapes_rain.py	dataset/cityscapes_list (train_rain_fog, val_rain_fog)
Real Fog - Foggy Zurich	dataset/foggy_zurich/train(test).py	dataset/foggy_zurich/lists_file_names
Real, Night Driving - Dark Zurich	dataset/dark_zurich/train(test).py	dataset/dark_zurich/lists_file_names
Heterogeneous Real, Rain + Night - Raincouver	dataset/raincouver/raincouver.py	dataset/raincouver (train_rain_fog, val_rain_fog)
Heterogeneous Real, Berkeley Deep Drive	dataset/bdd/bdd_{train,val}.py	dataset/bdd/bdd_list

Models

model/drnd38.py - DRN-D-38 model
model/drnd38_attention.py - DRN-D-38 model with self-attention

Pretrained models

The pretrained models can be found here. Use eval.py to test them.

Our network

Training your own model

Stage 1: The network is pre-trained on a clear weather dataset such as CityScapes.

python train_stage1.py

Stage 2: SS-SFDA

I. Arrange the images in the increasing order of intensity (of rain/fog/snow/light), and divide into minibatches - provided by the dataset in most cases.
II. For each minibatch:
a. Initialize the network with weights from the minibatch (For the first minibatch, initialize from Stage 1)
b. python train_stage2step1.py 
c. python train_stage2step3.py. 

Stage 3: Further finetuning for heterogeneous real datasets.

I. Pick a small subset (random) of images (of the order of 5-10).
II. Initialize the network with Stage 2 weights.
III. Finetune the network with knowledge distillation
python train_stage3.py.

Evaluation:

python eval.py

Make sure to set appropriate paths to the folders containing the datasets, and the models in the training and evaluation files.

Datasets

• Clear weather: CityScapes
• Synthetic: Rain and Fog
• Real dataset (night): Dark Zurich
• Real dataset (fog): Foggy Zurich
• Heterogeneous Real dataset (rain+night): Raincouver
• Heterogeneous Real dataset (multiple weather, lighting conditions): Berkeley Deep Drive

Dependencies

PyTorch
NumPy
SciPy
Matplotlib

Acknowledgements

This code is heavily borrowed from AdaptSegNet, and SAGAN