Skip to content

jamycheung/ISSAFE

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

ISSAFE & EDCNet

PWC
PWC
PWC
PWC

Introduction

This is the implementation of our papers below, including the code and dataset (DADA-seg).

ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data, IROS 2021, [paper].

Exploring Event-Driven Dynamic Context for Accident Scene Segmentation, T-ITS 2021, [paper].

Demo

issafe_demo

Updates

  • [04/12/2022] Generated event data release.
  • [12/27/2021] Initial release.

Installation

The requirements are listed in the requirement.txt file. To create your own environment, an example is:

conda create -n issafe python=3.7
conda activate issafe
cd /path/to/ISSAFE
pip install -r requirement.txt

Datasets

For the basic setting of this work, please prepare datasets of Cityscapes, and DADA-seg.

Our proposed DADA-seg dataset is a subset from DADA-2000.

The DADA-seg dataset and event data of Cityscapes are now available:

Dataset Image & label Event Label-only
DADA-seg (val) 46.8MB [Baidu Drive] 401.2MB [Baidu Drive] 2.7MB [Google Drive]
Cityscapes (train + val) - 7.5GB [Baidu Drive] -

Note: The event data is generated by EventGAN. The anchor and its previous frames are needed.

A structure of dataset should be:

dataset
├── Cityscapes
│   ├── event
│   │   ├── train
│   │   │   ├─aachen
│   │   │   │  ├─aachen_000000_000019_gtFine_event.npz	# event volume
│   │   └── val
│   ├── gtFine
│   │   ├── train
│   │   └── val
│   ├─leftImg8bit_prev # for event synthesic
│   │   ├─train
│   │   │  ├─aachen
│   │   │  │  ├─aachen_000000_000019_leftImg8bit_prev.png
│   │   └─val
│   ├── leftImg8bit
│   │   ├── train
│   └── └── val
└── DADA_seg
   ├── dof
   │   └── val
   ├── event
   │   └── val
   ├── gtFine
   │   └── val
   └── leftImg8bit
       ├── train
       └── val

(optional) other dataset sources used in EDCNet: BDD3K, KITTI-360, ApolloScape.

(optional) other modalities: dense optical flow.

Training

The model of EDCNet can be found at models/edcnet.py.

Before run the training script, please modify your own path configurations at mypath.py.

The training configurations can be adjusted at train.py.

An example of training is python train.py

Evaluation

The evaluation configurations can be adjusted at eval.py.

To achieve the evaluation result of EDCNet in D2S mode with 2 event time bins, the weights can be downloaded in Google Drive.

Put the weight at run/cityscapesevent/test_EDCNet_r18/model_best.pth.

An example of evaluation of the EDCNet at B=2 event time bins is python eval.py.

License

This repository is under the Apache-2.0 license. For commercial use, please contact with the authors.

Citation

If you are interested in this work, please cite the following work:

@INPROCEEDINGS{zhang2021issafe,
  author={Zhang, Jiaming and Yang, Kailun and Stiefelhagen, Rainer},
  booktitle={2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, 
  title={ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data}, 
  year={2021},
  pages={1132-1139},
  doi={10.1109/IROS51168.2021.9636109}}
  
@ARTICLE{zhang2021edcnet,
  author={Zhang, Jiaming and Yang, Kailun and Stiefelhagen, Rainer},
  journal={IEEE Transactions on Intelligent Transportation Systems}, 
  title={Exploring Event-Driven Dynamic Context for Accident Scene Segmentation}, 
  year={2021},
  pages={1-17},
  doi={10.1109/TITS.2021.3134828}}

About

Improving Semantic Segmentation in Accidents by Fusing Event-based Data

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages