Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time


GAIS-Net: Geometry-Aware Instance Segmentation with Disparity Maps

Cho-Ying Wu, Xiaoyan Hu, Michael Happold, Quangeng Xu, Ulrich Neumann, CVPR Workshop on Scalability in Autonomous Driving, 2020.


πŸ‘ The first work to directly regress instances from depth maps to advance the multi-modal learning for outdoor scenarios.

πŸ‘ High performance and highly modulized. The codes are based on the mask-rcnn benchmark framework.

Please visit out Project site for paper and intorduction.

This project is developed upon Mask-RCNN and is done during an internship at Argo AI

Feature: Resolve overlapping areas between instances by introducing geometry information


Check for installation instructions.

Note that cocoapi, cityscapesScripts, and apex are needed for the evaluation.


The geometry-aware fusion module is implemented under

Check these files and related files for the features we implement.


  1. The pre-generated stereo disparity maps by PSMNet for Cityscapes could be downloaded here. Please first create folders "datasets/cityscapes" at the root of the repo and extract the zip file under datasets/cityscapes/

  2. Go to Cityscapes to download images of train/val set. Put the images under "datasets/cityscapes/train/image_2" and also "val". Download the annotations and put under datasets/cityscapes/annotations/


NUM_GPUS=4 python -m torch.distributed.launch --nproc_per_node=$NUM_GPUS ./tools/ --config-file "configs/cityscapes_v4.yaml" TEST.IMS_PER_BATCH 4

Pretrain model

The pretrained weights could be downloaded here put the .pth file under "ckpt/"



If you find the work useful, please condisder to cite:

title={Geometry-Aware Instance Segmentation with Disparity Maps},
author={Wu, Cho-Ying and Hu, Xiaoyan and Happold, Michael and Xu, Qiangeng and Neumann, Ulrich},
booktitle={CVPR Workshop on Scability in Autonomous Driving},

Thanks to the Third Parties

Thank Mask-RCNN and Mask Scoring RCNN