Skip to content

joe-siyuan-qiao/ViP-DeepLab

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
December 7, 2020 18:17
November 30, 2020 15:27
July 12, 2021 19:10
December 10, 2020 20:07

ViP-DeepLab

Introduction

In this repository, we present the datasets and the toolkits of ViP-DeepLab. ViP-DeepLab is a unified model attempting to tackle the long-standing and challenging inverse projection problem in vision, which we model as restoring the point clouds from perspective image sequences while providing each point with instance-level semantic interpretations. Solving this problem requires the vision models to predict the spatial location, semantic class, and temporally consistent instance label for each 3D point. ViP-DeepLab approaches it by jointly performing monocular depth estimation and video panoptic segmentation. We name this joint task as Depth-aware Video Panoptic Segmentation (DVPS), and propose a new evaluation metric along with two derived datasets for it. This repository includes the datasets SemKITTI-DVPS and Cityscapes-DVPS along with the evaluation toolkits.

Demo

Datasets

SemKITTI-DVPS

SemKITTI-DVPS is derived from SemanticKITTI dataset. SemanticKITTI dataset is based on the odometry dataset of the KITTI Vision benchmark. SemanticKITTI dataset provides perspective images and panoptic-labeled 3D point clouds. To convert it for DVPS, we project the 3D point clouds onto the image plane and name the derived dataset as SemKITTI-DVPS. SemKITTI-DVPS is distributed under Creative Commons Attribution-NonCommercial-ShareAlike license. The dataset and the evaluation toolkit are in the folder semkitti-dvps.

SemKITTI-DVPS example.

Cityscapes-DVPS

Cityscapes-DVPS is derived from Cityscapes-VPS by adding re-computed depth maps from Cityscapes dataset. Cityscapes-DVPS is distributed under Creative Commons Attribution-NonCommercial-ShareAlike license. The dataset and the evaluation toolkit are in the folder cityscapes-dvps.

Cityscapes-DVPS example.

Citation

If you use the datasets in your research, please cite our project.

@article{vip_deeplab,
  title={ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentation},
  author={Siyuan Qiao and Yukun Zhu and Hartwig Adam and Alan Yuille and Liang-Chieh Chen},
  journal={arXiv preprint arXiv:2012.05258},
  year={2020}
}

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages