RT-K-Net: Revisiting K-Net for Real-Time Panoptic Segmentation

This repository contains the official implementation of our IV 2023 paper RT-K-Net: Revisiting K-Net for Real-Time Panoptic Segmentation.

Installation

See INSTALL.md for instructions on how to prepare your environment to use RT-K-Net.

Usage

See datasets/README.md for instructions on how to prepare datasets for RT-K-Net.

See GETTING_STARTED.md for instructions on how to train and evaluate models, or run inference on demo images.

Model Zoo

The model files provided below are made available under the CC BY-NC-SA 4.0 license. The Cityscapes model was trained using 4 NVIDIA 2080Ti GPUs, the Mapillary Vistas model was trained using 8 NVIDIA 2080Ti GPUs.

Name	PQ	PQ_St	PQ_Th	Download
RT-K-Net Cityscapes Fine	60.2	66.5	51.5	model
RT-K-Net Mapillary Vistas	33.2	45.8	23.6	model

Reference

Please use the following citations when referencing our work:

RT-K-Net: Revisiting K-Net for Real-Time Panoptic Segmentation (IV 2023)
Markus Schön, Michael Buchholz and Klaus Dietmayer, [arxiv]

@InProceedings{Schoen_2023_IV,
    author    = {Sch{\"o}n, Markus and Buchholz, Michael and Dietmayer, Klaus},
    title     = {RT-K-Net: Revisiting K-Net for Real-Time Panoptic Segmentation},
    booktitle = {IEEE Intelligent Vehicles Symposium},
    year      = {2023}
}

Acknowledgement

We used and modified code parts from other open source projects, we especially like to thank the authors of:

• detectron2
• K-Net
• Mask2Former