Skip to content
Switch branches/tags
Go to file

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time


We have made the detectron2-based code public. But there are still some bugs in it. We will fix them as soon as possible.


This repository is for SOGNet: Scene Overlap Graph Network for Panoptic Segmentation which has been accepted by AAAI2020 and won the Innovation Award in COCO 2019 challenge,

by Yibo Yang, Hongyang Li, Xia Li, Qijie Zhao, Jianlong Wu, Zhouchen Lin

This repo is modified from UPSNet. We have been transfering the code into detectron2 framework. Not finished yet.


The panoptic segmentation task requires a unified result from semantic and instance segmentation outputs that may contain overlaps. However, current studies widely ignore modeling overlaps. In this study, we aim to model overlap relations among instances and resolve them for panoptic segmentation. Inspired by scene graph representation, we formulate the overlapping problem as a simplified case, named scene overlap graph. We leverage each object's category, geometry and appearance features to perform relational embedding, and output a relation matrix that encodes overlap relations. In order to overcome the lack of supervision, we introduce a differentiable module to resolve the overlap between any pair of instances. The mask logits after removing overlaps are fed into per-pixel instance id classification, which leverages the panoptic supervision to assist in the modeling of overlap relations. Besides, we generate an approximate ground truth of overlap relations as the weak supervision, to quantify the accuracy of overlap relations predicted by our method. Experiments on COCO and Cityscapes demonstrate that our method is able to accurately predict overlap relations, and outperform the state-of-the-art performance for panoptic segmentation. Our method also won the Innovation Award in COCO 2019 challenge.



  • Pytorch1.0 or above and Python 3 are needed.
  • Run first for getting the pretrained models and panopticapi and compiling the project.
  • Run for generating the ground truth file. Note that we use the panoptic annotation for our panoptic head, which is different from UPSNet.


python tools/ --cfg PATH_TO_CONFIG_FILE.yaml


python tools/ --cfg PATH_TO_CONFIG_FILE.yaml --weight_path PATH_TO_CHECKPOINT.pth

We provide our best model based on resnet50, you can download here.

test split PQ SQ RQ PQ_th PQ_st
SOGNet-50 val 43.7 78.7 53.5 50.6 33.1


  • Reimplement the code in detectron2.


If you find SOGNet useful in your research, please consider citing:

 author={Yibo Yang, Hongyang Li, Xia Li, Qijie Zhao, Jianlong Wu, Zhouchen Lin},
 title={SOGNet: Scene Overlap Graph Network for Panoptic Segmentation},
 journaltitle = {{arXiv}:1911.07527 [cs]},


SOGNet: Scene Overlap Graph Network for Panoptic Segmentation




No releases published


No packages published