PanopticFCN

Training and evaluating on Cityscapes

In this repo, we apply Panoptic FCN to Cityscapes.

Doing:

• Adding data pipeline for Cityscapes (adapted from original Detectron2 code)
• Adapting config to Cityscapes settings (config here)
• Reproducing results from paper (training + evaluation)

To be done:

• Saving qualitative results
• Adding image summaries in Tensorboard

Current status:

Method	Backbone	LR	Batch size	Iters	PQ	PQ_th	PQ_st	config	download
PanopticFCN	R50	0.02	32	65k	59.0	51.3	64.6	config	TBD

Original

Fully Convolutional Networks for Panoptic Segmentation

Yanwei Li, Hengshuang Zhao, Xiaojuan Qi, Liwei Wang, Zeming Li, Jian Sun, Jiaya Jia

[arXiv] [BibTeX]

This project provides an implementation for the CVPR 2021 Oral paper "Fully Convolutional Networks for Panoptic Segmentation" based on Detectron2. Panoptic FCN is a conceptually simple, strong, and efficient framework for panoptic segmentation, which represents and predicts foreground things and background stuff in a unified fully convolutional pipeline.

Installation

This project is based on Detectron2, which can be constructed as follows.

• Install Detectron2 following the instructions.
• Setup the dataset following the structure.
• Copy this project to /path/to/detectron2/projects/PanopticFCN

Training

To train a model with 8 GPUs, run:

cd /path/to/detectron2
python3 projects/PanopticFCN/train.py --config-file <config.yaml> --num-gpus 8

For example, to launch PanopticFCN training on Cityscapes with ResNet-50 backbone on 4 GPUs, one should execute:

cd /path/to/detectron2
python3 projects/PanopticFCN_cityscapes/train.py --config-file projects/PanopticFCN_cityscapes/configs/cityscapes/PanopticFCN-R50-cityscapes.yaml --num-gpus 4

Evaluation

To evaluate a pre-trained model with 8 GPUs, run:

cd /path/to/detectron2
python3 projects/PanopticFCN/train.py --config-file <config.yaml> --num-gpus 8 --eval-only MODEL.WEIGHTS /path/to/model_checkpoint

Results

Reproduced Cityscapes results will be presented below.

Method	Backbone	Sched	PQ	SQ	RQ	AP	mIoU	FPS	download
PanopticFCN	R50	1x	tbd	tbd	tbd	tbd	tbd	tbd	tbd

Citing PanopticFCN

Consider cite PanopticFCN in your publications if it helps your research.

@inproceedings{li2021panopticfcn,
  title={Fully Convolutional Networks for Panoptic Segmentation},
  author={Yanwei Li, Hengshuang Zhao, Xiaojuan Qi, Liwei Wang, Zeming Li, Jian Sun, and Jiaya Jia},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2021}
}

Consider cite this project in your publications if it helps your research.

@misc{PanopticFCN,
    author = {Yanwei Li},
    title = {PanopticFCN},
    howpublished = {\url{https://github.com/yanwei-li/PanopticFCN}},
    year ={2021}
}