PanopticFCN + Intra-Batch Supervision

[Project page] [Paper]

Code for 'Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images', Daan de Geus and Gijs Dubbelman, WACV 2023.

This code applies Intra-Batch Supervision to Panoptic FCN, and is built upon the official Panoptic FCN code.

Installation

This project is based on Detectron2, which can be constructed as follows.

• Install Detectron2 following these instructions.
• Create the environment variable DETECTRON2_DATASETS, where the datasets are stored
• Setup the Cityscapes dataset following this structure.
• Setup the Mapillary Vistas dataset following this structure.
• Copy this project to /path/to/detectron2/projects/PanopticFCN-IBS, replacing path/to/detectron2 with the path where Detectron2 is located.
• To prepare the datasets for our crop sampling, run these two commands:
  • python /path/to/detectron2/projects/PanopticFCN-IBS/data/cityscapes/prepare_cityscapes_sampling.py
  • python /path/to/detectron2/projects/PanopticFCN-IBS/data/mapillaryvistas/prepare_mapillary_sampling.py

Training

To train a model with 8 GPUs, run:

cd /path/to/detectron2
python projects/PanopticFCN-IBS/train.py --config-file <config.yaml> --num-gpus 8

For example, to launch a training with Panoptic FCN + IBS on Cityscapes with ResNet-50 backbone on 4 GPUs, one should execute:

cd /path/to/detectron2
python projects/PanopticFCN-IBS/train.py --config-file projects/PanopticFCN-IBS/configs/cityscapes/PanopticFCN-R50-cityscapes-ibs-cropsampling.yaml --num-gpus 4

For example, to launch a training with Panoptic FCN + IBS on Mapillary Vistas with ResNet-50 backbone on 8 GPUs, one should execute:

cd /path/to/detectron2
python projects/PanopticFCN-IBS/train.py --config-file projects/PanopticFCN-IBS/configs/mapillary-vistas/PanopticFCN-R50-mapillaryvistas-ibs-cropsampling.yaml --num-gpus 8

Evaluation

To evaluate a pre-trained model with 4 GPUs, run:

cd /path/to/detectron2
python projects/PanopticFCN-IBS/train.py --config-file <config.yaml> --num-gpus 4 --eval-only MODEL.WEIGHTS /path/to/model_checkpoint

Results

Results and models on Cityscapes. Note: like for the original Panoptic FCN, there is a large variance between results of different trainings with Panoptic FCN on Cityscapes.

Method	Crop sampling	Backbone	Iters	PQ	PQ_th	PQ_st	Acc_th	Prec_th	config	model
PanopticFCN	no	R50	65k	59.5	52.2	64.8	81.3	86.8	config	TBD
PanopticFCN + IBS	yes	R50	65k	60.8	54.7	65.3	87.1	92.6	config	TBD

Results and models on Mapillary Vistas

Method	Crop sampling	Backbone	Iters	PQ	PQ_th	PQ_st	Acc_th	Prec_th	config	model
PanopticFCN	no	R50	150k	35.5	31.8	40.3	74.7	77.9	config	TBD
PanopticFCN + IBS	yes	R50	150k	36.3	33.6	40.0	77.0	82.2	config	TBD

Citing us

Please consider citing our work if it is useful for your research.

@inproceedings{degeus2023ibs,
  title={Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images},
  author={{de Geus}, Daan and Dubbelman, Gijs},
  booktitle={IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  year={2023}
}

Also consider citing the original Panoptic FCN paper.

@inproceedings{li2021panopticfcn,
  title={Fully Convolutional Networks for Panoptic Segmentation},
  author={Yanwei Li, Hengshuang Zhao, Xiaojuan Qi, Liwei Wang, Zeming Li, Jian Sun, and Jiaya Jia},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2021}
}