Path Aggregation Network for Instance Segmentation

by Shu Liu, Lu Qi, Haifang Qin, Jianping Shi, Jiaya Jia.

Introduction

This repository is for the CVPR 2018 Spotlight paper, 'Path Aggregation Network for Instance Segmentation', which ranked 1st place of COCO Instance Segmentation Challenge 2017 , 2nd place of COCO Detection Challenge 2017 (Team Name: UCenter) and 1st place of 2018 Scene Understanding Challenge for Autonomous Navigation in Unstructured Environments (Team Name: TUTU).

Citation

If PANet is useful for your research, please consider citing:

@inproceedings{liu2018path,
  author = {Shu Liu and
            Lu Qi and
            Haifang Qin and
            Jianping Shi and
            Jiaya Jia},
  title = {Path Aggregation Network for Instance Segmentation},
  booktitle = {Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2018}
}

Disclaimer

• The original code was implemented based on the modified version of Caffe maintained by Sensetime Research. Due to several reasons, we could not release our original code.
• In this repository, we provide our re-implementation of PANet based on Pytorch. Note that our code is heavily based on Detectron.pytorch. Thanks Roy for his great work!
• Several details, e.g., weight initialization and RPN joint training, in Detectron is fairly different from our origin implementation. In this repository, we simply follow Detectron because it achieves a better baseline than the codebase used in our paper.
• In this repository, we test our code with BN layers in the backbone fixed and use GN in other part. We expect to achieve a better performance with Synchronized Batch Normalization Layer and train all parameter layers as what we have done in our paper. With those differences and a much better baseline, the improvement is not same as the one we reported. But we achieve a better performance than our original implementation.
• We trained with image batch size 16 using 8*P40. The performance should be similar with batch size 8.

Compilation

Compile the CUDA code:

cd lib  # please change to this directory
sh make.sh

If your are using Volta GPUs, uncomment this line in lib/mask.sh and remember to postpend a backslash at the line above. CUDA_PATH defaults to /usr/loca/cuda. If you want to use a CUDA library on different path, change this line accordingly.

It will compile all the modules you need, including NMS, ROI_Pooing, ROI_Crop and ROI_Align. (Actually gpu nms is never used ...)

Note that, If you use CUDA_VISIBLE_DEVICES to set gpus, make sure at least one gpu is visible when compile the code.

Additionally, note that the architectures specified in CUDA ARCH variable in make.sh matches the architecture of your GPU

Installation

For environment requirements, data preparation and compilation, please refer to Detectron.pytorch.

WARNING: pytorch 0.4.1 is broken, see pytorch/pytorch#8483. Use pytorch 0.4.0

Usage

For training and testing, we keep the same as the one in Detectron.pytorch. To train and test PANet, simply use corresponding config files. For example, to train PANet on COCO:

python tools/train_net_step.py --dataset coco2017 --cfg configs/panet/e2e_panet_R-50-FPN_2x_mask.yaml

To evaluate the model, simply use:

python tools/test_net.py --dataset coco2017 --cfg configs/panet/e2e_panet_R-50-FPN_2x_mask.yaml --load_ckpt {path/to/your/checkpoint}

Alternatively, you could use the shell scripts provided.

• train.sh                        Train
• eval_pan_gpu.sh        Evaluate (COCO) on multiple GPUs
• run_pan.sh                  Inference on single gpu
• run_pan_gpu.sh         Inference on multiple GPUs

Main Results

Backbone	Type	Batch Size	LR Schedules	Box AP	Mask AP	Download Links
R-50-PANet (paper)	Faster	16	1x	39.2	-	-
R-50-PANet	Faster	16	1x	39.8	-	model
R-50-PANet-2fc (paper)	Faster	16	1x	39.0	-	-
R-50-PANet-2fc	Faster	16	1x	39.6	-	model
R-50-PANet (paper)	Mask	16	2x	42.1	37.8	-
R-50-PANet	Mask	16	2x	43.1	38.3	model

Results on COCO 2017 val subset produced by this repository. In our paper, we used Synchronized Batch Normalization following all parameter layers. While in this repository, we fix BN layers in the backbone and use GN layers in other part. With the same set of hyper-parameters, e.g., multi-scales, this repository can produce better performance than that in our origin paper. We expect a better performance with Synchronized Batch Normalization Layer.

Docker

Clone the repository and then build the container via the Dockerfle provided.

git clone https://github.com/ash1995/PANet.git
cd PANet
# Build the docker image
nvidia-docker build --no-cache -t panet_image .
# Create a container and mount the repository volume
nvidia-docker run -it -d --name=panet -v /path/to/PANet:/workspace/PANet panet_image /bin/bash

Note:

For running the shell script to split images across multiple GPUs for inference, use the following command:

nvidia-docker exec container_id /workspace/PANet/run_pan_gpu.sh

Questions

Please contact 'liushuhust@gmail.com'