No description, website, or topics provided.
Matlab C++ Cuda M
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
experiments init Aug 12, 2017
external added caffe files Aug 14, 2017
functions init Aug 12, 2017
models init Aug 12, 2017
.gitignore init Aug 12, 2017
LICENSE init Aug 12, 2017
build_nms.m init Aug 12, 2017 spelling correction Nov 9, 2017
startup.m init Aug 12, 2017

Illuminating Pedestrians via Simultaneous Detection & Segmentation

Garrick Brazil, Xi Yin, Xiaoming Liu


Pedestrian detection framework using simultaneous detection and segmentation as detailed in arXiv report, accepted to ICCV 2017.

Our SDS-RCNN framework is derivative work of Faster R-CNN and RPN+BF. Tested with Ubuntu 14.04, CUDA 7.5, Matlab 2016a, Titan X GPU, and a modified version of Caffe v1.0 as provided. Unless otherwise stated the below scripts and instructions assume cwd in MATLAB is the project root of SDS-RCNN.

    title={Illuminating Pedestrians via Simultaneous Detection \& Segmentation},
    author={Brazil, Garrick and Yin, Xi and Liu, Xiaoming},
    booktitle={Proceedings of the IEEE International Conference on Computer Vision},
    address={Venice, Italy},


  • Build Caffe

    Build caffe and matcaffe following the usual instructions. We provide an upgraded version of Caffe v1.0 which includes the required layers necessary to run Faster R-CNN in external/caffe.

  • Data

    Download the full Caltech dataset. In order to evaluate you must extract or soft-link a folder called data-USA into the directory external/caltech_toolbox/ such that such that the annotation and video files can be accessed as: data-USA/annotations/*.vbb and data-USA/videos/*.seq.

    Then extract the datasets for train, val, test in Matlab as below (or setup softlinks as desire).

    dbInfo('usatrain');    dbExtract('datasets/caltechx10/train', 1, 3);
    dbInfo('usatrainval'); dbExtract('datasets/caltechval/val', 1);
    dbInfo('usatest');     dbExtract('datasets/caltechx1/test', 1);
  • Misc

    1. Download the pretrained VGG16 on ImageNet and place in SDS-RCNN/pretrained/vgg16.caffemodel.
    2. Run build_nms to compile nms mex files.
    3. Review the config files in experiments/+Config/+[rcnn|rpn] for additional information.


Training both stages takes about 18 hours on a single Titan X.

rpn_config  = 'caltech_VGG16_weak_seg';
rcnn_config	= 'caltech_VGG16_weak_seg';
gpu_id = 1;

% train both stages
train_all(rpn_config, rcnn_config, gpu_id);


We provide the collective SDS-RCNN trained models for RPN and BCN (7.36% MR), as well as the RPN only file with cost-sensitive off (9.63% MR). There are associated artifact files of anchors, bbox_stds, bbox_means, and basic configurations which should be loaded into memory at test time as depicted below. All files are packed into

gpu_id = 1;

% test RPN only
test_rpn(test_prototxt_path, weights_path, rpn_conf, anchors, bbox_means, bbox_stds, gpu_id)

% test RPN and BCN (full SDS-RCNN)
test_rcnn(test_prototxt_path, weights_path, rpn_conf, anchors, bbox_means, bbox_stds, ...
    rcnn_prototxt, rcnn_weights, rcnn_conf, gpu_id)