Category-Aware Spatial Constraint for Weakly Supervised Detection
By Yunhang Shen, Rongrong Ji, Kuiyuan Yang, Cheng Deng, Changhu Wang.
TIP 2019 Paper.
Citing CSC
If you find CSC useful in your research, please consider citing:
@article{CSC_2019_TIP,
author = {Shen, Yunhang and Ji, Rongrong and Yang, Kuiyuan and Deng, Cheng and Wang, Changhu},
journal = {IEEE TRANSACTIONS ON IMAGE PROCESSING (TIP)},
title = {Category-Aware Spatial Constraint for Weakly Supervised Detection},
year = {2019}
}
Contents
Requirements: software
- Requirements for
Caffe
andpycaffe
(see: Caffe installation instructions)
Note: Caffe must be built with support for Python layers!
# In your Makefile.config, make sure to have this line uncommented
WITH_PYTHON_LAYER := 1
# Unrelatedly, it's also recommended that you use CUDNN
USE_CUDNN := 1
- Python packages you might not have:
cython
,python-opencv
,easydict
- [Optional] MATLAB is required for official PASCAL VOC evaluation only. The code now includes unofficial Python evaluation code.
Requirements: hardware
- For training smaller networks (VGG_CNN_F, VGG_CNN_M_1024), a GPU with about 6G of memory suffices.
- For training lager networks (VGG16), you'll need a GPU with about 8G of memory.
Installation
- Clone the CSC repository
# Make sure to clone with --recursive
git clone --recursive https://github.com/shenyunhang/CSC.git
-
We'll call the directory that you cloned CSC into
CSC_ROOT
Ignore notes 1 and 2 if you followed step 1 above.
Note 1: If you didn't clone CSC with the
--recursive
flag, then you'll need to manually clone thecaffe-wsl
submodule:git submodule update --init --recursive
Note 2: The
caffe-wsl
submodule needs to be on thewsl
branch (or equivalent detached state). This will happen automatically if you followed step 1 instructions. -
Build the Cython modules
cd $CSC_ROOT/lib make
-
Build Caffe and pycaffe
cd $CSC_ROOT/caffe-wsl # Now follow the Caffe installation instructions here: # http://caffe.berkeleyvision.org/installation.html # If you're experienced with Caffe and have all of the requirements installed # and your Makefile.config in place, then simply do: make -j8 && make pycaffe
-
Download the training, validation, test data and VOCdevkit
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCdevkit_08-Jun-2007.tar
-
Extract all of these tars into one directory named
VOCdevkit
tar xvf VOCtrainval_06-Nov-2007.tar tar xvf VOCtest_06-Nov-2007.tar tar xvf VOCdevkit_08-Jun-2007.tar
-
It should have this basic structure
$VOCdevkit/ # development kit $VOCdevkit/VOCcode/ # VOC utility code $VOCdevkit/VOC2007 # image sets, annotations, etc. # ... and several other directories ...
-
Create symlinks for the PASCAL VOC dataset
cd $CSC_ROOT/data ln -s $VOCdevkit VOCdevkit2007
Using symlinks is a good idea because you will likely want to share the same PASCAL dataset installation between multiple projects.
-
[Optional] follow similar steps to get PASCAL VOC 2010 and 2012
-
[Optional] If you want to use COCO, please see some notes under
data/README.md
-
Follow the next sections to download pre-trained ImageNet models
Download object proposals
- Selective Search: original matlab code, python wrapper
- EdgeBoxes: matlab code
- MCG: matlab code
Download pre-trained ImageNet models
Pre-trained ImageNet models can be downloaded for the three networks described in the paper: ZF and VGG16.
cd $CSC_ROOT
./data/scripts/fetch_imagenet_models.sh
Usage
To train and test a CSC detector, use experiments/scripts/csc.sh
.
Output is written underneath $CSC_ROOT/output
.
cd $CSC_ROOT
./experiments/scripts/csc.sh [GPU_ID] [NET] [--set ...]
# GPU_ID is the GPU you want to train on
# NET in {VGG_CNN_F, VGG_CNN_M_1024, VGG16} is the network arch to use
# --set ... allows you to specify configure options, e.g.
# --set EXP_DIR seed_rng1701 RNG_SEED 1701
Example:
./experiments/scripts/csc.sh 0 VGG16 pascal_voc --set EXP_DIR csc
This will reproduction the VGG16 result in paper.
Trained CSC networks are saved under:
output/<experiment directory>/<dataset name>/
Test outputs are saved under:
output/<experiment directory>/<dataset name>/<network snapshot name>/
Other method
WSDDN:
./experiments/scripts/wsddn.sh 0 VGG16 pascal_voc --set EXP_DIR wsddn
or
./experiments/scripts/wsddn_x.sh 0 VGG16 pascal_voc --set EXP_DIR wsddn_x
ContextLocNet:
./experiments/scripts/contextlocnet.sh 0 VGG16 pascal_voc --set EXP_DIR contextlocnet
or
./experiments/scripts/contextlocnet_x.sh 0 VGG16 pascal_voc --set EXP_DIR contextlocnet_x