Release code

The code has been released and the README.md is updated.

ImageSets/README.md

@@ -0,0 +1 @@
+Please copy this folder to `$RFCN_ROOT/data/VOCdevkit0712/VOC0712`

README.md

@@ -1,18 +1,159 @@
-# DeRPN
-This repository is going to release the code for [*DeRPN*](https://arxiv.org/abs/1811.06700). 
+# DeRPN:  Taking a further step toward more general object detection
+DeRPN is a novel region proposal network which concentrates on improving the adaptivity of current detectors. The paper is available [here](https://arxiv.org/abs/1811.06700).
 
-The code is under preparation and will be released soon.
+## Recent Update
+**·** Jan. 25, 2019: The code is released.
 
-## Contact
-For any questions, please contact Lele Xie (xie.lele@mail.scut.edu.cn) or Prof. Jin (eelwjin@scut.edu.cn).
+## Contact Us
+Welcome to improve DeRPN together. For any questions, please feel free to contact Lele Xie (xie.lele@mail.scut.edu.cn) or Prof. Jin (eelwjin@scut.edu.cn).
 
 ## Citation
-
+If you find DeRPN useful to your research, please consider citing our paper as follow:
 ```
 @article{xie2019DeRPN,
   title     = {DeRPN: Taking a further step toward more general object detection},
   author    = {Lele Xie, Yuliang Liu, Lianwen Jin*, Zecheng Xie}
   joural    = {AAAI}
   year      = {2019}
 }
-```
+```
+## Main Results
+
+**Note**: The reimplemented results are different from those presented  in the paper for different training settings, but the conclusions are still consistent. For example, this code doesn't use multi-scale training which should boost the results for both DeRPN and RPN.
+
+### COCO-Text
+
+training data:  COCO-Text train
+
+test data: COCO-Text test
+
+|                   | network | AP@0.5  | recall@0.5 | AP@0.75 | recall@0.75 |
+|:-----------------:|:---------------------:|:---------------------:|:---------------------:|:---------------------:|:---------------------:|
+| RPN+Faster R-CNN | VGG16 | 32.48 | 52.54 | 7.40 | 17.59 |
+| DeRPN+Faster R-CNN | VGG16 | 47.39 | 70.46 | 11.05 | 25.12 |
+|RPN+R-FCN  |ResNet-101  | 37.71 | 54.35 | 13.17 | 22.21 |
+|DeRPN+R-FCN |ResNet-101 | 48.62 | 71.30 | 13.37 | 27.57 |
+
+### Pascal VOC
+
+training data:  VOC 07+12 trainval 
+
+test data: VOC 07 test
+
+Inference time is evaluated on one TITAN XP GPU.
+
+|                     |  network   | inference time | AP@0.5 | AP@0.75 |  AP   |
+| :-----------------: | :--------: | :------------: | :----: | :-----: | :---: |
+|  RPN+Faster R-CNN   |   VGG16    |     64 ms      | 75.53  |  42.08  | 42.60 |
+| DeRPN+Faster R-CNN  |   VGG16    |     65 ms      | 76.17  |  44.97  | 43.84 |
+|      RPN+R-FCN      | ResNet-101 |     85 ms      | 78.87  |  54.30  | 50.04 |
+| DeRPN+R-FCN (900) * | ResNet-101 |     84 ms      | 79.21  |  54.43  | 50.28 |
+
+( "*": On Pascal VOC dataset, we found that it is more suitable to train the DeRPN+R-FCN model with 900 proposals. For other experiments, we use the default proposal number to train the models, i.e., 2000 proposals fro Faster R-CNN, 300 proposals for R-FCN. )
+
+### MS COCO
+
+training data:  COCO 2017 train
+
+test data: COCO 2017 test/val
+
+| test set | network |  AP  | AP50 | AP75 | AP<sub>S</sub>  | AP<sub>M</sub>  | AP<sub>L</sub>  |
+| :----------------: | :-----: | :--: | :----: | :-----: | ---- | ---- | :--: |
+|  RPN+Faster R-CNN  |  VGG16  | 24.2 | 45.4 | 23.7 | 7.6 | 26.6 | 37.3 |
+| DeRPN+Faster R-CNN |  VGG16  | 25.5 | 47.2 | 25.2 | 10.3 | 27.9 | 36.7 |
+|     RPN+R-FCN      | ResNet-101 | 27.7 | 47.9 | 29.0 | 10.1 | 30.2 | 40.1 |
+|    DeRPN+R-FCN     | ResNet-101 | 28.4 | 49.0 | 29.5 | 11.1 | 31.7 | 40.5 |
+
+
+| val set | network |  AP  | AP50 | AP75 |AP<sub>S</sub>  | AP<sub>M</sub>  | AP<sub>L</sub>  |
+| :----------------: | :-----: | :--: | :----: | :-----: | ---- | ---- | :--: |
+|  RPN+Faster R-CNN  |  VGG16  | 24.1 | 45.0 | 23.8 | 7.6            | 27.8           |      37.8      |
+| DeRPN+Faster R-CNN |  VGG16  | 25.5 | 47.3 | 25.0 | 9.9 | 28.8 | 37.8 |
+|     RPN+R-FCN      | ResNet-101 | 27.8 | 48.1 | 28.8 | 10.4 | 31.2 | 42.5 |
+|    DeRPN+R-FCN     | ResNet-101 | 28.4 | 48.5 | 29.5 | 11.5 | 32.9 | 42.0 |
+
+## Getting Started
+
+1. Requirements
+2. Installation
+3. Preparation for Training & Testing
+4. Usage
+
+## Requirements
+1. Cuda 8.0 and cudnn 5.1. 
+2. Some python packages: cython, opencv-python, easydict et. al. Simply install them if your system misses these packages.
+3. Configure the caffe according to your environment ([Caffe installation instructions](http://caffe.berkeleyvision.org/installation.html)). As the code requires pycaffe, caffe should be built with python layers. In Makefile.config, make sure to uncomment this line:
+
+  ```
+  WITH_PYTHON_LAYER := 1
+  ```
+4. An NVIDIA GPU with more than 6GB is required for ResNet-101.
+
+## Installation
+1. Clone the DeRPN repository
+    ```
+    git clone https://github.com/HCIILAB/DeRPN.git
+    ```
+2. Build the Cython modules
+    ```Shell
+    cd $DeRPN_ROOT/lib
+    make
+    ```
+
+3. Build caffe and pycaffe
+    ```Shell
+    cd $DeRPN_ROOT/caffe
+    make -j8 && make pycaffe
+    ```
+
+## Preparation for Training & Testing
+### Dataset
+
+1. Download the datasets of [Pascal VOC 2007 & 2012](http://host.robots.ox.ac.uk/pascal/VOC/), [MS COCO 2017](http://cocodataset.org/#download) and [COCO-Text](http://rrc.cvc.uab.es/?ch=5&com=introduction).
+
+2. You need to put these datasets under the $DeRPN_ROOT/data folder (with symlinks). 
+
+3. For COCO-Text, the folder structure is as follow:
+
+    ```Shell
+    $DeRPN_ROOT/data/coco_text/images/train2014
+    $DeRPN_ROOT/data/coco_text/images/val2014
+    $DeRPN_ROOT/data/coco_text/annotations  
+    # train2014, val2014, and annotations are symlinks from /pth_to_coco2014/train2014, /pth_to_coco2014/val2014 and /pth_to_coco2014/annotations2014/, respectively.
+    ```
+4. For COCO, the folder structure is as follow:
+
+    ```Shell
+    $DeRPN_ROOT/data/coco/images/train2017
+    $DeRPN_ROOT/data/coco/images/val2017
+    $DeRPN_ROOT/data/coco/images/test-dev2017
+    $DeRPN_ROOT/data/coco/annotations  
+    # the symlinks are similar to COCO-Text
+    ```
+
+5. For Pascal VOC, the folder structure is as follow:
+
+    ```Shell
+    $DeRPN_ROOT/data/VOCdevkit2007
+    $DeRPN_ROOT/data/VOCdevkit2012
+    #VOCdevkit2007 and VOCdevkit2012 are symlinks from $VOCdevkit whcich contains VOC2007 and VOC2012.
+    ```
+
+### Pretrained models
+
+Please download the ImageNet pretrained models ([VGG16](https://pan.baidu.com/s/1BDDl5xtrBznlyIrVj9g_zQ) and [ResNet-101](https://pan.baidu.com/s/1BDDl5xtrBznlyIrVj9g_zQ), password: k4z1), and put them under
+```Shell
+$DeRPN_ROOT/data/imagenet_models
+```
+
+
+## Usage
+```Shell
+cd $DeRPN_ROOT
+./experiments/scripts/faster_rcnn_derpn_end2end.sh [GPU_ID] [NET] [DATASET]
+
+# e.g., ./experiments/scripts/faster_rcnn_derpn_end2end.sh 0 VGG16 coco_text
+```
+
+## Copyright
+This code is free to the academic community for research purpose only.

data/scripts/fetch_faster_rcnn_models.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )/../" && pwd )"
+cd $DIR
+
+FILE=faster_rcnn_models.tgz
+URL=http://www.cs.berkeley.edu/~rbg/faster-rcnn-data/$FILE
+CHECKSUM=ac116844f66aefe29587214272054668
+
+if [ -f $FILE ]; then
+  echo "File already exists. Checking md5..."
+  os=`uname -s`
+  if [ "$os" = "Linux" ]; then
+    checksum=`md5sum $FILE | awk '{ print $1 }'`
+  elif [ "$os" = "Darwin" ]; then
+    checksum=`cat $FILE | md5`
+  fi
+  if [ "$checksum" = "$CHECKSUM" ]; then
+    echo "Checksum is correct. No need to download."
+    exit 0
+  else
+    echo "Checksum is incorrect. Need to download again."
+  fi
+fi
+
+echo "Downloading Faster R-CNN demo models (695M)..."
+
+wget $URL -O $FILE
+
+echo "Unzipping..."
+
+tar zxvf $FILE
+
+echo "Done. Please run this command again to verify that checksum = $CHECKSUM."

data/scripts/fetch_imagenet_models.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )/../" && pwd )"
+cd $DIR
+
+FILE=imagenet_models.tgz
+URL=http://www.cs.berkeley.edu/~rbg/faster-rcnn-data/$FILE
+CHECKSUM=ed34ca912d6782edfb673a8c3a0bda6d
+
+if [ -f $FILE ]; then
+  echo "File already exists. Checking md5..."
+  os=`uname -s`
+  if [ "$os" = "Linux" ]; then
+    checksum=`md5sum $FILE | awk '{ print $1 }'`
+  elif [ "$os" = "Darwin" ]; then
+    checksum=`cat $FILE | md5`
+  fi
+  if [ "$checksum" = "$CHECKSUM" ]; then
+    echo "Checksum is correct. No need to download."
+    exit 0
+  else
+    echo "Checksum is incorrect. Need to download again."
+  fi
+fi
+
+echo "Downloading pretrained ImageNet models (1G)..."
+
+wget $URL -O $FILE
+
+echo "Unzipping..."
+
+tar zxvf $FILE
+
+echo "Done. Please run this command again to verify that checksum = $CHECKSUM."

data/scripts/fetch_selective_search_data.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )/../" && pwd )"
+cd $DIR
+
+FILE=selective_search_data.tgz
+URL=http://www.cs.berkeley.edu/~rbg/fast-rcnn-data/$FILE
+CHECKSUM=7078c1db87a7851b31966b96774cd9b9
+
+if [ -f $FILE ]; then
+  echo "File already exists. Checking md5..."
+  os=`uname -s`
+  if [ "$os" = "Linux" ]; then
+    checksum=`md5sum $FILE | awk '{ print $1 }'`
+  elif [ "$os" = "Darwin" ]; then
+    checksum=`cat $FILE | md5`
+  fi
+  if [ "$checksum" = "$CHECKSUM" ]; then
+    echo "Checksum is correct. No need to download."
+    exit 0
+  else
+    echo "Checksum is incorrect. Need to download again."
+  fi
+fi
+
+echo "Downloading precomputed selective search boxes (0.5G)..."
+
+wget $URL -O $FILE
+
+echo "Unzipping..."
+
+tar zxvf $FILE
+
+echo "Done. Please run this command again to verify that checksum = $CHECKSUM."

experiments/README.md

@@ -0,0 +1,5 @@
+Scripts are under `experiments/scripts`.
+
+Each script saves a log file under `experiments/logs`.
+
+Configuration override files used in the experiments are stored in `experiments/cfgs`.

experiments/cfgs/faster_rcnn_alt_opt.yml

@@ -0,0 +1,5 @@
+EXP_DIR: faster_rcnn_alt_opt
+TRAIN:
+  BG_THRESH_LO: 0.0
+TEST:
+  HAS_RPN: True

experiments/cfgs/faster_rcnn_derpn_end2end.yml

@@ -0,0 +1,11 @@
+EXP_DIR: faster_rcnn_derpn_end2end
+TRAIN:
+  HAS_RPN: True
+  IMS_PER_BATCH: 1
+  BBOX_NORMALIZE_TARGETS_PRECOMPUTED: True
+  RPN_POSITIVE_OVERLAP: 0.7
+  RPN_BATCHSIZE: 256
+  PROPOSAL_METHOD: gt
+  BG_THRESH_LO: 0.0
+TEST:
+  HAS_RPN: True

experiments/cfgs/faster_rcnn_end2end.yml

@@ -0,0 +1,11 @@
+EXP_DIR: faster_rcnn_end2end
+TRAIN:
+  HAS_RPN: True
+  IMS_PER_BATCH: 1
+  BBOX_NORMALIZE_TARGETS_PRECOMPUTED: True
+  RPN_POSITIVE_OVERLAP: 0.7
+  RPN_BATCHSIZE: 256
+  PROPOSAL_METHOD: gt
+  BG_THRESH_LO: 0.0
+TEST:
+  HAS_RPN: True

experiments/cfgs/rfcn_alt_opt_5step_ohem.yml

@@ -0,0 +1,12 @@
+EXP_DIR: rfcn_alt_opt_5step_ohem
+TRAIN:
+  BG_THRESH_LO: 0.0
+  RPN_PRE_NMS_TOP_N: 6000
+  RPN_POST_NMS_TOP_N: 300
+  AGNOSTIC: True
+  BATCH_SIZE: -1
+  RPN_NORMALIZE_TARGETS: True
+TEST:
+  PROPOSAL_METHOD: 'rpn'
+  HAS_RPN: False
+  AGNOSTIC: True

experiments/cfgs/rfcn_derpn_end2end_ohem.yml

@@ -0,0 +1,19 @@
+EXP_DIR: rfcn_derpn_end2end_ohem
+TRAIN:
+  HAS_RPN: True
+  IMS_PER_BATCH: 1
+  BBOX_NORMALIZE_TARGETS_PRECOMPUTED: True
+  RPN_POSITIVE_OVERLAP: 0.7
+  RPN_NORMALIZE_TARGETS: False
+  RPN_BATCHSIZE: 256
+  PROPOSAL_METHOD: gt
+  BG_THRESH_LO: 0.0
+  BATCH_SIZE: -1
+  AGNOSTIC: True
+  SNAPSHOT_ITERS: 10000
+  RPN_PRE_NMS_TOP_N: 6000
+  RPN_POST_NMS_TOP_N: 300
+  DeRPN_final_top_M: 300 
+TEST:
+  HAS_RPN: True
+  AGNOSTIC: True

experiments/cfgs/rfcn_end2end.yml

@@ -0,0 +1,17 @@
+EXP_DIR: rfcn_end2end
+TRAIN:
+  HAS_RPN: True
+  IMS_PER_BATCH: 1
+  BBOX_NORMALIZE_TARGETS_PRECOMPUTED: True
+  RPN_POSITIVE_OVERLAP: 0.7
+  RPN_BATCHSIZE: 256
+  PROPOSAL_METHOD: gt
+  BG_THRESH_LO: 0.1
+  BATCH_SIZE: 128
+  AGNOSTIC: True
+  SNAPSHOT_ITERS: 10000
+  RPN_PRE_NMS_TOP_N: 6000
+  RPN_POST_NMS_TOP_N: 300
+TEST:
+  HAS_RPN: True
+  AGNOSTIC: True

experiments/cfgs/rfcn_end2end_ohem.yml

@@ -0,0 +1,18 @@
+EXP_DIR: rfcn_end2end_ohem
+TRAIN:
+  HAS_RPN: True
+  IMS_PER_BATCH: 1
+  BBOX_NORMALIZE_TARGETS_PRECOMPUTED: True
+  RPN_POSITIVE_OVERLAP: 0.7
+  RPN_NORMALIZE_TARGETS: True
+  RPN_BATCHSIZE: 256
+  PROPOSAL_METHOD: gt
+  BG_THRESH_LO: 0.0
+  BATCH_SIZE: -1
+  AGNOSTIC: True
+  SNAPSHOT_ITERS: 10000
+  RPN_PRE_NMS_TOP_N: 6000
+  RPN_POST_NMS_TOP_N: 300
+TEST:
+  HAS_RPN: True
+  AGNOSTIC: True

experiments/logs/.gitignore

@@ -0,0 +1 @@
+*.txt*