how to set configuration when training faster rcnn on VOC2007

[absorbguo]

hi，all
I want to train faster-rcnn on my own dataset，and I begin with training FRCN on Pascal VOC2007 dataset。
After going through the faster-rcnn package，I figure out that there're two configuration files needed by the training process。
Firstly，the config.py which locates at “lib/fast-rcnn/config.py”specifies some training and testing options
training options are below

#
# Training options
#

__C.TRAIN = edict()
# Scales to use during training (can list multiple scales)
# Each scale is the pixel size of an image's shortest side
__C.TRAIN.SCALES = (600,)
# Max pixel size of the longest side of a scaled input image
__C.TRAIN.MAX_SIZE = 1000
# Images to use per minibatch
__C.TRAIN.IMS_PER_BATCH = 2
# Minibatch size (number of regions of interest [ROIs])
__C.TRAIN.BATCH_SIZE = 128

# Fraction of minibatch that is labeled foreground (i.e. class > 0)
__C.TRAIN.FG_FRACTION = 0.25

# Overlap threshold for a ROI to be considered foreground (if >= FG_THRESH)
__C.TRAIN.FG_THRESH = 0.5

# Overlap threshold for a ROI to be considered background (class = 0 if
# overlap in [LO, HI))
__C.TRAIN.BG_THRESH_HI = 0.5
__C.TRAIN.BG_THRESH_LO = 0.1

# Use horizontally-flipped images during training?
__C.TRAIN.USE_FLIPPED = True

# Train bounding-box regressors
__C.TRAIN.BBOX_REG = True

# Overlap required between a ROI and ground-truth box in order for that ROI to
# be used as a bounding-box regression training example
__C.TRAIN.BBOX_THRESH = 0.5

# Iterations between snapshots
__C.TRAIN.SNAPSHOT_ITERS = 10000

# solver.prototxt specifies the snapshot path prefix, this adds an optional
# infix to yield the path: <prefix>[_<infix>]_iters_XYZ.caffemodel
__C.TRAIN.SNAPSHOT_INFIX = ''

# Use a prefetch thread in roi_data_layer.layer
# So far I haven't found this useful; likely more engineering work is required
__C.TRAIN.USE_PREFETCH = False

# Normalize the targets (subtract empirical mean, divide by empirical stddev)
__C.TRAIN.BBOX_NORMALIZE_TARGETS = True
# Deprecated (inside weights)
__C.TRAIN.BBOX_INSIDE_WEIGHTS = (1.0, 1.0, 1.0, 1.0)
# Normalize the targets using "precomputed" (or made up) means and stdevs
# (BBOX_NORMALIZE_TARGETS must also be True)
__C.TRAIN.BBOX_NORMALIZE_TARGETS_PRECOMPUTED = False
__C.TRAIN.BBOX_NORMALIZE_MEANS = (0.0, 0.0, 0.0, 0.0)
__C.TRAIN.BBOX_NORMALIZE_STDS = (0.1, 0.1, 0.2, 0.2)

# Train using these proposals
__C.TRAIN.PROPOSAL_METHOD = 'gt'

# Make minibatches from images that have similar aspect ratios (i.e. both
# tall and thin or both short and wide) in order to avoid wasting computation
# on zero-padding.
__C.TRAIN.ASPECT_GROUPING = True

# Use RPN to detect objects
__C.TRAIN.HAS_RPN = False
# IOU >= thresh: positive example
__C.TRAIN.RPN_POSITIVE_OVERLAP = 0.7
# IOU < thresh: negative example
__C.TRAIN.RPN_NEGATIVE_OVERLAP = 0.3
# If an anchor statisfied by positive and negative conditions set to negative
__C.TRAIN.RPN_CLOBBER_POSITIVES = False
# Max number of foreground examples
__C.TRAIN.RPN_FG_FRACTION = 0.5
# Total number of examples
__C.TRAIN.RPN_BATCHSIZE = 256
# NMS threshold used on RPN proposals
__C.TRAIN.RPN_NMS_THRESH = 0.7
# Number of top scoring boxes to keep before apply NMS to RPN proposals
__C.TRAIN.RPN_PRE_NMS_TOP_N = 12000
# Number of top scoring boxes to keep after applying NMS to RPN proposals
__C.TRAIN.RPN_POST_NMS_TOP_N = 2000
# Proposal height and width both need to be greater than RPN_MIN_SIZE (at orig image scale)
__C.TRAIN.RPN_MIN_SIZE = 16
# Deprecated (outside weights)
__C.TRAIN.RPN_BBOX_INSIDE_WEIGHTS = (1.0, 1.0, 1.0, 1.0)
# Give the positive RPN examples weight of p * 1 / {num positives}
# and give negatives a weight of (1 - p)
# Set to -1.0 to use uniform example weighting
__C.TRAIN.RPN_POSITIVE_WEIGHT = -1.0

and testing options

#
# Testing options
#

__C.TEST = edict()

# Scales to use during testing (can list multiple scales)
# Each scale is the pixel size of an image's shortest side
__C.TEST.SCALES = (600,)

# Max pixel size of the longest side of a scaled input image
__C.TEST.MAX_SIZE = 1000

# Overlap threshold used for non-maximum suppression (suppress boxes with
# IoU >= this threshold)
__C.TEST.NMS = 0.3

# Experimental: treat the (K+1) units in the cls_score layer as linear
# predictors (trained, eg, with one-vs-rest SVMs).
__C.TEST.SVM = False

# Test using bounding-box regressors
__C.TEST.BBOX_REG = True

# Propose boxes
__C.TEST.HAS_RPN = False

# Test using these proposals
__C.TEST.PROPOSAL_METHOD = 'gt'

## NMS threshold used on RPN proposals
__C.TEST.RPN_NMS_THRESH = 0.7
## Number of top scoring boxes to keep before apply NMS to RPN proposals
__C.TEST.RPN_PRE_NMS_TOP_N = 6000
## Number of top scoring boxes to keep after applying NMS to RPN proposals
__C.TEST.RPN_POST_NMS_TOP_N = 300
# Proposal height and width both need to be greater than RPN_MIN_SIZE (at orig image scale)
__C.TEST.RPN_MIN_SIZE = 16

In the config.py script,I only modified PROPOSAL_METHOD :

__C.TRAIN.PROPOSAL_METHOD = 'gt'

secondly,there's another faster_rcnn_end2end.yml locate at "experiments/faster_rcnn_end2end" contains these infomation also specify some training and testing options.I didn't modify anything in this script.

EXP_DIR: faster_rcnn_end2end
TRAIN:
  HAS_RPN: True
  IMS_PER_BATCH: 1
  BBOX_NORMALIZE_TARGETS_PRECOMPUTED: True
  RPN_POSITIVE_OVERLAP: 0.7
  RPN_BATCHSIZE: 256
  PROPOSAL_METHOD: gt
  BG_THRESH_LO: 0.0
TEST:
  HAS_RPN: True

finally,there is another parameter which is scale to be set.The original setting in the anchor_target_layer.py and proposal_layer.py is (8,16,32) which differs a lot in the paper (128,256,512),but I keep it unchanged.

with these settings,I trained a model with 60000 iters and get a very bad result. The detector could propose some foreground regions but the cls_score is quite low nearly 10e-4. Does anyone know what's going wrong with these settings?

The model I used is vgg_cnn_m_1024 and I trained it from the begining without finetuning. I use solvers and train_val prototxt with raw settings under the root "models/pascal_voc/VGG_CNN_M_1024/faster_rcnn_end2end" .This means I take the end2end training strategy to train rpn and fast-rcnn network.
Solver is presented below:

train_net: "models/pascal_voc/VGG_CNN_M_1024/faster_rcnn_end2end/train.prototxt"
base_lr: 0.001
lr_policy: "step"
gamma: 0.1
stepsize: 5000
display: 40
average_loss: 100
momentum: 0.9
weight_decay: 0.0005
# We disable standard caffe solver snapshotting and implement our own snapshot
# function
snapshot: 5000
# We still use the snapshot prefix, though
snapshot_prefix: "vgg_cnn_m_1024_faster_rcnn"

the training prototxt I used is defined here..
best regards

[rohitghosh]

Hi @ absorbguo,

Well, if you've made sure rest of the boiler-plate code for training (like num_classes and num_outputs) have been taken care of, then I would suggest 2 - 3 things :

• Try optimising with maximum iterations & learning rates, very likely you could be overfitting with the default no of iterations (40k)

[absorbguo]

@rohitghosh thanks for your suggestion.
I add the solver and training prototxt in my issue. I run 60000 iterations last time, still got no good results.I wonder if there is something wrong beyond the settings.
Is there any other details I should pay attention?

[absorbguo]

Hi,all
I've tried the alt-opt training strategy last time and got a good result. It seems that the end2end strategy is not so effective as far as I can see even so it could save some training time.

[Po-Hsuan-Huang]

Can you give some marks for comparison ?

[315386775]

@absorbguo Do you have any changes other than the training methods?

[absorbguo]

@315386775 Except the training method, I changed the 'sigma' value in the stage1_rpn_train.pt. I set sigma=1. you can take a try.

[thanasissdr]

@absorbguo
Did you make any other changes to the config.py file other than

__C.TRAIN.PROPOSAL_METHOD = 'gt' ?

[fanw52]

hello,
I find the code will use the proposal method 'selective search' when test the network,and have you use the method 'gt' @thanasissdr @absorbguo