Adding Anchor scales

[ljtruong]

Hi I'm adding smaller anchor scales to detect smaller objects with also large objects.

I'm making changes to this line

args.set_cfgs = ['ANCHOR_SCALES', '[8, 16, 32]', 'ANCHOR_RATIOS', '[0.5,1,2]', 'MAX_NUM_GT_BOXES', '20']

to

args.set_cfgs = ['ANCHOR_SCALES', '[4, 8, 16, 32]', 'ANCHOR_RATIOS', '[0.5,1,2]', 'MAX_NUM_GT_BOXES', '20']

it works for the training script. but when I run it in demo.py it has issues:

error occurs here: link
fasterRCNN.load_state_dict(checkpoint['model'])

error message:
fasterRCNN.load_state_dict(checkpoint['model']) File "/home/ubuntu/py3/lib/python3.5/site-packages/torch/nn/modules/module.py", line 721, in load_state_dict self.__class__.__name__, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for resnet: While copying the parameter named "RCNN_rpn.RPN_cls_score.bias", whose dimensions in the model are torch.Size([18]) and whose dimensions in the checkpoint are torch.Size([24]). While copying the parameter named "RCNN_rpn.RPN_cls_score.weight", whose dimensions in the model are torch.Size([18, 512, 1, 1]) and whose dimensions in the checkpoint are torch.Size([24, 512, 1, 1]). While copying the parameter named "RCNN_rpn.RPN_bbox_pred.bias", whose dimensions in the model are torch.Size([36]) and whose dimensions in the checkpoint are torch.Size([48]). While copying the parameter named "RCNN_rpn.RPN_bbox_pred.weight", whose dimensions in the model are torch.Size([36, 512, 1, 1]) and whose dimensions in the checkpoint are torch.Size([48, 512, 1, 1]).

this is assuming the incorrect architecture is provided. I suspect there is something wrong with the configuration file fed in. but I'm not sure. Has anyone experienced this and has a solution?

thank you in advance!

[ljtruong]

Solved. A change in the yml config file is required. using --cfgs and adding in the configuration variable with required list of anchor scales.

[CyanideCentral]

This problem appears for model trained on COCO dataset. I fixed this by changing this line to [4,8,16,32].

[kangkang59812]

@Worulz what should I add in res101.yml ? SCALES: [4,8,16,32] ?

[andres-fr]

@kangkang59812 The important thing is that (ANCHOR_SCALES times ANCHOR_RATIOS) should be the number of anchors that your architecture expects. The paper and the default config have 3*3=9, but the COCO pretrained model expects 12. Therefore you can more or less "choose" the combination you want, as long as they multiply to 12 (CLARIFICATION: ideally you should reproduce the exact set of anchors that was used to train, I don't know how to reverse engineer that.). The anchor config for the pretrained COCO model seems to be here:

faster-rcnn.pytorch/trainval_net.py

Line 166 in aec4244

args.set_cfgs = ['ANCHOR_SCALES', '[4, 8, 16, 32]', 'ANCHOR_RATIOS', '[0.5,1,2]', 'MAX_NUM_GT_BOXES', '50']

In my case (at the PyTorch 1.0 branch), the demo runs flawlessly with the Pascal pretrained model, but I had this issue when trying it with COCO as @CyanideCentral reports. Apart from what I already mentioned, I also had a class mismatch using the demo (21 vs. 81). This is because the demo instantiates with the pascal_classes, but the COCO pretrained model expects the COCO classes. Changing the list (in the official order given at the link) plus __background__ at the beginning made the demo work with COCO. For the copypasters:

coco_classes = np.asarray(["__background__", "person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "couch", "potted plant", "bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"])

Hope this helps!
Andres

[kangkang59812]

thanks a lot @andres-fr. But it still has the SIZE ERROR. I will spend some time to figure it out. Thanks again!
RuntimeError: Error(s) in loading state_dict for resnet: size mismatch for RCNN_rpn.RPN_cls_score.weight: copying a param with shape torch.Size([24, 512, 1, 1]) from checkpoint, the shape in current model is torch.Size([18, 512, 1, 1]). size mismatch for RCNN_rpn.RPN_cls_score.bias: copying a param with shape torch.Size([24]) from checkpoint, the shape in current model is torch.Size([18]). size mismatch for RCNN_rpn.RPN_bbox_pred.weight: copying a param with shape torch.Size([48, 512, 1, 1]) from checkpoint, the shape in current model is torch.Size([36, 512, 1, 1]). size mismatch for RCNN_rpn.RPN_bbox_pred.bias: copying a param with shape torch.Size([48]) from checkpoint, the shape in current model is torch.Size([36]). size mismatch for RCNN_cls_score.weight: copying a param with shape torch.Size([81, 2048]) from checkpoint, the shape in current model is torch.Size([21, 2048]). size mismatch for RCNN_cls_score.bias: copying a param with shape torch.Size([81]) from checkpoint, the shape in current model is torch.Size([21]). size mismatch for RCNN_bbox_pred.weight: copying a param with shape torch.Size([324, 2048]) from checkpoint, the shape in current model is torch.Size([84, 2048]). size mismatch for RCNN_bbox_pred.bias: copying a param with shape torch.Size([324]) from checkpoint, the shape in current model is torch.Size([84]).

[andres-fr]

@kangkang59812 those errors seem to be exactly the ones I fixed as I said. Did you try my approach?

[kangkang59812]

@andres-fr I have tried. But it results this error as before.

[Chinmoy07]

@andres-fr
Trying to run the demo with Coco classes. Getting size error. Already changed the sizes as mentioned above:

load checkpoint /home/jarvis/Downloads/faster-rcnn.pytorch-pytorch-1.0/res101/pascal_voc/faster_rcnn_1_6_9771.pth
load model successfully!
load checkpoint /home/jarvis/Downloads/faster-rcnn.pytorch-pytorch-1.0/res101/pascal_voc/faster_rcnn_1_6_9771.pth
RuntimeError: shape '[1, -1, 84]' is invalid for input of size 97200

[dreamedrainbow]

@kangkang59812 those errors seem to be exactly the ones I fixed as I said. Did you try my approach?

Hi,there. When running test_net.py with my own res101 pretrained model in PASCAL VOC dataset, I can not flawlessly perform and get the error below:
RuntimeError: Error(s) in loading state_dict for resnet: size mismatch for RCNN_bbox_pred.weight: copying a param with shape torch.Size([84, 2048]) from checkpoint, the shape in current model is torch.Size([4, 2048]). size mismatch for RCNN_bbox_pred.bias: copying a param with shape torch.Size([84]) from checkpoint, the shape in current model is torch.Size([4]).
Any advices?

[kangkang59812]

@dreamedrainbow The official version of pytorch is easy to understand.
https://pytorch.org/docs/stable/torchvision/models.html?highlight=faster#torchvision.models.detection.fasterrcnn_resnet50_fpn

[hackerchenzhuo]

COCO classes

thanks