Improvement for small object detection

[shadyatscu]

Thank you for the great work! I ran the proposed code on my custom dataset of medical image, the result list as below:
2019-05-06 03:04:17,399 maskrcnn_benchmark.inference INFO:
OrderedDict([('bbox', OrderedDict([('AP', 0.5055817771868518),
('AP50', 0.8926599742997058), ('AP75', 0.4691991724725123),
('APs', 0.0007072135785007071), ('APm', 0.10701570554699777),
('APl', 0.5285697565878185)]))])

compared with yolov3 on the same dataset, i found that the proposed algorithm works not very well on small object ('APs', 0.0007072135785007071), as mentioned in the paper this is a anchor free method. Is there any idea to improve the detection performance on small object? or any hyperparameters to finetune?

[tianzhi0549]

@shadyatscu The finest feature maps we use is 1/8 resolution of an input image. If you want to improve the performance of very small objects, you may use the feature maps with 1/4 resolution.

[shadyatscu]

@shadyatscu The finest feature maps we use is 1/8 resolution of an input image. If you want to improve the performance of very small objects, you may use the feature maps with 1/4 resolution.

thanks for your reply, i change the configs followed your advice from the default
_C.MODEL.FCOS.FPN_STRIDES = [8, 16, 32, 64, 128]
to
_C.MODEL.FCOS.FPN_STRIDES = [4, 8, 16, 32, 64]

hope that works! thanks a lot!

[tianzhi0549]

@shadyatscu It is not correct if you only changed that. The code at

FCOS/maskrcnn_benchmark/modeling/backbone/backbone.py

Line 50 in 79fd4ac

def build_resnet_fpn_p3p7_backbone(cfg):

needs to be changed as well.

[shadyatscu]

Follow your advice i changed the code from the default:
in_channels_p6p7 = in_channels_stage2 * 8 if cfg.MODEL.RETINANET.USE_C5
else out_channels
fpn = fpn_module.FPN( in_channels_p6p7 = in_channels_stage2 * 8 if cfg.MODEL.RETINANET.USE_C5
else out_channels
in_channels_list=[
0,
in_channels_stage2 * 2,
in_channels_stage2 * 4,
in_channels_stage2 * 8,
],
to this:
fpn = fpn_module.FPN(
in_channels_list=[
0,
in_channels_stage2,
in_channels_stage2 * 2,
in_channels_stage2 * 4,
],
error occoured like this:
RuntimeError: Given groups=1, weight of size [256, 1024, 1, 1], expected input[2, 2048, 7, 8] to have 1024 channels, but got 2048 channels instead
i am confused about this part, could you plz give me some detailed advices?

[tianzhi0549]

@shadyatscu in order to use the feature maps with 1/4 resolution, you need to change the first 0 to in_channels_stage2 and add 4 to FPN_STRIDES (i.e., [4, 8, 16, 32, 64, 128]).
You also need to add the object_sizes_of_interest for the level of feature maps to

FCOS/maskrcnn_benchmark/modeling/rpn/fcos/loss.py

Line 38 in 79fd4ac

object_sizes_of_interest = [

.

[Baby47]

@shadyatscu Have you tried as the author suggested, and if it works well too?

[shadyatscu]

@shadyatscu Have you tried as the author suggested, and if it works well too?

I tried the suggestions of author, an error always occoured when i change 'the first 0 to in_channels_stage2', so still things to work on

[ZhengMengbin]

@shadyatscu I guess you do not change this code(the number 5 should be modified to 6)：
self.scales = nn.ModuleList([Scale(init_value=1.0) for _ in range(5)])
（the code is in /maskrcnn_benchmark/modeling/rpn/fcos/fcos.py)
It will report an Error!

[fangchengji]

I found FCOS was hard to detect small objects too. I think this problem is caused by center-ness branch. For example, a 16 * 16 bbox. The center-ness in P3 is probably 0.333. This value multiply class_pred gets a negative result.
I try to abandon the center-ness branch, a lot of low-quality predicted bounding boxes appear. Maybe a tricked center-ness ground truth for small objects is a solution.

[stigma0617]

@shadyatscu @Baby47 @ZhengMengbin @SM047

VoVNet backbone is stronger for small object detection.

I also plugged VoVNet-57 into the FCOS and got this result.

This is the initial version and I expect further accuracy gain after optimizing the code.

[tianzhi0549]

@shadyatscu That's wonderful!!! Would you like to make your models publicly available? It will be very nice.

[aysark]

@stigma0617 AP@0.5 of 0.6 is not exactly 'stronger.' I've used SNIPER with similar results. The search continues...

https://github.com/mahyarnajibi/SNIPER