retinanet num_classes includes the background

[ABCCASa]

📚 The doc issue

https://pytorch.org/vision/master/models/generated/torchvision.models.detection.retinanet_resnet50_fpn_v2.html

Hi, I am currently use the retinanet, the documentation says num_classes of retinanet includes the background, but when I look at the source code of RetinaNetClassificationHead, it use a tensor with all zeros to represent background. So do we still need to use index=0 as the background?

Suggest a potential alternative/fix

Maybe num_classes is exclude background?

[NicolasHug]

Sorry @ABCCASa , I don't understand the question.

So do we still need to use index=0

What does that refer to?

[ABCCASa]

Hi, @NicolasHug , Thanks for your reply,
I mean, for example, should I use ["background", "apple", "cat", "bottle", "people"] or ["apple", "cat", "bottle", "people"].
The value at index = 0 should always be background in faster rcnn. But in retinanet, it may not. I looked at the source code, and it threatens each class separately as a binary problem. For a box, if the value is close to 1 (e.g. [0.9, 0.1, 0.1]), then it belongs to this class. Otherwise, it's not. If the box does not belong to any of the classes, it will be the background (e.g., [0.001, 0.001, 0.013]). The retinanet uses sigmoid, not softmax, all class scores sum up do not need to be 1.

[NicolasHug]

Does this help: #4106 (comment) ?

the num_classes should include the background which is encoded with 0. During inference the model predicts labels starting from 1.

[ABCCASa]

@NicolasHug Yes, Thank you so much.