False Negative detections

[dmitrysarov]

We can name False Negative detection as those, which we received after hight confidences threshold.
But at the same time we got FN from IoU calculation stage as rejecting gt boxes. For example on image with 3 gt boxes we detect only one prediction box, and it got big enough IoU, mAP metric don't care about those 2 not detected?

[rafaelpadilla]

Dear @dmitrysarov ,

I believe your question is about detections G, H and I on the Image 3. In this case as 3 detections overlap the same ground truth, we need to consider the detection with the highest IOU as a TP and the other 2 as FP. Therefore, as G has the highest IOU, G is the TP and H and I are FP.

Even though H and I are FP, they do influence on the calculation of the Average Precision (AP). Check how they are taken into account on this table by influencing on the accumulated FP (acc FP).

As AP can be seen as the area under the curve (AUC), notice on this graph and this how H and I are important points and influence on the AUC.

I hope you got the point.

Regards.

[dmitrysarov]

Thank you for your respond @rafaelpadilla.
Unfortunately it not quite what I'm talking about
Here is illustration. On the first picture we have one ground truth and one prediction box, and we can decide whether it TP or FP by choosing IoU threshold.

At same time on the second picture we have additional ground truth boxes. Anyway those boxes are False Negative boxes, because model reject them. But it seems to be that mAP metric don't care about it and will be the same as for first picture case.

[dmitrysarov]

I will add reference to original paper http://host.robots.ox.ac.uk/pascal/VOC/pubs/everingham15.pdf

Ground truth objects with no matching detection are false negatives

Can you point in your code where you consider such type of False Negative please.

[rafaelpadilla]

Dear @dmitrysarov .

I think I got what you mean now. Your question is very relevant! :)

But yes, the FN are taken into account. Let me try to explain where the False Negatives are considered:

We defined recall as being: Recall = TP / (TP+FN)
and (TP+FN) = total amount of ground truths
Therefore, Recall = TP / (total amount of ground truths)
It was explained here in the README.md

So when I calculate the accumulated Precision and Recall values in the last columns (see here), notice that for recall we have:
Recall = Acc TP / 15

And 15 is the total amount of ground truths of the dataset, which already takes into consideration all the FP :)

Got it?

[dmitrysarov]

Finally I got it =)
thank you alot!