fix(metrics): count false positives on empty-GT images - #2397
Merged
Conversation
- legacy `MeanAveragePrecision.from_tensors` appended per-image stats only when ground truth was present, so predictions on background images were never counted as false positives and mAP was inflated - append an FP-only stats tuple for prediction-present, target-empty images, mirroring the pattern in `f1_score.py` - add regression test asserting background false positives lower mAP --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #2397 +/- ##
=======================================
Coverage 85% 85%
=======================================
Files 70 70
Lines 10035 10036 +1
=======================================
+ Hits 8489 8490 +1
Misses 1546 1546 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the legacy MeanAveragePrecision.from_tensors computation in src/supervision/metrics/detection.py so that predictions made on images with no ground-truth objects (background images) are counted as false positives and reduce the resulting mAP, aligning the metric with expected detection-evaluation behavior.
Changes:
- Updated
MeanAveragePrecision.from_tensorsto record all predictions on GT-empty images as false positives in the collected per-image stats. - Added tests ensuring background false positives reduce
map50and that the GT-present path remains numerically stable.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/supervision/metrics/detection.py |
Adjusts from_tensors stats collection to include GT-empty images so predictions are penalized as false positives. |
tests/metrics/test_detection.py |
Adds regression tests covering background false positives and a pinned GT-present scenario. |
- Guard empty average_precisions in from_tensors: return 0.0 for map50/map75/map50_95 when no GT classes exist (all-background dataset) - Fix dtype of empty GT class array: np.zeros((0,), dtype=np.float32) to match explicit-dtype pattern at line 1408 and avoid float64 upcasting [resolve #1] Copilot + /review — foundry:qa-specialist + foundry:challenger (gh) [resolve #6] /review finding by foundry:linting-expert (report) --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
- Add test_all_background_images_return_zero_not_nan: verifies that a dataset with only background images (no GT objects anywhere) returns map50=0.0 rather than NaN after the from_tensors fix - Strengthen test_background_false_positives_lower_map: add upper-bound assertion (map50 < 0.5), add map75 and map50_95 penalization checks [resolve #2] Copilot + /review — foundry:qa-specialist (gh) [resolve #4] /review finding by foundry:qa-specialist (report) --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
…ring - Remove "Legacy" from TestMeanAveragePrecisionBackgroundFalsePositives docstring — from_tensors is the active tensor constructor, not deprecated - Add N=0 background-image contract to targets arg description in from_tensors docstring (empty array = all predictions are false positives) - Add docstring example demonstrating that predictions on a background image produce map50=0.0 (covers new empty-stats code path) - Fix Returns section: add MeanAveragePrecision type prefix per Google style [resolve #3] /review finding by foundry:doc-scribe (report) [resolve #5] /review finding by foundry:doc-scribe (report) [resolve #7] /review finding by foundry:doc-scribe (report) [resolve #8] /review finding by foundry:doc-scribe (report) --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Borda
added a commit
that referenced
this pull request
Jul 3, 2026
Resolve duplicate-change conflicts: several commits from this branch were cherry-picked into separate PRs (#2391/#2393/#2394/#2396/#2397/#2398/#2399) and squash-merged to develop, so merging develop back produced same-change conflicts across 19 files. Resolution: - Keep this branch's later refactors: public `check_no_basename_collisions` (standardized collision-guard layer, renamed all `_check_*` call sites in dataset/core, yolo, labelme, tests), `save_pascal_voc_annotations` helper, value-based from_lmm->from_vlm mapping (drop dead lmm_to_vlm dict), CropAnnotator source-scene snapshot for overlapping boxes. - Take develop's hardening/refinements: from_vlm empty-list guard, explicit float() mAP casts, np.float32 dtypes, MJPG/avi VideoSink tests, shared make_panoptic_png test helper, expanded empty-input adapter tests. - Combine where both improve: np.int32 crop dtype + snapshot; keep both sides' added tests. Full suite: 2924 passed, 1 xfailed. pre-commit (ruff, mypy, codespell) clean. --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Draft
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request updates the mean average precision (mAP) metric calculation for object detection, ensuring that predictions on images with no ground-truth objects (background images) are properly penalized as false positives. This change corrects the prior behavior where such predictions did not reduce the mAP score, making the metric more accurate. The update is accompanied by new tests to verify the improved logic.
Metric calculation improvements:
from_tensorsinsrc/supervision/metrics/detection.pyto treat all predictions on ground-truth-empty images as false positives, ensuring these predictions lower the mAP score as expected.Testing enhancements:
TestMeanAveragePrecisionBackgroundFalsePositivesintests/metrics/test_detection.pywith tests verifying that background false positives reduce mAP, and that normal cases without false positives are unaffected.