docs: add Before/After Detection Diff cookbook - #2477
Open
adawgwats wants to merge 6 commits into
Open
Conversation
Add a cookbook that classifies detections across a before/after photo pair as removed, persisted, or new: RF-DETR into sv.Detections, InferenceSlicer for small objects in high-resolution photos, ResNet18 crop embeddings, and Hungarian assignment over an appearance + position + class cost. Narrated as verifying a litter cleanup on the CC BY-SA 2007 Saint-Mandrier #trashtag pair; generalizes to shelf audits and site inspections. Registers the cookbook card. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pin every package in the before/after diff cookbook install cell to the exact versions the committed outputs were produced with, led by supervision==0.29.1 (the latest release), and bump the cookbook card's data-version to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reports how the technique performs on 1,236 real before/after cases: 693 verified cleanup pairs and 543 rejected candidates, of which 276 are same-scene-no-change hard negatives. Recall 73.9%, specificity 80.7%. Also records the finding that generalises past litter — check the detector's per-side recall before tuning the matcher, because a cost matrix cannot match objects that were never detected. In only 14% of those cases did the matcher face a genuine two-sided assignment problem. Evaluation images are third-party copyrighted and are not redistributed; these are reported measurements, not a downloadable benchmark. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #2477 +/- ##
=======================================
- Coverage 88% 88% -0%
=======================================
Files 85 85
Lines 12042 12042
=======================================
- Hits 10543 10540 -3
- Misses 1499 1502 +3 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds a new cookbook, Before/After Detection Diff (
docs/notebooks/before-after-detection-diff.ipynb), and registers its card indocs/theme/cookbooks.html.The cookbook answers a question none of the existing cookbooks cover: given two photos of the same scene taken at different times, which objects were removed, which persisted, and which are new? The walkthrough narrative is verifying a litter cleanup from a before/after photo pair — the workflow behind cleanup-verification features in field apps — and the closing notes generalize the recipe to shelf audits, equipment inspections, and parking surveys.
What the notebook demonstrates, using public supervision APIs only (zero library changes):
sv.Detections—rfdetr'spredictreturns supervision's container directly; nofrom_*conversion.sv.InferenceSliceron 3648x2736 photos — small litter vanishes at DETR input resolution; tiled inference recovers it (11 objects vs 2 without slicing on the demo pair). Cross-links the SAHI cookbook.sv.crop_imageper detection, ResNet18 with the classifier head dropped, L2-normalized 512-d vectors.scipy.optimize.linear_sum_assignment; matches costing > 0.75 rejected.BoxAnnotator+LabelAnnotatoroverlays, a pandas summary table, and a conservative verdict rule (verified only when the before photo had >= 1 detection and >= 1 went missing).sheepstill matches itself across the pair — matching doesn't need the label to be right) and where a fine-tuned RF-DETR checkpoint and a re-ID embedding slot in.Demo images
The default pair is the original 2007 Saint-Mandrier-sur-Mer beach cleanup (the pair that later inspired #trashtag), photographed by Stephane Belgrand Rousson and published on Wikimedia Commons by User:Aeroceanaute under CC BY-SA 4.0:
License and authorship are credited in the notebook's markdown. URLs are the stable
upload.wikimedia.orgoriginals and were verified to fetch.Result preview
On the demo pair the diff finds 11 objects before (bottles, cups, the blue crate, moored boats, one mislabeled bush) and 1 after; 1 persists (the bush, matched at cost 0.36), 10 come back removed, 0 new — verdict: cleanup VERIFIED.
Checklist (cookbook guidelines, CONTRIBUTING.md)
docs/notebooks/docs/theme/cookbooks.html(path, title, labels, author, supervision version)## Install required packagesheader, pinnedsupervision>=0.29.0)InferenceSlicer,crop_image, annotators,linear_sum_assignment)How it was tested
Executed top to bottom (fresh kernel) on Python 3.9 / supervision 0.29.1 / torch 2.8.0 CPU (Apple Silicon): both images fetch, tiled detection runs in ~10 s total, matching and verdict cells reproduce the numbers above,
nbformat.validatepasses.Related: #2476 (the
match_detections()proposal). This cookbook is docs-only and does not depend on that issue landing — it hand-rolls the assignment step withscipy, which is exactly the duplication the proposal is about.