Skip to content

docs: add Before/After Detection Diff cookbook - #2477

Open
adawgwats wants to merge 6 commits into
roboflow:developfrom
adawgwats:cookbook/before-after-detection-diff
Open

docs: add Before/After Detection Diff cookbook#2477
adawgwats wants to merge 6 commits into
roboflow:developfrom
adawgwats:cookbook/before-after-detection-diff

Conversation

@adawgwats

@adawgwats adawgwats commented Aug 7, 2026

Copy link
Copy Markdown

Description

This PR adds a new cookbook, Before/After Detection Diff (docs/notebooks/before-after-detection-diff.ipynb), and registers its card in docs/theme/cookbooks.html.

The cookbook answers a question none of the existing cookbooks cover: given two photos of the same scene taken at different times, which objects were removed, which persisted, and which are new? The walkthrough narrative is verifying a litter cleanup from a before/after photo pair — the workflow behind cleanup-verification features in field apps — and the closing notes generalize the recipe to shelf audits, equipment inspections, and parking surveys.

What the notebook demonstrates, using public supervision APIs only (zero library changes):

  • RF-DETR -> sv.Detectionsrfdetr's predict returns supervision's container directly; no from_* conversion.
  • sv.InferenceSlicer on 3648x2736 photos — small litter vanishes at DETR input resolution; tiled inference recovers it (11 objects vs 2 without slicing on the demo pair). Cross-links the SAHI cookbook.
  • Appearance embeddingssv.crop_image per detection, ResNet18 with the classifier head dropped, L2-normalized 512-d vectors.
  • Hungarian matching — cost matrix blending appearance (0.6), normalized center distance (0.4), and a soft class-mismatch penalty (0.15); scipy.optimize.linear_sum_assignment; matches costing > 0.75 rejected.
  • Removed / persisted / new classification, color-coded BoxAnnotator + LabelAnnotator overlays, a pandas summary table, and a conservative verdict rule (verified only when the before photo had >= 1 detection and >= 1 went missing).
  • An honest treatment of COCO-checkpoint limits (a bush detected as sheep still matches itself across the pair — matching doesn't need the label to be right) and where a fine-tuned RF-DETR checkpoint and a re-ID embedding slot in.

Demo images

The default pair is the original 2007 Saint-Mandrier-sur-Mer beach cleanup (the pair that later inspired #trashtag), photographed by Stephane Belgrand Rousson and published on Wikimedia Commons by User:Aeroceanaute under CC BY-SA 4.0:

License and authorship are credited in the notebook's markdown. URLs are the stable upload.wikimedia.org originals and were verified to fetch.

Result preview

On the demo pair the diff finds 11 objects before (bottles, cups, the blue crate, moored boats, one mislabeled bush) and 1 after; 1 persists (the bush, matched at cost 0.36), 10 come back removed, 0 new — verdict: cleanup VERIFIED.

Checklist (cookbook guidelines, CONTRIBUTING.md)

  • New notebook in docs/notebooks/
  • Card added to docs/theme/cookbooks.html (path, title, labels, author, supervision version)
  • Structure follows the most recent merged cookbooks (Colab badge at top, ## Install required packages header, pinned supervision>=0.29.0)
  • Self-contained: all data downloaded in-notebook, licenses documented, all installs in the first code cell
  • Code annotated with links to the docs of each tool used (InferenceSlicer, crop_image, annotators, linear_sum_assignment)
  • Executed end to end; outputs committed in the notebook

How it was tested

Executed top to bottom (fresh kernel) on Python 3.9 / supervision 0.29.1 / torch 2.8.0 CPU (Apple Silicon): both images fetch, tiled detection runs in ~10 s total, matching and verdict cells reproduce the numbers above, nbformat.validate passes.


Related: #2476 (the match_detections() proposal). This cookbook is docs-only and does not depend on that issue landing — it hand-rolls the assignment step with scipy, which is exactly the duplication the proposal is about.

adawgwats and others added 4 commits August 1, 2026 01:42
Add a cookbook that classifies detections across a before/after photo
pair as removed, persisted, or new: RF-DETR into sv.Detections,
InferenceSlicer for small objects in high-resolution photos, ResNet18
crop embeddings, and Hungarian assignment over an appearance + position
+ class cost. Narrated as verifying a litter cleanup on the CC BY-SA
2007 Saint-Mandrier #trashtag pair; generalizes to shelf audits and
site inspections. Registers the cookbook card.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pin every package in the before/after diff cookbook install cell to the
exact versions the committed outputs were produced with, led by
supervision==0.29.1 (the latest release), and bump the cookbook card's
data-version to match.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reports how the technique performs on 1,236 real before/after cases: 693
verified cleanup pairs and 543 rejected candidates, of which 276 are
same-scene-no-change hard negatives. Recall 73.9%, specificity 80.7%.

Also records the finding that generalises past litter — check the
detector's per-side recall before tuning the matcher, because a cost
matrix cannot match objects that were never detected. In only 14% of
those cases did the matcher face a genuine two-sided assignment problem.

Evaluation images are third-party copyrighted and are not redistributed;
these are reported measurements, not a downloadable benchmark.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@adawgwats
adawgwats marked this pull request as ready for review August 7, 2026 07:01
@adawgwats
adawgwats requested a review from SkalskiP as a code owner August 7, 2026 07:01
@Borda
Borda requested a lite review from Copilot August 7, 2026 16:03
@codecov

codecov Bot commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88%. Comparing base (fb22686) to head (1e2a9c6).

Additional details and impacted files
@@           Coverage Diff           @@
##           develop   #2477   +/-   ##
=======================================
- Coverage       88%     88%   -0%     
=======================================
  Files           85      85           
  Lines        12042   12042           
=======================================
- Hits         10543   10540    -3     
- Misses        1499    1502    +3     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant