Skip to content

feat: add roboflow_core/detections_difference@v1 fusion block - #2765

Merged
PawelPeczek-Roboflow merged 4 commits into
roboflow:mainfrom
adawgwats:feat/detections-difference-block
Aug 7, 2026
Merged

feat: add roboflow_core/detections_difference@v1 fusion block#2765
PawelPeczek-Roboflow merged 4 commits into
roboflow:mainfrom
adawgwats:feat/detections-difference-block

Conversation

@adawgwats

@adawgwats adawgwats commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

feat: add roboflow_core/detections_difference@v1 fusion block

Description

Adds a new fusion block that compares two sets of object-detection predictions of the
same scene at different times and reports the object-level set difference:

  • removed_detections — reference ("before") detections with no acceptable match in the
    candidate ("after") set, returned in the reference image's coordinates
  • persisted_detections / new_detections — matched / unmatched candidate detections,
    returned in the candidate image's coordinates
  • removed_count, new_count — the corresponding counts
  • verified — true when the reference set was non-empty and at least
    min_removed_to_verify detections were removed

Motivating use case: cleanup verification

The block was built for before/after cleanup verification: run a litter detector on a
photo of a littered area, run it again on an "after" photo taken from roughly the same
viewpoint once cleanup is claimed, and use removed_detections + verified as
photographic evidence that the reported cleanup actually happened — per-object, not just
"the image changed". The same Workflow shape covers any before/after audit:

  • shelf restocking / planogram checks (which SKUs disappeared or appeared),
  • construction-site or warehouse inspection rounds,
  • parking-lot occupancy deltas,
  • object-level disagreement between two models run on the same image.

How it relates to existing blocks

  • roboflow_core/overlap_analysis@v1 is the direct interface precedent: the manifest
    mirrors its reference_predictions / candidate_predictions two-detection-set input
    pair. Overlap Analysis relates two detection sets from the same image geometrically
    and emits per-pair overlap records; this block is its cross-time counterpart — it
    performs a one-to-one assignment between the two sets and emits the resulting set
    difference as first-class sv.Detections outputs that downstream blocks (filters,
    visualizations, sinks, webhooks) can consume directly.
  • Model Comparison Visualization (model_comparison) is the same-image cousin on the
    visualization side: it paints where two prediction sets disagree on one image, but its
    output is pixels. detections_difference produces the structured equivalent —
    matched/unmatched detections and counts you can branch on (e.g. Continue-If on
    verified) or persist.

Matching model

Each reference detection is matched with at most one candidate detection (and vice
versa) by minimizing:

cost = spatial_weight * spatial_term (+ class_mismatch_penalty)
  • spatial_term = 1 - IoU for overlapping pairs; for disjoint pairs it falls back to
    1 + centre_distance / enclosing_box_diagonal (a DIoU-style gradient), so any
    overlapping pair always beats any disjoint pair while disjoint pairs still rank by
    proximity.
  • class_mismatch_penalty (default 0.15) is added when classes differ;
    class_strict=True forbids cross-class matches outright.
  • Pairs whose cost exceeds reject_cost (default 0.75) are discarded — both sides then
    count as removed / new respectively.
  • Assignment is a dependency-free greedy lowest-cost pass. scipy.optimize. linear_sum_assignment would be optimal (Hungarian), but scipy is not among the base
    requirements of this repository, so the block stays numpy-only; for the low detection
    counts typical of before/after comparisons the assignments rarely differ. The trade-off
    is documented in the greedy_match docstring and pinned by a dedicated unit test, so a
    future switch to an optimal solver shows up as a deliberate test change.

Outputs are produced by fancy-indexing the input sv.Detections, so every data key
(detection ids, parent coordinates, ...) is propagated and downstream coordinate
re-projection keeps working. No new kind is introduced — all outputs use existing
object_detection_prediction / integer / boolean kinds with their registered
serializers.

Type of change

  • New feature (non-breaking change which adds functionality)

How has this change been tested

tests/workflows/unit_tests/core_steps/fusion/test_detections_difference.py
(31 tests, mirrors test_overlap_analysis.py conventions) covers:

  • manifest validation: happy path, DetectionsDifference alias + defaults, out-of-bounds
    rejection for spatial_weight, class_mismatch_penalty, reject_cost,
    min_removed_to_verify
  • describe_outputs() names == run() keys
  • outcome matrix on synthetic detections: identical sets (all persist), full cleanup
    (all removed, verified), partial cleanup (removed + persisted mix), new detections,
    output types
  • reject-cost edge: a disjoint same-class pair whose cost lands just above the default
    0.75 (rejected → removed + new) vs. the same geometry with the threshold raised by 1e-6
    (matched); an IoU-driven variant; and exact-equality acceptance at reject_cost
  • class handling: penalty pushing a pair over the threshold, class_strict with an
    absurdly permissive reject_cost, class-less inputs matching purely spatially
  • verified gating via min_removed_to_verify; empty reference / empty candidate /
    both empty
  • assignment behaviour: 2x2 nearest pairing, one candidate claimed at most once, greedy
    global-cheapest-first order, and the documented greedy-vs-Hungarian suboptimality case
  • loader registration (block present in load_blocks())

tests/workflows/integration_tests/execution/test_workflow_with_detections_difference_block.py
(3 tests) runs the block inside a compiled workflow, hermetically: detections are
injected via WorkflowBatchInput entries of kind object_detection_prediction
(EE >= 1.3.0) instead of chaining two model steps, so no model weights or network
access are needed. Covers the removed/persisted/new split with detection_id
propagation, verified gating on the no-change case, and
serialize_results=True round-tripping of the detection outputs to the wire
format.

Results (local, Python 3.12.13, base requirements + repo on PYTHONPATH):

tests/workflows/unit_tests/core_steps/fusion/test_detections_difference.py    31 passed
tests/workflows/unit_tests/core_steps/fusion/  (full directory, regression)  287 passed
tests/workflows/integration_tests/execution/test_workflow_with_detections_difference_block.py  3 passed

black --check / isort --check clean on the touched source and test files.

Docs

Block docs are autogenerated from the manifest Field descriptions and
LONG_DESCRIPTION — no hand-written docs page added, per
docs/workflows/create_workflow_block.md.

Screenshot

(two Object Detection Model steps → Detections Difference → Continue-If on verified)
to be added here.

Follow-ups (not in this PR)

  • Optional model-driven integration test wiring the block behind two
    RoboflowObjectDetectionModel steps (pattern:
    test_workflow_with_overlap_analysis_block.py, needs weights + network) +
    add_to_workflows_gallery entry. A hermetic WorkflowBatchInput-based
    integration test is already included in this PR.

Related: roboflow/supervision#2476 proposes the same primitive at the library level. This block is self-contained and does not depend on it — the cost matrix here is plain numpy, and the base requirements stay torch-free.

adawgwats and others added 2 commits August 1, 2026 01:36
Match object detections across before/after images of the same scene and
report removed, persisted and new objects plus a verified flag. Cost is
spatial (1 - IoU, DIoU-style normalised centre-distance fallback for
disjoint boxes) plus an optional class-mismatch penalty; assignment is a
dependency-free greedy lowest-cost pass (scipy is not in base
requirements) with a reject-cost cutoff. Cross-time counterpart of
overlap_analysis@v1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Exercise the block inside a compiled workflow via WorkflowBatchInput
entries of kind object_detection_prediction (EE >=1.3.0), avoiding the
model-weight and network dependencies of the model-step pattern used by
the overlap_analysis integration test. Covers the removed/persisted/new
split with detection_id propagation, verified gating on the no-change
case, and serialize_results=True round-tripping of the detection
outputs to the wire format.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@PawelPeczek-Roboflow
PawelPeczek-Roboflow merged commit 66e7124 into roboflow:main Aug 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants