fix: measure a torso against the frame it was actually cropped from - #36
Merged
Conversation
Identifying a child only ever labels them where the user happened to look, and re-identification could only confirm a binding where one already existed. On a real game that left an athlete known for 31.7s of 300s across six fragments, all inside the one 32-second window originally clicked. Every signal that follows the athlete was dark for the other 90%, and the moments that survived were scene-wide ones with nothing to do with them. A jersey is the one thing about a child a detector can see that stays the same all afternoon, so the worker gains an `appearance` command: a coarse HSV histogram of the torso, from one pass over the 540p proxy. Boxes arrive on stdin because there are thousands of them. Colour is a veto, not an identifier, and that is the whole design. Measured against the production game, three ways: colour only (>= 0.55) 661 of 1152 tracks, 2306s of "athlete" in a 300s video continuity only 56 tracks, 120.3s both 14 tracks, 51.2s (from 6 tracks, 31.7s) Teammates wear the same shirt, so colour alone selects a *team* — and the children it wrongly volunteers are exactly the ones standing next to yours. Continuity alone links whoever happens to be nearby. The identity claim therefore rests on continuity — a fragment that begins where and when another ended, within 2s and a distance a child could actually run — with colour able only to rule a link out. At a 4s gap the accepted links reached 894px of a 1920-wide frame for eight more seconds of coverage, which is where that trade stops being worth taking. Nothing is assigned. Matches are pre-selected in the picker with the evidence behind each one — the gap, the distance, the colour agreement — and a human confirms, because the cost of a confident wrong answer is another family's child in your highlight reel. Also: `run` can write to a child's stdin, which the box list needs. Verified against production via scripts/appearance-probe.mjs and scripts/stitch-probe.mjs, both read-only; the numbers above are theirs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
#35 shipped appearance matching and it returned zero matches on the very footage it was designed against — where a read-only probe of the same algorithm had found eight. Two faults, both in the seam between the parts, both invisible to every unit test because each half was right on its own. The coordinate space. Tracks are stored in source-video pixels, and appearance deliberately reads the 540p proxy because a shirt's colour survives that and decodes in a fraction of the time. The worker took the box space from the file it had opened, so 1920-wide boxes were measured against a 960-wide frame at a scale of 1: every torso rect landed off the right-hand edge, was clamped to nothing or to background, and the signatures came back empty or meaningless. The space now travels with the boxes, in the same JSON payload, because it is a property of the coordinates and not of whatever file happens to be open. The candidate floor. Proposals reused the picker's 1.5s minimum, which exists so a human has a crop long enough to recognise. Stitching wants the opposite: short fragments are the connective tissue, and five of the eight links that recovered a real athlete were under 1.5s. Lowered to 0.25s, where continuity and colour carry the claim instead of a person's eyes. `decodePlanFor` is now a pure function with the scaling rule stated once, and signatures.integration.test.ts builds a two-colour video with ffmpeg and checks that a box quoted in a larger space still lands on the right half of it — skipped where ffmpeg is absent, as the model test already is. Verified against real pixels: left vs right 0.0000, left vs left 1.0000, both crops non-empty. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PR #35 was squash-merged as fea3483, which is this branch's own 2833a9d with a new hash and no ancestry link. Git therefore saw the four files that commit introduced as add/add conflicts against themselves. There is no content disagreement to settle: origin/master's tree is byte-identical to 2833a9d, so every conflict resolves to this branch's side and the merge result is exactly the tree of c8efaef. Verified with `git diff c8efaef` being empty after resolution. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…footage The reason #35 shipped returning zero matches is not that the judgement was wrong — it was that nothing could exercise it. Every decision lived in a module that also reaches a database, a subprocess and a filesystem, so importing any of it drags in a native driver, so the only way to try it against real footage was to re-implement it in a probe. A probe that agrees with a re-implementation proves nothing about what ships, and the bug was in the plumbing between the two halves, where neither was looking. `stitch.ts` now holds the whole decision and touches nothing: the link rule, the colour veto, the candidate filter and the iterative chain. `appearance.ts` keeps only fetch, call and return. The probe imports the same `chooseAthleteTracks` the app calls rather than a copy of it. Run against the production project, read-only, with the shipped code: reference 6 track(s), candidates 2598, 10090 boxes, 600 frames in 5s +218.0-218.9s gap 0.60s dist 120px colour 0.770 +251.5-253.8s gap 1.07s dist 379px colour 0.786 +255.7-256.3s gap 1.93s dist 152px colour 0.867 +256.3-259.5s gap 0.03s dist 60px colour 0.861 +260.7-261.0s gap 1.27s dist 85px colour 0.769 +253.9-254.4s gap 1.33s dist 50px colour 0.724 +262.0-266.9s gap 0.97s dist 405px colour 0.734 +266.9-273.9s gap 0.07s dist 52px colour 0.787 before: 6 track(s), 31.7s after : 14 track(s), 51.2s Ten tests over the isolated logic, including the two that matter: a matching shirt that continues nothing is refused, and a fragment on screen beside the athlete is refused however well its colour agrees. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… into fix/appearance-coordinate-space
ralyodio
added a commit
that referenced
this pull request
Aug 10, 2026
PR #36 was squash-merged as 9e07183, which carries this branch's own 2833a9d, c8efaef, 73320b9 and aa3276b under a single new hash with no ancestry link. Git therefore saw scripts/stitch-probe.mjs as a content conflict against a version it already contains. There is nothing to settle: origin/master's tree is byte-identical to aa3276b, and this branch is aa3276b plus 5bffe86, whose only change to that file is the four lines that print track ids. The conflict resolves to this branch's side and the merge result is exactly the tree of 5bffe86, verified by `git diff 5bffe86` being empty afterwards. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #35's feature, which returned zero matches on the exact footage it was designed against — where a read-only probe of the same algorithm found eight.
I caught this by calling the live endpoint after deploying, not from the test suite: every unit test passed, because each half was correct on its own. Both faults were in the seam.
1. The coordinate space
Tracks are stored in source-video pixels (1920×1080). Appearance deliberately reads the 540p proxy, because a shirt's colour survives that and decodes in a fraction of the time.
The worker took the box space from the file it had opened. So 1920-wide boxes were measured against a 960-wide frame at a scale of 1 — every torso rect landed off the right-hand edge, got clamped to nothing or to background, and the signatures came back empty or meaningless. Downstream everything behaved: no crash, no error, just a confident "no matches".
The space now travels with the boxes, in the same JSON payload. It is a property of the coordinates, not of whatever file happens to be open.
2. The candidate floor
Proposals reused the picker's
minSeconds: 1.5, which exists so a human has a crop long enough to recognise a face in.Stitching wants the opposite. Short fragments are the connective tissue — five of the eight links that recovered a real athlete were under 1.5s. Lowered to 0.25s, where continuity and colour carry the identity claim instead of someone's eyes.
Tests
decodePlanForis now a pure function with the scaling rule stated once, covered by unit tests including the exact 1920→960 case that broke.signatures.integration.test.tsbuilds a two-colour video with ffmpeg and checks that a box quoted in a larger space still lands on the right half of it. It usesdescribe.skipIf, matchingmodel.integration.test.ts; the container image has ffmpeg.Verified against real pixels on the box that has it:
556 tests pass, 8 skipped. Lint and full build clean.
Still to confirm
Whether Joe's coverage actually grows from 31.7s once this is live. The read-only probe says 51.2s across 8 more fragments; this PR is what makes the shipped path agree with it. I'll measure on the live service after deploy and report the real number.
🤖 Generated with Claude Code