fix: give a track an id the next ten thousand tracks cannot take - #51
Merged
Conversation
"SQLITE_CONSTRAINT: UNIQUE constraint failed: tracks.id", partway through building a reel from clips. Every id in the system was the first eight hex characters of a UUID: 4.3 billion values, which is ample for the tables a person can name and far too few for the one that fills up on its own. Detection writes a row per track, and a single production video already carried 8,394 of them -- the number is in analyze.ts, in the comment explaining why a score-only re-run reports zero. Vetting a shirt before a track may claim whoever is standing there (#50) trades spliced identities for fragments, so it multiplies that count rather than reducing it, and a project assembled from a dozen clips holds the sum: detection clears the tracks of the video it is re-analysing, not of the project. The birthday bound over 32 bits, measured rather than assumed: 8k tracks lose one run in 125, 50k lose one in four, 100k lose more often than they win. That is not a rare collision, it is the expected outcome of the flow that accumulates the most, which is why building from clips is where it surfaced. So ids carry 64 bits. The same 100k rows now collide with probability 3e-10. Widening rather than retrying on the constraint is the point: a retry loop would hide the rate instead of removing it, and it would have to sit on every insert to be honest about where else this was possible. Ids already written stay valid and stay readable -- eight characters and sixteen cannot collide with each other, so no migration is owed and no existing project has to be re-analysed for the fix to hold. The test generates 200k ids and expects 200k distinct ones. Against the old scheme it returns 199,995. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes
SQLITE_CONSTRAINT: UNIQUE constraint failed: tracks.idwhen building a reel from clips.Cause
newIdbuilt every id from the first eight hex characters of a UUID — 32 bits, 4.3 billion values. Ample for projects, videos and reels; far too few fortracks, the one table that fills up on its own. Detection writes a row per track, and a single production video already carried 8,394 of them (the number is inanalyze.ts, in the comment explaining why a score-only re-run reports zero). Vetting a shirt before a track may claim whoever is standing there (#50) trades spliced identities for fragments, so it multiplies that count rather than reducing it.A project assembled from a dozen clips holds the sum, because detection clears the tracks of the video it is re-analysing, not of the project. That is why building from clips is where this surfaced.
The birthday bound, measured
Not a rare collision — the expected outcome of the flow that accumulates the most.
Fix
Ids carry 64 bits (
randomBytes(8)), so the same 100k rows collide with probability 3e-10.Widening rather than retrying on the constraint is deliberate: a retry loop would hide the rate instead of removing it, and it would have to sit on every insert to be honest about where else this was possible. Ids already written stay valid and stay readable — eight characters and sixteen cannot collide with each other — so no migration is owed and no existing project has to be re-analysed for the fix to hold. Re-analysing a broken project now succeeds because it re-inserts with wide ids.
Checked for format assumptions:
parseFrameFileNamesplits on the last_(length-agnostic), and the uploadCLIENT_IDregex already accepted{8,32}. One test pinned the old width and is updated.Verification
packages/core/src/ids.test.tsgenerates 200k ids and expects 200k distinct. Against the old scheme it returns 199,995 — confirmed by temporarily reverting the implementation and watching the test fail.Full suite 691 passed / 8 skipped;
pnpm lintandpnpm -r typecheckclean.🤖 Generated with Claude Code