Repository navigation
ClipMind v1.1.0 — transcript novelty on every visual state
Adds a deterministic signal describing how much of a visual state's on-screen
text the speech never carried, and gives preview derivation one safety net.
Evidence Pack schema moves to 1.1.0; packs written by 1.0.0 stay readable.
Why
Burned-in captions repeat what is being said, so the transcript already holds
that information. A slide, document or code pane shows text nobody reads
aloud, and the frame is the only place it exists.
Measured on the 64-second interview video: document frames carry 68–344
characters the nearby speech never covers, caption frames carry about 7.
New fields on every visual state
| field | meaning |
|---|---|
ocr_char_count |
characters across recognised tokens |
transcript_novelty_char_count |
characters the nearby speech does not cover |
transcript_overlap_ratio |
1 - novelty / total |
Comparison is a multiset over tokens — one per CJK ideograph, one per latin
word — after NFKC normalisation and case folding, windowed around the state's
own interval, so a term shown three times but spoken once is not treated as
fully covered.
What this is not
Not importance. A silent video makes every frame novel, code identifiers are
novel because nobody says them out loud, and an ASR slip manufactures novelty.
The field is a signal for a downstream agent to weigh, not a verdict.
Evidence stays untouchable
Annotation runs after canonical retention, never before, so an ASR or OCR
mistake cannot delete evidence. The preview safety net requires enough
unspoken text and no already-selected state showing substantially the same
terms; on all five real packs it adds zero frames, because every high-novelty
state was already selected. It is insurance against a brief document inside
one long visual scene, demonstrated by a synthetic case rather than asserted.
Compatibility
The published schema accepts 1.0.0 and 1.1.0. Existing packs remain readable
and reusable by the cache; scripts/rebuild_preview.py upgrades one in place,
adding the measurements without re-downloading or re-running OCR.
Also fixed
A failed resync silently restored the stuck state the resync exists to escape.
refreshJobs() now retries with backoff and surfaces a message.
Verification
91 tests (11 new), CI green, linear history.
Full changelog: v1.0.1...v1.1.0