feat(captions): whisper transcription and SRT/VTT import#58
Merged
Conversation
- new captions command: transcription via a detected whisper.cpp
binary (whisper-cli/whisper-cpp/main, subprocess like the ffmpeg
philosophy; actionable install/model hints), or fully-offline
--from-srt/--from-vtt with hand-rolled parsers (BOM, CRLF,
multiline, styling tags, both timestamp forms); word timing spread
uniformly across each cue (documented approximation)
- output {words: [{text, start, end}]} plugs straight into --props
with "words": "$words" (verified end-to-end srt -> render)
- CaptionStyle gains word_pop (active word only, ease-out-back pop,
rounded pill background) and karaoke_pop (line layout, active word
1.15x + pill); pill_color on Caption, default 70% black; existing
modes untouched
- captions-workflow skill rule documents the two-step pipeline
10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #45.
Goal: word-level captions from speech — the caption component existed but words were 100% manual.
rustmotion captions: transcription through a detected whisper.cpp binary (whisper-cli→whisper-cpp→main, help-probed; subprocess-first like ffmpeg — zero build deps), with actionable errors for missing binary (brew hint +--from-srtsuggestion) and missing model (full huggingface curl command). Word splitting via-ml 1 -sow -oj; punctuation-only segments merged into the previous word. Flags coded against the documented JSON format (no binary on this machine — noted in the module doc).<i>/<b>stripping,,/./short timestamps, VTT NOTE/STYLE/REGION blocks). Word timing spread uniformly per cue — documented approximation.{"words": [...]}plugs straight into the feat(cli): props injection and batch rendering over the variables system #56 props system (--props words.json+"words": "$words") — verified end-to-end SRT → validate → still.word_pop(active word only, ease-out-back pop, rounded pill) andkaraoke_pop(line layout, active word 1.15× + pill),pill_colorwith 70%-black default; the 3 existing modes untouched.captions-workflowskill rule documents the pipeline.Tests: 20 new — 15 CLI (parsers, timing spread, JSON shape, hints, whisper-JSON parse) + 5 pixel tests on the presets through the real pipeline. 292 total, all green.
Verify:
cargo test --workspace→ 292 passed ✓ ·cargo fmt --check✓ · clippy 0 warnings (--all-targets) ✓ · manual SRT→JSON smoke ✓