feat(run_episodes): batch runner for episode directories (depends on #41)#42
Open
xiaogang-sudo wants to merge 8 commits into
Open
feat(run_episodes): batch runner for episode directories (depends on #41)#42xiaogang-sudo wants to merge 8 commits into
xiaogang-sudo wants to merge 8 commits into
Conversation
Independent helper that assembles a final cut by aligning source ranges to an SRT timeline, bypassing the existing transcript-based EDL flow. Use when you have a finished script (script.srt = final captions timeline) and a list of source ranges keyed by SRT id. Pipeline: parse SRT + plan -> strict validate -> align -> extract segments (per-source ffprobe, HDR tone-map, sync tails, cache) -> gap clips for non-contiguous SRT cues -> lossless concat -> final pass with optional global voice mix + subtitle burn LAST (Hard Rule 1). Key correctness properties: - All intermediates land in a safe-ASCII temp work_dir; CJK / quoted user paths never reach libavfilter or the concat demuxer. - SRT input decoded with utf-8-sig / utf-8 / gb18030 / cp936 / cp1252 fallback; cue settings (position:90% etc.) tolerated. - Per-segment cache keyed by ffmpeg version + encoding params + effective bg_volume so encoder tweaks invalidate stale clips. - Source streams probed once; no-audio source auto-degrades bg_volume to 0 for its segments; out-of-bounds ranges fail fast. - Global --voice spans the whole timeline (apad/atrim to total_duration in the final compose), not per-segment — a 5s VO does not restart at every cut. - 30ms audio fades + fps=24,setpts and aresample sync tails on every segment prevent A/V drift through many short concats. - burn_subtitles is self-defending: unsafe subs paths are copied to a temp ASCII SRT before being fed to libavfilter. - Batch (jobs.json / .csv) auto-isolates outputs by manifest index; --continue-on-error skips failing rows; --no-overwrite refuses to clobber existing outputs. Includes examples (Form A array, Form B object with multi-source + voices, batch manifest, CJK SRT) and pytest coverage (14 e2e + batch tests using lavfi-synthesized media; passes against ffmpeg 8.x on Windows). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…cript
Bridges the gap between Scribe word-level transcripts and the
srt_driven_edit pipeline. Given a final-cut script.srt and a source
recording's Scribe JSON, produces an edit_plan.json (Form A or B) plus
a sidecar review markdown for human-in-the-loop QA.
Matching strategy is intentionally local (no LLM, no API):
1. Filter the transcript to timestamped 'word' tokens (audio_event /
spacing skipped; --keep-audio-events keeps markers as context).
2. Group consecutive words into non-overlapping candidates, breaking
on sentence-end punctuation, silences >= gap_threshold, or speaker
change. Long candidates split at phrase punctuation, then by hard
word-level windows. All edges land on word boundaries.
3. Score each (cue, candidate) pair as
0.7 * (0.6 * SequenceMatcher + 0.4 * Jaccard)
+ 0.3 * 1/(1+|dur_delta|/cue_dur)
where Jaccard auto-switches between Latin word-token and CJK
character-bigram representations.
4. Greedy assignment; --allow-reuse drops the no-reuse constraint.
5. Emit Form A (default, drop-in for srt_driven_edit --plan) or Form
B; review markdown lists matched text, score, duration delta, and
warnings (low score / duration mismatch / candidate-shorter-than-
cue).
Hard failure modes (exit 1): any cue with no assignable candidate;
malformed transcript JSON; transcript with no word tokens.
Soft failures (warnings only): low score, candidate too short for cue.
The matcher cannot understand storyline — if SRT narration words do
not appear in the source transcript, scores will be low. The sidecar
review.md is the manual QA surface; it is intentionally not pulled
into the plan (parse_plan in srt_driven_edit stays strict).
--packed (takes_packed.md) and --context-window flags are reserved
placeholders only; both raise no error but do not yet alter behavior.
Includes 11 pytest tests including a full end-to-end:
recommend -> sde.run_job -> final.mp4 against lavfi-synthesized media.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
CLAUDE.md is auto-loaded by Claude Code when working in this directory, giving sessions a consistent picture of the project's scope, tech constraints, and out-of-bounds behaviors before the user has to say it. AGENTS.md does the same for Codex review sessions, classifying review output into must-fix / should-improve / later so suggestions are actionable rather than open-ended rewrites. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Convention-driven multi-episode runner. Given a root containing one subdirectory per episode, discovers eps that have the required file set and runs srt_driven_edit on each. Complements the existing jobs.json / jobs.csv manifest path with a flatter, zero-config workflow. Per-episode layout (all under <root>/<ep>/): source.mp4 required script.srt required edit_plan.json required (Form A or B) voice.wav optional — wired in as the ep's global voice Outputs land at <root>/<ep>/final.mp4 with edit/ artifacts (EDL, QC, cache) inside each ep dir; an aggregate summary lands at <root>/run_episodes_summary.json. Dirs missing required files are SKIPPED with a printed reason rather than aborting, so a partial batch is still actionable. Hard-fails only when no usable ep is found. --continue-on-error makes per-ep ffmpeg failures non-fatal too; without it, the first failure aborts the run. Process exits non-zero if any episode failed, even in continue mode. Includes 7 pytest cases: - discover skips incomplete dirs without erroring - discover picks up voice.wav when present - empty root raises - full e2e with 3 synthetic eps each producing final.mp4 - continue-on-error skips ep with out-of-bounds range, finishes others - hard abort without continue-on-error - per-ep voice.wav reflected in QC audio.mode Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Until now a failed batch row only recorded {job, ok: False, error}. To
diagnose an ffmpeg crash you had to scroll the terminal back; for a
malformed manifest row you had no idea which row index errored. This
commit adds a structured diagnostic payload to every failure entry in
both the srt_driven_edit batch path and the run_episodes path.
New shape per failed entry:
{job, ok: False, index, error,
srt, plan, source, output,
stderr_tail}
- `index` is the 0-based position in the manifest / discovered ep list,
so the summary trivially round-trips back to the bad row.
- `srt` / `plan` / `source` / `output` come from the resolved Job when
available; for rows that crash inside job_from_dict (no Job yet),
they fall back to the raw manifest_row dict so context is never lost.
- `stderr_tail` is the last 30 lines / 2 KB of ffmpeg's stderr,
populated only when the failure originated in run_ff. Pre-flight /
validation errors leave it empty by design.
To carry the stderr tail without breaking the existing
`except SystemExit:` pattern, add a `PipelineError(SystemExit)` subclass
with a `.stderr_tail` attribute, raised by `run_ff` on non-zero exit.
Existing handlers continue to work via `getattr(e, "stderr_tail", "")`.
The new helper `make_failure_record(...)` is exported from
srt_driven_edit and reused by both the CLI's batch loop and
run_episodes.run_episodes so the two paths stay in sync.
Tests added (4):
- test_run_ff_raises_pipeline_error_with_stderr — direct unit test
of PipelineError carrying real ffmpeg stderr
- test_batch_failure_record_includes_paths — out-of-bounds range fails
pre-extract; record carries index/srt/plan/source/output, empty
stderr_tail
- test_batch_malformed_row_failure_record — row missing 'plan' still
yields a usable record sourced from the raw manifest row
- test_run_episodes_failure_record_includes_paths — same for the
directory-based runner
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Thin wrapper over helpers/srt_driven_edit.py that fills in the layout
described in CLAUDE.md:
input/source.mp4 + input/script.srt + input/edit_plan.json
--(python main.py)-->
output/final.mp4
Behavior:
- `--srt`, `--plan`, and `-o` defaults are injected when the user did
not supply them; output/ is auto-created.
- `--source` and `--voice` defaults are injected only when the
corresponding file actually exists under input/, so Form B users
without input/source.mp4 do not get a misleading "missing on disk"
error from a defaulted flag they never wanted.
- Both bare (`--srt foo`) and equals (`--srt=foo`) forms count as
user-supplied; no double-injection.
- `--batch <manifest>` short-circuits all single-job defaults so the
manifest fully owns its paths.
The wrapper performs argv rewriting then forwards to
srt_driven_edit.main(), so every existing flag (style, bg-volume,
no-overwrite, continue-on-error, etc.) keeps working unchanged.
7 unit tests cover: bare defaults, source/voice file-gated injection,
user-flag precedence, equals-form recognition, batch short-circuit,
and the short -o alias.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A standalone, self-contained scaffold for the SRT-driven editor that
deliberately does not touch video. Reads script.srt + edit_plan.json
(Form A only), validates id matching with clear error messages, and
prints each cue's planned source-time range alongside the cue's output
range and a text preview.
Why a separate, smaller file when helpers/srt_driven_edit.py already
exists: this version is meant to be read top-to-bottom in one sitting.
It has zero imports from helpers/, no dependency on ffmpeg, and ~150
lines including comments and blank space. It is the natural starting
point for someone learning the pipeline before the production code.
Scope strictly per spec:
- parse SRT (utf-8-sig, CRLF, cue settings tolerated)
- parse plan (Form A only — Form B is explicitly rejected with a
pointer to the full pipeline)
- validate id sets match, with duplicate detection on both sides
- print the cue/source-range table
Deliberately NOT implemented:
- ffmpeg invocation
- EDL or QC artifact emission
- Form B sources / voices maps
- global voice mixing
- subtitle burn
- cache, batch, run_episodes integration
Defaults to input/script.srt and input/edit_plan.json so the canonical
project layout from CLAUDE.md works with no flags.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Extends the minimal entry point with the first ffmpeg pass. The
existing SRT parsing, plan parsing, and id validation are unchanged;
print_report still runs before extraction so you see the planned
mapping before the cutter touches the disk.
Adds two functions:
cut_clip(source, start, end, out)
-ss before -i + libx264 re-encode (frame-accurate). Keeps the
original audio via -c:a aac. Raises SystemExit with the full
command and the complete ffmpeg stderr on non-zero exit; raises
a friendly "ffmpeg not on PATH" message instead of FileNotFoundError
when the binary is missing.
extract_clips(cues, plan, source, temp_dir)
Iterates cues in id order, computes source_end - source_start,
hard-fails with the offending id on duration <= 0, prints
`id / start / end / out_path` per clip before invoking the cutter,
and writes to `<temp_dir>/clip_<id:03d>.mp4`. Filenames are keyed
by cue id (not enumerate) so a clip is traceable to its cue at
a glance even with sparse ids.
Two new CLI flags:
--source <path> defaults to input/source.mp4
--temp-dir <path> defaults to temp/ (auto-created)
Deliberately NOT done (still out of scope for the minimal scaffold):
concatenation, audio fades, sync tails, HDR tone-map, subtitle
burn, EDL artifact, QC report. Those live in helpers/srt_driven_edit.py.
Smoke-verified end-to-end:
- 3-cue happy path: 3 clip_NNN.mp4 files emitted
- bad plan (source_end < source_start on id=2): clip 1 cuts, run
aborts with `plan id=2: source_end ... <= source_start ...`
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a convention-driven episode-batch runner that complements the existing
jobs.json/jobs.csvmanifest path insrt_driven_edit. Given a root directory whose immediate subdirectories are episodes, it discovers eps that have the required file set and runs the edit pipeline on each.One command:
Each ep gets its own
final.mp4+edit/artifacts; an aggregaterun_episodes_summary.jsonlands at the root.Behavior
--continue-on-error, the first failing ep aborts the run. With it, the run completes and individual failures appear in the summary. Process exits non-zero whenever any ep failed._make_job(...)just composes aJobdataclass and hands it tosrt_driven_edit.run_job. So global voice mixing, sync tails, cache invalidation, etc. all behave identically to single-job runs.Reviewer notes
ffmpeg+ffprobeonPATH;conftest.pyskips otherwise.Test plan
pip install -e ".[dev]"python -m pytest tests/test_run_episodes.py -v(7 tests, ~12s)python -m pytest tests/ -v(35 tests including the full suite, ~55s)7 pytest cases included:
voice.wavwhen presentfinal.mp4--continue-on-errorskips ep with out-of-bounds range, finishes others--continue-on-errorvoice.wavreflected in QCaudio.mode🤖 Generated with Claude Code
Summary by cubic
Adds a convention-based batch runner that discovers episode folders and runs the SRT-driven edit pipeline for each, plus a project-root
main.pywrapper for single runs. Updatessrt_video_editor.pyto cut per-cue clips with ffmpeg instead of only validating.New Features
source.mp4,script.srt,edit_plan.json; picks up optionalvoice.wav. Skips incomplete dirs; errors only if none are valid.--continue-on-errorfinishes all and exits non-zero if any episode failed.<ep>/final.mp4and per-episodeedit/artifacts; writes<root>/run_episodes_summary.jsonwith structured failure records (index, paths, stderr tail).srt_driven_edit.Jobandrun_jobfor parity with manifest mode.main.pywrapper: auto-fills--srt input/script.srt,--plan input/edit_plan.json, and-o output/final.mp4; injects--source input/source.mp4and--voice input/voice.wavonly if files exist;--batchskips these defaults.srt_video_editor.py: Form A only; validates ids, prints SRT-to-source mapping, and now extracts per-cue clips totemp/clip_<id:03d>.mp4via ffmpeg (-ssbefore-i, H.264 video, AAC audio). Defaults--srt input/script.srt,--plan input/edit_plan.json,--source input/source.mp4,--temp-dir temp/; clear errors on bad ranges or missing ffmpeg.Dependencies
devextra withpytest>=7.Written for commit 09783f5. Summary will update on new commits. Review in cubic