Skip to content

0.12.0

Choose a tag to compare

@kajisho5 kajisho5 released this 07 Sep 13:55
· 19 commits to main since this release
81fd69e

A hardening-focused release: no new tools, no new features. Everything here closes a gap between
what the contract/docs/evals claimed and what the implementation actually did, or fixes a real
runtime defect found by reproducing it first. contract_version is unchanged — every contract
field addition here is additive.

  • Safety: a tool could be made to overwrite its own input via a same-file-different-string
    output path.
    -o ./same.mp4 against an input opened as same.mp4 (or any relative/absolute
    pair, .. segment, or symlink) resolves to the same file but passed ffmpeg's own
    byte-identical-string "Output same as Input" guard — -y then silently clobbered the source
    mid-encode. Reproduced on crop.py before the fix. _check_no_overwrite_input() in
    _common.py's run() compares os.path.realpath() of every -i argument against the output
    path and refuses before ffmpeg starts, covering every writing tool from one choke point.

  • Safety: a failed run() call could leave a partial (often 0-byte) output file behind.
    verify_output()'s cleanup only ran on the success path; a failure after ffmpeg had already
    opened the output (muxer header written, then a mid-stream error) left a stray file a caller
    could mistake for a real artifact. _cleanup_partial_output() now runs for every nonzero
    ffmpeg exit, check=True or check=False.

  • color.py --retag's re-encode fallback used to silently drop every stream beyond
    video+audio-0.
    The stream-copy path (-map 0 -c copy) keeps every stream — extra audio
    tracks, subtitles, chapters; the re-encode fallback (triggered when the copy fails) dropped all
    of them with no signal in --json. Added a middle tier that tries to keep subtitle/data
    streams via -c:s/-c:d copy alongside the required video/audio re-encode, and --json now
    reports reencoded/dropped_non_av_streams honestly instead of a bare "completed".

  • sync.py's REENCODE_META claimed video="never"; --trim-second actually re-encodes
    video
    whenever the second recording starts later than the reference (the common case,
    offset>=0) or --fix-drift is used — only the offset<0 stream-copy path leaves video
    untouched. Fixed to "conditional"/"conditional" with a note.

  • Error taxonomy (additive): error.code and error.retryable now sit alongside every
    failure's existing error.kind/error.messagecode is a static relabelling of the same 4
    kinds this codebase has always used (INPUT_INVALID/DEPENDENCY_MISSING/
    FFMPEG_EXECUTION_FAILED/OUTPUT_INVALID, INTERNAL_ERROR fallback), not a new taxonomy the
    code can't back up; retryable is currently always false (no kind is distinguishable from a
    deterministic failure without exit-code/stderr sniffing this codebase doesn't do). kind's
    existing values and the rest of the JSON shape are unchanged.

  • loudness.py --json now includes the second-pass (post-normalization) measurement as a
    result field — it was computed but only ever printed to stderr, so a caller had to make a
    separate --measure-only call to learn what loudness was actually achieved.

  • --audio-stream N extended to overlay.py, graphics.py, color.py; fit.py gained
    explicit audio mapping.
    Every tool that re-encodes audio from a multi-track input now behaves
    consistently instead of silently defaulting to track 0 (or, for fit.py, to ffmpeg's own
    implicit "best stream" heuristic, which for audio favours channel count over track order).
    join.py/multicam.py are out of scope — they combine separate input files, a different
    problem shape. Closes #62 (the same gap
    caption.py/audio.py already closed in #55).

  • doctor --json gains a fonts field, informational like gpu_encoders: drawtext's default
    font (caption.py --animate/--karaoke, graphics.py) can silently substitute a different
    family when the requested one isn't installed — a drawtext exit code can't detect this
    (fontconfig substitutes for any name, valid or not), so fc-match is queried directly.
    Never gates ok/usable; a substituted font doesn't make the tool unusable, just possibly
    styled differently than intended.

  • probe.py gains subtitle_stream_details, a detailed per-subtitle-stream array
    (index/codec/language/title) mirroring audio_streams' shape — subtitle_streams'
    existing int-count type and meaning are unchanged.

  • Doc-vs-implementation drift fixes, each with a regression test pinning the doc text against
    the live code so the same drift can't recur silently:

    • docs/contract.md's hand-copied skill.version example had drifted to a stale 0.9.1
      while package.json had moved to 0.11.0.
    • docs/contract.md's failure-JSON example was missing the code/retryable fields above.
    • SKILL.md/references/scripts.md claimed unconditionally that every script's --dry-run
      runs nothing; sync/multicam/scenes/report genuinely run ffmpeg/ffprobe to measure or
      analyse under --dry-run (they just don't write the final artifact), and verify accepts
      the flag but ignores it — all three doc locations now name the real exception set.
    • SKILL.md's Workflow section never mentioned doctor/contract at all, so an agent on an
      unfamiliar machine had no documented step to check capability before running a tool that
      depends on an optional filter/encoder. Added a step 0.
    • SKILL.md's "Look at the picture" step told the agent to judge subject framing and
      text-over-faces as part of its own job, directly contradicting "What this skill does and does
      not decide"'s statement that this belongs to the calling agent. Split into a mechanical tier
      this skill verifies directly and a judgement tier reported to the calling agent — explicitly
      not flagging fit.py --fit pad's letterboxing as a defect, since that's that mode's correct
      output. Also formalized Look: PATH (pixels not inspected; agent has no image view) for an
      execution environment that can't actually view images.
    • bin/install.js's contract/doctor subcommand hardcoded python3; Windows Python
      installers commonly expose python/py instead (only the Microsoft Store package ships
      python3) — now falls back through python3pythonpy on Windows. The same
      hardcoding was also present, unfixed by that change, in mcp/server.py's and README's MCP
      client config examples — both now note the Windows alternative.
    • evals/agent_prompts_exec.json's f05-unsupported claimed "no reverse tool in the skill"
      and scored refusing a reverse request as correct — reverse.py has existed the whole time
      and SKILL.md's own routing table names it. It also endorsed a hand-written raw-ffmpeg
      fallback as acceptable, contradicting this project's own "never fall back to raw ffmpeg"
      policy stated elsewhere. Replaced with a genuinely unsupported case and a normal reverse.py
      success case; evals/results/exec-1.json's historical record is left unedited with a note
      explaining the old grading was wrong.
  • Cross-platform: end-to-end non-ASCII filename coverage. Filter-graph string escaping for
    Unicode paths was already tested; nothing exercised a non-ASCII filename as the actual -i/
    output argument through subprocess argv. Added a test copying a fixture to a CJK/accented
    filename and running probe/cut against it both directions — passes on all 3 CI platforms.

  • Docs: CONTRIBUTING.md, a .github/workflows/release.yml that automates GitHub Release
    creation once a version tag is pushed (tag creation itself stays a manual, deliberate act), and
    a one-line honest note that GPU-accelerated encoding stays off the roadmap without a
    real-hardware-verified design (build-presence detection, which gpu_encoders already limits
    itself to, is not proof a job succeeds).

  • --dry-run's probe stub no longer fabricates plausible-looking 1920x1080/30fps dimensions
    for a not-yet-written output.
    A first attempt at this (reporting the honest 0/0.0 "not
    measured" value instead, matching duration/size_bytes's existing convention in the same
    stub) had to be reverted mid-pass: join.py and fit.py both divide by a probed source
    width/height when computing the other dimension from an aspect ratio, and dry-run probes chain
    across multi-stage pipelines (a prior stage's still-unwritten dry-run output gets probed as the
    next stage's input), so a zero source dimension reached those divisions and crashed with
    ZeroDivisionError. Root-cause fixed instead: both division sites now treat a zero/unknown
    source dimension as "can't compute a ratio" and fall back to the requested dimension rather than
    dividing by it; every other tool touching probed width/height for aspect-ratio math was audited
    and either doesn't divide by it or hands it straight to an ffmpeg filter (moot under --dry-run,
    since ffmpeg never runs). --json was never affected by any of this — it always omitted the
    placeholder; only a dry-run's human-readable summary line could echo the fake number. Closes
    #77.

  • probe.py: subtitle_stream_details. subtitle_streams was a plain integer count while audio_streams was already a detailed array, so nothing could tell which subtitle index was which language on a multi-track input (e.g. an MKV with Japanese and English subs already muxed in). subtitle_stream_details adds that detail as a new, purely additive array — [{"index", "codec", "language", "title"}, ...], one entry per embedded subtitle stream in file order (index n is -map 0:s:n), mirroring audio_streams' shape minus the audio-only fields (channels, layout, sample rate) ffprobe doesn't expose for subtitle streams. subtitle_streams' existing type and meaning (the int count) are unchanged. No writing tool selects among existing embedded subtitle streams yet; this is a probe.py-only enrichment. Closes #63.

  • caption.py --audio-stream N: explicit multi-audio-track selection. Confirmed caption.py did silently pick a track on a multi-audio-track input (dubbed languages, M&E stems): burn mode had no -map at all (ffmpeg's own automatic stream-selection heuristic, not necessarily index 0, decided), mux mode and the karaoke energy-timing/--transcribe audio extraction both hardcoded 0:a:0. --audio-stream N (default 0, matching audio.py's existing flag and unchanged prior behaviour) now threads the same explicit track index through all four: burn's re-encoded audio, mux's stream-copied audio, --transcribe's speech-to-text source, and karaoke's energy-timing analysis, refusing an out-of-range index the same way audio.py --audio-stream already does. Closes #55.

  • caption.py --text: SMPTE non-drop-frame timecode cues. Cues were positioned by decimal seconds only; broadcast-style deliverables often supply cue timing as hh:mm:ss:ff frame timecode instead. --text cue lines now also accept that format (e.g. 00:00:03:15 --> 00:00:06:00 ...) — the frame count is converted to seconds with --fps, or the input video's own probed fps when --input is given and --fps is not. A cue that is shaped like a timecode but has no fps available (no --fps, no --input) is refused with a clear error naming the missing --fps, rather than silently misread as a plain text line the way an ordinary unparseable cue line already is. _common.py gains the reusable pieces other tools can build on later: parse_time() takes an optional fps argument for the hh:mm:ss:ff case (raising the new MissingFpsError, a ValueError subclass, when fps is needed but absent), and fmt_smpte_time() formats seconds back to hh:mm:ss:ff — used here to echo the interpreted cue range in the wrote ... .srt report line so a caller can confirm the timecode was read correctly. The written .srt itself stays decimal-millisecond SRT timing, since that is the only timing SRT/mux subtitle codecs actually carry; nothing claims frame-exact precision it can't hold. Drop-frame (29.97/59.94 fps) counting is out of scope. Closes #54.

  • references/process-pitfalls.md: development-process mistakes already made once. Distinct from references/ci-platform-pitfalls.md (ffmpeg/CI behaviour differences): this is about the process of making a change safely, not FFmpeg itself. Three entries to start: narrowing a TOOL_META[...]["required"] capability list without first grepping tests/test_contract.py's DoctorDetectionTests for the fixture-pinned usable outcome it protects (nearly broken twice, in #51 and #52, caught before landing both times); retrying a git tag push or GitHub Release creation in this environment, where both are scoped out (branch pushes work, tag pushes 403 at the git-receive-pack level; no create_release/create_tag MCP tool exists; the outbound proxy itself blocks a raw REST API call to the releases endpoint) rather than accepting it after one confirmation; and redesigning a test fixture a third time instead of recognising, after two independently-redesigned fixtures failed differently on the same platform, that the platform's real behaviour (not the fixture) is the actual cause (stabilize.py's macOS libvidstab test). A living document — add to it whenever one of these recurs. Docs-only; no behaviour changed.

  • doctor --json: gpu_encoders. No tool here uses GPU-accelerated encoding — every tool assumes CPU x264/x265 — but doctor had no way to answer "is GPU encoding available on this machine" at all, unlike every other capability it already reports yes/no/unknown for. gpu_encoders reports GPU-backed encoders (nvenc, videotoolbox, qsv, vaapi, amf) present in this ffmpeg build, read from the same -encoders listing doctor already parses — {"status": "parsed"|"unparsed"|"failed"|"missing", "present": [...]}. Deliberately build-presence only: proving a real GPU/driver will accept a job would need an actual encode, which doctor's introspection never runs (matching its existing 10s-timeout, listing-only philosophy). Purely informational — no tool declares or requires a GPU encoder, so this field never affects ok or any tool's usable. The human-readable doctor output gets one line naming what's present (or "none"). Closes #52.

  • caption.py --mode mux: soft subtitle stream instead of burn-in. Every caption call previously re-encoded both streams to render pixels (reencodes_video/reencodes_audio: "always") even when the caller only wanted a subtitle track added, not the picture changed. --mode mux (new; --mode burn stays the default) copies video and audio untouched (-c:v copy -c:a copy) and adds the SRT as a separate, player-toggleable subtitle stream — reencodes_video/reencodes_audio are now "conditional" with a note explaining the split. Only takes a plain SRT (--srt/--text/--transcribe), not --ass, --animate or --karaoke: styling and animation render pixels, so they have no soft-subtitle equivalent and are refused with a pointer to --mode burn. The subtitle codec is picked from the output container (mov_text for .mp4/.m4v/.mov, srt for .mkv, webvtt for .webm); an unrecognized container is refused rather than guessed at. contract --json gains three new optional capabilities (encoder:mov_text/encoder:webvtt/encoder:srt, each "when"-gated to the matching output container) so doctor can report them honestly; the existing required capabilities (encoder:libx264, encoder:aac, filter:subtitles) are unchanged, since --mode burn is still the default and doctor's usability model doesn't vary by flag. Closes #51 — timecode-aware cue timing and explicit multi-audio-track selection were split out to #54 and #55 to keep this change reviewable.

  • Test: the prose tool count in README/docs/contract.md/package.json is now checked against the real tool list. 0.11.0 was cut to fix exactly this drift (README said 28 twice and 22 once; package.json said 21) by hand, with nothing to stop it recurring. tests/test_contract.py's test_docs_tool_count_matches_the_real_tool_list scans all three for "<N> tools" wording and fails if any number doesn't match scripts/'s actual public-tool count, so the next tool added/removed without updating every mention fails CI instead of drifting silently. contract_version/skill.version were already split (0.9.0) and already documented as "additive keeps contract_version, breaking bumps it" — docs/contract.md now also repeats the tagged-version pin guidance next to that table, since capability_map makes this the most cross-repo-facing part of the contract. Closes #50.

  • Tests: overlay.py --image on an audio-less video is now covered. Every existing overlay test used a
    source with audio; investigating a downstream report of overlay.py "hanging" on audio-less input (the
    historical 0.9.x defect this tool's own -t <duration> fix, added in 0.10.0, was meant to close) found the
    fix already works — the run had just been mistaken for a hang under a too-short timeout while it was still
    transcoding a 1080p60 frame with a fade filter. No code change; test_overlay_on_audio_less_video_terminates
    closes the coverage gap so this defect class can't silently regress.

What's Changed

  • tests: cover overlay.py --image on an audio-less video by @kajisho5 in #49
  • Mechanically check the tool count wording against the real tool list by @kajisho5 in #57
  • caption.py: add --mode mux for a soft, toggleable subtitle stream by @kajisho5 in #58
  • doctor: report GPU-backed encoders present in the ffmpeg build by @kajisho5 in #59
  • Document development-process pitfalls already made once by @kajisho5 in #60
  • caption.py: SMPTE non-drop-frame timecode cues by @kajisho5 in #61
  • caption.py: --audio-stream for explicit multi-audio-track selection by @kajisho5 in #64
  • fix(docs): correct stale skill.version example in docs/contract.md by @kajisho5 in #65
  • probe.py: add subtitle_stream_details, mirroring audio_streams by @kajisho5 in #68
  • Hardening Phase 2: input-clobber safety, sync reencode fix, additive error taxonomy by @kajisho5 in #69
  • test: exercise non-ASCII filenames end-to-end through probe/cut argv by @kajisho5 in #70
  • doctor: add font-availability detection for drawtext defaults by @kajisho5 in #71
  • overlay.py/graphics.py/color.py: --audio-stream for multi-track consistency; fit.py: explicit audio mapping by @kajisho5 in #72
  • Final defect audit fixes: stream preservation, partial-output cleanup, JSON completeness by @kajisho5 in #78
  • evals: fix f05-unsupported teaching agents to refuse a supported reverse.py request by @kajisho5 in #83
  • SKILL.md: fix dry-run overclaim, add doctor/contract to workflow, split Look into mechanical/judgement by @kajisho5 in #84
  • docs: CONTRIBUTING.md, automated GitHub Releases from tags, honest GPU roadmap note by @kajisho5 in #85
  • release: 0.12.0 — hardening pass by @kajisho5 in #86
  • SKILL.md: complete the dry-run exception list in step 3, fix stale retag description by @kajisho5 in #87
  • Fix join.py/fit.py aspect-ratio division by zero, restore honest dry-run dims by @kajisho5 in #88
  • CHANGELOG: fold #77's dry-run dims fix into the unreleased 0.12.0 entry by @kajisho5 in #89
  • Catch CHANGELOG drift automatically; document the pitfall by @kajisho5 in #90

Full Changelog: v0.11.0...v0.12.0