v0.21.0 — captions look at a 1080p frame

Vision captions look at a 1080p frame, not a 4K one

Contributed by @jardelapp in #21.

caption_shots now asks render_frame for a frame with its short edge capped at 1080 px (vlm.CAPTION_SHORT_SIDE), aspect preserved: 3840x2160 becomes 1920x1080, a 2160x3840 phone clip becomes 1080x1920, and a source already smaller than the cap keeps its own size. Qwen-VL tokenizes by pixel area, so a 4K frame costs roughly 4x the vision tokens for the same sentence.

Measured by the contributor on a 2160x3840 HEVC frame with Qwen3-VL-4B on an M4 Pro: 32 s per caption at full size, 4.8 s at 1080p, same sentence out of the model. find_index caps captions at 40 per source, so a 4K B-roll clip with 40 cuts goes from ~20 minutes of indexing to ~3. Phone footage is 4K by default — this is the common case, not the edge. (Their numbers were taken on Qwen3-VL-4B rather than the shipped Qwen2-VL-2B default; the mechanism is the same on both, but the 2B was not measured under the same conditions.)

Short edge rather than long: capping the long edge would shrink a portrait 2160x3840 clip to 608x1080 — a quarter of the pixels — and portrait phone footage is exactly where faces and on-screen text are the details the captions are for.

render_frame is unchanged by default and takes the cap only as an explicit max_short_side keyword. The preview tools keep getting full-resolution frames: preview_check measures them, so a silently downscaled frame there would be the verification instrument lying. Existing cached captions are not invalidated — the index keys them by source (path, mtime, size), not frame size.

Also

frame_scale_filter bounds its cap to 1..8192. Unbounded, a zero or negative value exits ffmpeg non-zero, render_frame returns None, and caption_shots reads that as a shot with nothing to caption — a bad constant would have been indistinguishable from uncaptionable footage. graph_to_args already bounds its height for the same reason.

Verification

Both the argument list and the real binary: a mocked-ffmpeg test asserts -vf is absent unless asked for, and a real-ffmpeg test synthesizes 3840x2160, 2160x3840 and 2160x2160 sources and reads the output JPEG dimensions back with ffprobe — 1920x1080, 1080x1920, 1080x1080, and a 320x240 fixture unchanged. Mutation-checked: a filter that caps the long edge instead turns that test red.

1696 tests across 60 suites. mcp floor 1689 pass / 7 skip · mcp 2.x 1690 / 6 · FCP_MCP_INDEX=off 1665 / 31 · no ffmpeg on PATH 1668 / 28.