Skip to content

v1.8.0 — Pre-flight gate: FL2V frames, prompt lint, palette, beat alignment

Latest

Choose a tag to compare

@IvenKooLab IvenKooLab released this 06 Nov 10:07

Thanks to @YIAN6557 (X: @EianLu) — this release ports the pre-flight method from his article "Claude code/Codex+Seedance: from an idea to a finished AI video" to a local-first ComfyUI studio.

感谢 @YIAN6557 公开这套「把错误挡在花钱之前」的生产方法,本版把它本地化到 ComfyAgent(付费环节换成 GPU 时间,审核换成确定性门禁)。


Changelog

Credit: this release is based on the production method shared by Eian (@EianLu · GitHub @YIAN6557) in his X long-form piece
"Claude code/Codex+Seedance: from an idea to a finished AI video" — first/last-frame anchoring, the seven-section
video prompt, the pre-submit audit pass and the retrospective. Thank you for publishing it. Here the "paid" stage is
GPU time instead of per-second API billing, and the audit agent becomes a deterministic pre-flight gate plus an
optional LLM visual audit.

  • First/last-frame video (FL2V): new H3 FL2V workflow — MiniMaxH3ImageToVideo natively takes first_frame + last_frame (the bundled weights are FL2VA). Shots gain a last-frame slot; a shot with a last frame is auto-upgraded to FL2V when queued. submit_workflow gained per-node image targeting (__image_map__) so the two frames land in two different LoadImage nodes instead of overwriting each other
  • Pre-flight check (precheck_shot, offline): empty/short prompt, missing seven-section blocks (style / shot / composition / characters / per-second / audio / constraints), negative phrasing that models ignore, filmability lint (hands holding small props, on-screen text, mirrors, crossing people, multiple actions per shot, top-down faces), last-frame without first-frame, workflow without last_frame support, first/last frame size mismatch, and first/last SSIM too low (camera drift) or too high (no motion)
  • Gate: failing shots are marked "blocked" at queue time with the reason and can be force-submitted (timestamped) — no more burning ~8.5 min per clip on a shot with a known defect; warnings never block
  • Visual audit (the article's "open another agent before submitting"): /api/pipeline/audit sends downscaled first/last frames plus the prompt to the vision model and accepts only "pass / fail + itemized issues"; results are stored on the shot and degrade gracefully without an API key
  • Retrospective (step 8): /api/pipeline/retro writes a per-batch markdown — outcomes, GPU minutes, audit pass rate, gate hit counts, per-shot hits and next-time fixes — under data/retro/
  • Palette up front: batch-level palette (background / subject / accent / light) and a /api/pipeline/palette-sheet page that renders, per shot, a 1920×1080 colour-block sketch with 38%/62% golden-ratio guides and a side-by-side black-and-white version; a value gap below 25 is flagged by the pre-flight check ("subject and background merge in B&W") — the article's "design the palette before the model guesses" turned into a number you can test
  • Beat alignment: /api/pipeline/beat-align decodes audio with ffmpeg and estimates BPM from an onset envelope + autocorrelation (pure stdlib), then suggests per-shot durations snapped to whole bars and reports the whole-film beat drift; measure beats on the original track, and the UI says so (the article's "turns on the chorus, key actions on the beat")
  • Seven-section template: /api/pipeline/shot-template returns a paste-ready structure, including the "0–0.2s static, identical to the first frame" and "last 0.3s identical to the last frame" anchors
  • Pipeline UI: last-frame button, per-shot check/audit, batch "Check all" and "Retro"; blocked shots render as "Blocked"
  • New tests: tests/test_pipeline_gate.py (16) and tests/test_palette_beat.py (8); full suite 69 tests plus 3 integration scripts green

Full changelog: CHANGELOG.md. Requires local ComfyUI and the WebView2 runtime.


更新日志

致谢:本版方法来自 X 作者 Eian(@EianLu · GitHub @YIAN6557) 的长文《Claude code/Codex+Seedance AI视频:从一个想法到成片的全流程》——
首尾帧锚定、七段式视频提示词、提交前审核、复盘沉淀,感谢他公开分享。在本项目里,「花钱」的环节是 GPU 时间而不是按秒计费,
审核 agent 换成了确定性预检 + 可选的 LLM 视觉审核。

  • 首尾帧(FL2V)落地:新增 H3 首尾帧生视频(FL2V) 工作流——MiniMaxH3ImageToVideo 原生支持 first_frame + last_frame(随附权重本就是 FL2VA)。镜头新增尾帧槽位,设了尾帧的镜头排队时自动升级到 FL2V;submit_workflow 新增按节点定向注图 __image_map__,首尾帧分别写入两个 LoadImage,不再互相覆盖
  • 出片前预检(precheck_shot,离线可跑):提示词空/过短、七段式缺段(画风/镜头/构图/人物/逐秒/声音/约束)、「不要…」类负面表述、可拍性 lint(手拿小物/画面文字/镜子倒影/人物交叉/一镜多动作/俯拍无脸)、只设尾帧、工作流不支持 last_frame、首尾帧尺寸不一致、首尾帧 SSIM 过低(机位跑偏)或过高(等于不动)
  • 门禁拦截:fail 级镜头排队时标记「已拦截」并说明原因,可强制提交(记录时间)——避免明知有硬伤还烧 8.5 分钟/条;warn 级只提示不拦
  • 视觉审核(对应文中「提交前另开一个 agent」):/api/pipeline/audit 把首/尾帧缩略图连同提示词发给视觉模型,只接受「通过/不通过 + 逐条问题」,结果记在镜头上;无 API Key 时明确降级
  • 复盘沉淀(第 8 步):/api/pipeline/retro 生成本批 markdown——成败、GPU 时间、审核通过率、门禁命中频次、逐镜命中与改进项,落 data/retro/
  • 配色前置:批次级色板(背景/主体/强调/光)+ /api/pipeline/palette-sheet 页面:每镜出 1920×1080 色块草图、38%/62% 黄金比例参考线、彩色与黑白并排对照;亮度差 <25 由预检直接提示「黑白后主体背景分不开」——把「配色别交给模型自由发挥」变成可测的数字
  • 节拍对齐:/api/pipeline/beat-align 用 ffmpeg 解码 + 能量包络自相关估 BPM(纯 stdlib),给出每镜建议生成秒数(吸附整小节)与全片落点偏差;只从原曲测拍点,界面也会这么提示(对应「转折对副歌、关键动作对重拍」)
  • 七段式模板:/api/pipeline/shot-template 提供可直接粘贴的结构,含「0–0.2 秒静止与首帧一致」「最后 0.3 秒与尾帧一致」两端锚定
  • 管线 UI:尾帧按钮、单镜预检/审核、批次「预检全部」「复盘」,拦截态显示「已拦截」
  • 新增 tests/test_pipeline_gate.py 16 项 + tests/test_palette_beat.py 8 项;全套 69 项 + 3 个集成脚本全绿

完整变更:CHANGELOG.md。需要本地 ComfyUI 与 WebView2 运行时。