test(producer): add webm-vp9 distributed regression fixture

[jrusso1020]

Description

PR 3 of 4 in the WebM distributed-rendering series. End-to-end
regression coverage for the webm distributed path PRs #950 and #951
wired up.

Adds packages/producer/tests/distributed/webm-vp9/ matching the
mp4-h264-sdr fixture pattern: a 2-second composition (60 frames @
30fps) with text, a crossfade across the frame-30 chunk seam, and a
continuous icon rotation. chunkSize: 15 produces 4 chunks so 3 seams
are tested, and the crossfade straddles the middle seam to surface
alpha-plane discontinuities introduced by alt-ref drift.

Baseline regenerated inside Dockerfile.test via
bun run --cwd packages/producer docker:test:update webm-vp9. Runs in:

• in-process mode: byte-identical match against baseline ✓
• distributed-simulated mode: PSNR 56.88–63.49 dB across 100
  checkpoints, well above the 30 dB threshold ✓

Wiring updates required to let webm flow through the harness:

• regression-harness-distributed.ts: checkDistributedSupport() no
  longer rejects webm. HDR mp4 + NTSC fps + non-{24,30,60} fps remain
  rejected. RunDistributedSimulatedInput.format widened to include
  webm.

• regression-harness-distributed.test.ts: webm-rejection test
  replaced with "accepts format=webm" test.

• regression-harness.ts: drops the now-incorrect format cast at
  the distributed-input call site.

• regression-harness-lambda-local-types.ts:
  RunLambdaLocalInput.format widened so lambda-local mode can also
  exercise webm fixtures.

• aws-lambda webm support:

  • formatExtension.ts: DistributedFormat gains "webm".
  • events.ts: RenderChunkEvent / AssembleEvent /
    PlanLambdaResult Format widened.
  • sdk/validateConfig.ts: ALLOWED_FORMATS gains "webm".
  • handler.ts: downloadChunkObjects format param widened.

Fixup commit docs(aws-lambda): drop stale webm rejection from validateConfig docblock

Review nit from Miguel: the validateConfig.ts file-header docblock
still claimed the SDK rejects webm. Updated to reflect that only
force-hdr remains an SDK-side rejection. Also scrubbed the docblock's
internal planning-doc section reference at the same time.

Testing

• bun run --cwd packages/producer docker:test webm-vp9 (in-process) — passes
• bun run --cwd packages/producer docker:test -- --mode=distributed-simulated webm-vp9 — passes (PSNR 56.88–63.49 dB)
• bun test packages/producer/src/regression-harness-distributed.test.ts — passes
• Other distributed fixtures (mp4-h264-sdr, mp4-h265-sdr, mov-prores, png-sequence) regression-simulated — no drift
• bunx tsc --noEmit on producer + aws-lambda — clean
• bunx oxlint + bunx oxfmt --check — clean

🤖 Generated with Claude Code

[jrusso1020]

• docs(lambda): document webm support + simplify-review fixes #953
• test(producer): add webm-vp9 distributed regression fixture #952 👈 (View in Graphite)
• feat(producer): enable webm in distributed mode via concat-copy #951
• feat(engine): closed-GOP VP9 encoder args + concat-copy smoke test #950
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[miguel-heygen]

Review: test(producer): add webm-vp9 distributed regression fixture

Summary

PR 3 of 4 in the WebM distributed-rendering stack. Adds an end-to-end regression fixture for VP9/webm in distributed mode, plus the type/validation wiring across aws-lambda and producer harness code to let webm flow through the distributed pipeline.

12 files changed (+338/-35): well-scoped for what it does.

What's good

• Fixture design is thoughtful. The composition intentionally exercises chunk-boundary continuity: crossfade straddling the frame-30 seam (alpha-plane test), continuous icon rotation (virtual clock regression detector), chunkSize: 15 producing 4 chunks / 3 seams. The HTML comment explaining why audio is omitted (Opus 20ms grain padding extending container duration) shows real codec-level understanding.

• Follows existing patterns exactly. Directory structure (meta.json, output/, src/), meta.json shape, GSAP timeline registration via window.__timelines, and the PSNR threshold (30 dB) all mirror the mp4-h264-sdr fixture. The webm format field and chunkSize are the only differences.

• Baseline generated via Docker (bun run --cwd packages/producer docker:test:update webm-vp9) per CLAUDE.md's requirement. output.webm is properly tracked via Git LFS.

• Type widening is consistent. "webm" added to Format unions in events.ts (3 interfaces), DistributedFormat in formatExtension.ts, downloadChunkObjects param in handler.ts, ALLOWED_FORMATS in validateConfig.ts, RunDistributedSimulatedInput.format, and RunLambdaLocalInput.format. The formatExtension switch gains the .webm case. No gaps in the type surface.

• Harness cleanup is clean. The old as "mp4" | "mov" | "png-sequence" cast in regression-harness.ts is replaced with a direct pass-through of outputFormat now that the type naturally includes webm. Comments updated throughout to remove "webm refused" language.

• Test flip is correct. regression-harness-distributed.test.ts flips the old "rejects format=webm" test to "accepts format=webm" with the right assertion.

• PSNR results are strong. 56.88-63.49 dB across 100 checkpoints, well above the 30 dB threshold. Indicates the closed-GOP concat-copy strategy produces near-identical output to single-pass.

One nit

Stale file-header comment in validateConfig.ts — The docblock at lines 13-18 still says:

plus the documented webm/force-hdr rejections from §5.3 of the distributed-rendering plan

Now that webm is allowed through ALLOWED_FORMATS, this should read something like plus the documented force-hdr rejection (or just drop the webm mention). Low priority since it's a comment, but it'll confuse anyone reading the validation module's purpose.

CI status

All green. 8 regression shards passed, preflight (lint + format) passed, no failures. Graphite mergeability check is pending (expected for a stacked PR).

Verdict

Clean, well-structured test PR. The fixture is purposeful (not just a token test), the type wiring is complete, and the baseline was generated correctly via Docker. The one stale comment is trivial.

Approve -- the stale validateConfig.ts comment can be fixed in a follow-up or folded into #953.

[miguel-heygen]

APPROVE — clean regression fixture with purposeful design.

Verified:

• 2s/60-frame composition with crossfade straddling chunk boundary + continuous rotation — designed to catch VP9 chunk-seam discontinuities
• Follows existing mp4-h264-sdr fixture patterns exactly (directory structure, meta.json, GSAP timeline)
• Baseline generated via Docker per CLAUDE.md requirements, tracked via Git LFS
• Type widening complete across all interfaces (RenderChunkEvent, AssembleEvent, PlanLambdaResult, DistributedFormat, etc.)
• formatExtension switch gains .webm case — no missing branch
• PSNR results: 56.88-63.49 dB across 100 checkpoints, well above 30 dB threshold
• Test correctly flips from "rejects webm" to "accepts webm"

Nit: file-header comment in validateConfig.ts still references "webm rejections" but webm is no longer rejected. Trivial follow-up.

All CI green.

[vanceingalls]

Additive review — @miguel-heygen already approved with coverage of fixture design, type-widening completeness, baseline LFS pinning, PSNR headroom, and the validateConfig.ts:13-18 stale-comment nit. Surfacing three gaps Magi didn't catch; none are blockers, but worth folding into the same diff before merge.

Important

• .github/workflows/regression.yml:62-77 — webm-vp9 is not in any shard's args. Sibling distributed fixtures are all wired in (mp4-h264-sdr shard-3, mp4-h265-sdr shard-4, mov-prores shard-1, png-sequence shard-6). The harness positional args at packages/producer/src/regression-harness.ts:409 are an allow-list — if (filterNames.length > 0 && !filterNames.includes(id)) return; — so the fixture lives in the tree but never runs in CI in any mode. The PR title is "add … regression fixture" but no CI regression coverage materializes; only bun run docker:test webm-vp9 (local explicit invocation) exercises it. Fix is a single-line YAML add (append webm-vp9 to shard-3's args, which already carries the matching in-process webm-transparency and mp4-h264-sdr workloads). Note that distributed-simulated mode isn't run in CI for any fixture today, so even with shard wiring the in-process-mode gate is what materializes here — but sibling-consistency still warrants the line.
• Stacked-PR CI gap. baseRefName is feat/producer-enable-webm-distributed, not main. .github/workflows/ci.yml:8-16 filters pull_request: branches: [main], so the CI workflow (Build / Lint / Test / Typecheck) didn't fire on this head — only regression, Player perf, preview-regression ran. The PR-body claims (bunx tsc --noEmit … clean, bun test … 18/18, bunx oxlint + oxfmt --check — clean) are local-only; the cross-package type widening across producer + aws-lambda would have been caught by tsc + vitest if they'd run on this head, but they didn't. The tests/distributed/_smoke/webm-concat-copy.test.ts smoke (from #951) is the real gate on the VP9 concat-copy contract, and it didn't run here either. Land #951 first so this PR's pull_request events re-evaluate against main and the gate actually fires.

Nit

• packages/producer/src/regression-harness-distributed.ts:70 doc comment opens with "The four hard gates:" but only enumerates three gates after this PR drops the webm bullet (fps.den=1, fps.num ∈ {24,30,60}, hdr). Trivial sibling to Magi's validateConfig.ts finding — same "removed webm bullet, didn't update prose around it" class.

Note for the stack (not a finding here)

• The CLI rejection at packages/cli/src/commands/lambda.ts:328 + packages/cli/src/commands/lambda/render.ts:26 and the stale user-facing doc claims at docs/deploy/migrating-to-hyperframes-lambda.mdx:45, 67-69, 81 ("Distributed mode refuses webm + HDR at plan time", "Lambda handler refuses webm with FORMAT_NOT_SUPPORTED_IN_DISTRIBUTED") are both explicitly resolved in #953. Surfacing only because the stack ships incoherent user-facing surface area in any merge order that lands #951 + #952 without #953 — keep them landing together.

Verdict: APPROVE
Reasoning: Magi covered the substantive code-review surface; the gaps I'm surfacing are CI-wiring and a parallel doc-staleness nit, all single-line fixes that don't change the fixture or the type wiring.

— Vai

[jrusso1020]

Thanks Miguel — the stale docblock nit is fixed in the follow-up commit docs(aws-lambda): drop stale webm rejection from validateConfig docblock. Cleaned up the file-header to reflect that only force-hdr remains an SDK-side rejection.

[jrusso1020]

Thanks Vai — all three folded in:

• CI shard wiring (important) — webm-vp9 added to shard-3's args (joining mp4-h264-sdr + webm-transparency per your suggestion).
• "four hard gates" → "two" (nit) — fixed in regression-harness-distributed.ts docstring; one less gate than before since the webm bullet is gone.
• Stacked-PR CI gap (process note) — noted; will resolve naturally as the stack lands.

Also surfaced a tangentially-related fallow-audit finding while committing: formatExtension tipped over the CRAP threshold when I added the case "webm" branch. Suppressed inline with the tool's suggested // fallow-ignore-next-line complexity — the function is a 5-arm switch dispatch over a closed enum where the default: never exhaustiveness check is load-bearing, so refactoring isn't the right move.

[vanceingalls]

Re-review at d8190d7e (was 5726d1ed on prior APPROVE id 4315967291).

Status of prior findings

• Shard wiring — RESOLVED. .github/workflows/regression.yml:67 shard-3 args now reads "style-7-prod style-8-prod style-10-prod css-spinner-render-compat webm-transparency mp4-h264-sdr webm-vp9". The webm-vp9 fixture is now in the harness allow-list (packages/producer/src/regression-harness.ts:409) for at least one shard's CI invocation; sibling-consistency with webm-transparency / mp4-h264-sdr in the same shard restored.
• Stacked-PR CI gap — partial / structurally unresolved at this head. baseRefName is still feat/producer-enable-webm-distributed, not main. .github/workflows/ci.yml:7-15 still has pull_request: branches: [main]. The diff does add types: [opened, synchronize, reopened, edited] with a comment explaining the rationale ("required so the workflow re-fires when a PR's base ref is set back to main after a Graphite stack restack"). That's a forward-looking fix for the restack scenario but it does not retroactively run Build / Lint / Test / Typecheck on this head while it's still stacked. Verbatim CI rollup at d8190d7e: passed = Detect changes (3 jobs), Preflight (1 of 3 ran, 2 skipped), player-perf, preview-regression, WIP; pending = 8 regression-shards in progress + Graphite mergeability; failed = none. The CI workflow (Build / Lint / Test / Typecheck) is again absent from the rollup. Land #951 first (or rebase onto main once parent merges) so the cross-package type widening + distributed-smoke gate from #951 actually fires on this head before merge.
• Nit (doc gates count) — RESOLVED. packages/producer/src/regression-harness-distributed.ts:70 now reads "Two hard gates:" with the matching two bullets (fps integer-only, hdr SDR-only) after the webm bullet drop. Sibling nit consistency with Magi's validateConfig.ts:13-18 finding addressed.

Verdict: APPROVE (re-affirming).
Reasoning: Two of three findings closed cleanly. The stacked-CI gap is structurally unresolved at this head but the edited trigger addition shows intent to handle the broader restack pattern; merge discipline (land #951 first, then rebase) is the real closure here, not a code change to this PR. No new blockers surfaced in the rebase delta.

Review by Vai (re-review)

[jrusso1020]

(Self-correction on the earlier reply: I initially used // fallow-ignore-next-line complexity to silence the formatExtension finding — fair point that fix > suppress. Refactored the switch dispatch to a Record<DistributedFormat, string> lookup table instead. Same exhaustiveness guarantee (TS errors if a new format is added to the union without a matching key), cyclomatic 5 → 1, no inline suppression. Fallow now reports zero new complexity findings.)

[vanceingalls]

Round-3 re-review at HEAD ced24546.

Delta since round 2 (d8190d7):

• aws-lambda/src/sdk/validateConfig.ts — docblock now reflects that only force-hdr is SDK-rejected. Miguel's nit.
• aws-lambda/src/formatExtension.ts — switch→Record<DistributedFormat, string> refactor. Exhaustiveness preserved at compile time via the closed mapped-type annotation: adding a new DistributedFormat variant without a matching key fails to typecheck. noUncheckedIndexedAccess doesn't degrade this — Record<K_literal_union, V> indexes as V, not V | undefined. The dropped runtime never arm was guarding against type-bypassed callers, fair to push that concern back to the boundary.

Round-2 fixes verified at new HEAD:

• .github/workflows/regression.yml:67 — webm-vp9 present in shard-3 args. ✓
• regression-harness-distributed.ts:70 — "Two hard gates" reads correctly. ✓

No new regressions. Stacked-PR CI gap still structural (closes when the stack lands; not blocking).

LGTM, ship it.

Review by Vai (re-review round 3)

[vanceingalls]

Re-stamp at 6f6dc307 — restacked. Prior approve (ced2454) still applies content-wise; this is the post-restack head. All round-2/3 findings remain ADDRESSED (shard-3 wiring, gates docstring, formatExtension Record refactor).

Re-review by Vai (post-restack re-stamp)

The base branch was changed.

[miguel-heygen]

Re-approving at 6f6dc30 — no content changes since last APPROVE at ced2454, only base ref updated after #951 merged.

[vanceingalls]

Re-stamp at 6f6dc307 after base-ref auto-restack to main. No content changes since ced24546; the round-3 review's verdict + findings still apply (shard-3 wiring + Two hard gates docstring + formatExtension Record refactor all in place).

Re-stamp by Vai (post-base-restack)