Skip to content

feat(distributed): add optional cfr flag for exact constant frame rate output#1054

Merged
jrusso1020 merged 2 commits into
mainfrom
feat/distributed-optional-cfr
May 24, 2026
Merged

feat(distributed): add optional cfr flag for exact constant frame rate output#1054
jrusso1020 merged 2 commits into
mainfrom
feat/distributed-optional-cfr

Conversation

@jrusso1020
Copy link
Copy Markdown
Collaborator

What

Adds an optional cfr boolean to the render config. When true, the assemble step re-encodes with -fps_mode cfr instead of -c copy, producing output with exact avg_frame_rate + exact duration. Default false preserves current behavior.

Why

The v0.6.39 fix (#1040) tightened r_frame_rate (container metadata) at the concat-demuxer-copy boundary, which is the achievable bar within stream-copy concat. Stream-level avg_frame_rate (PTS-derived) and sub-ms duration drift are inherent to -c copy-chunk-stitching and can't be eliminated without re-encoding.

For most consumers — browser playback, video-sharing platforms, human-facing pipelines — the difference between r_frame_rate=30/1, avg_frame_rate=27648000/921677 and r_frame_rate=30/1, avg_frame_rate=30/1 is invisible. For consumers that strict-check avg_frame_rate (broadcast workflows, frame-accurate compositors, some third-party transcoders), it matters.

This PR makes the trade-off opt-in:

  • Default (cfr: false): current behavior. Fast stitch (~1-3s for 60s 1080p), -c:v copy retention, PTS-level drift accepted.
  • Opt-in (cfr: true): re-encode at the assemble stage. ~2-5x the stitch time (~6-18s for 60s 1080p), exact CFR output, second-generation H.264 quality loss negligible at -crf 18 but non-zero.

How

  • New cfr?: boolean field on DistributedRenderConfig (producer) and AssembleEvent (aws-lambda).
  • assemble() accepts a new options.cfr flag.
  • When cfr: true, after the concat / single-chunk step, an additional ffmpeg pass re-encodes with -c:v libx264 -preset medium -crf 18 -pix_fmt yuv420p -fps_mode cfr -r <fps> before audio mux and faststart.
  • mp4 only — webm / mov stream-copy paths already produce exact avg_frame_rate, so cfr on those formats throws a typed error rather than silently re-encoding.
  • Lambda handler reads event.Cfr === true and forwards to the primitive.

Test plan

  • Added test asserting cfr: true output has avg_frame_rate === r_frame_rate exact (and equals the requested fps), and duration matches frames / fps within 1ms.
  • Added test asserting cfr: true on non-mp4 formats throws a clear error.
  • Existing cfr: false / unset tests still pass.
  • aws-lambda handler tests pass (the assemble primitive accepts the new options arg via positional-extras and existing mocks still satisfy).
  • Lint + typecheck + format clean.

Notes

This PR pairs with #1053 (single-chunk pass-through -r <fps> fix), which ensures r_frame_rate is exact even on 1-chunk renders. With both PRs landed, the achievable exactness across chunk counts + opt-in modes is:

Mode r_frame_rate avg_frame_rate Duration
Default (cfr: false) — multi-chunk exact (via #1040) PTS-derived (drift) drift ~5ms/60s
Default (cfr: false) — 1-chunk exact (via #1053) PTS-derived (drift) drift ~5ms/60s
Opt-in (cfr: true) — any chunk count exact exact (matches r_frame_rate) exact

Distributed-render output today uses -c:v copy through concat → mux →
faststart, which means PTS timestamps from each chunk pass through
unchanged. Container r_frame_rate is exact (#1040 + this PR's parent),
but stream-level avg_frame_rate stays PTS-derived and can land on
fractional rationals like 27648000/921677 over a 60s render. Same for
sub-ms duration drift.

This is the achievable bar within -c copy stream-copy concat. For most
consumers (browser playback, YouTube, etc.) the difference is invisible.
For downstream tools that strict-check avg_frame_rate or
ms-precision duration (broadcast workflows, frame-accurate compositors,
some third-party transcoders), it matters.

Adds an opt-in cfr config flag (default false). When true, the
assemble step's final pass re-encodes with -fps_mode cfr -r <fps>
instead of -c copy, producing exact CFR output. Trade-off: ~2-5x the
stitch time for a 60s 1080p clip; second-generation H.264 quality loss
is negligible at -crf 18 but is non-zero.
Copy link
Copy Markdown
Collaborator

@miguel-heygen miguel-heygen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Magi

Well-designed opt-in feature — flag threading, format guard, test coverage, and default preservation are all solid. Two issues worth addressing before merge:

Blocking: cfr re-encode hardcodes libx264, silently transcodes h265 to h264

The cfr re-encode step hardcodes -c:v libx264. DistributedRenderConfig supports codec: "h265" which encodes chunks with libx265. When a user configures { format: "mp4", codec: "h265", cfr: true }, the chunks are h265 but the cfr step transcodes them to h264 with no warning.

Simplest fix — throw a typed error when cfr + h265, similar to the webm/mov guard:

const encoderJson = JSON.parse(readFileSync(join(planDir, "meta", "encoder.json"), "utf-8"));
if (options?.cfr && encoderJson.encoder === "libx265-software") {
  throw new Error(
    `[assemble] cfr=true with libx265 is not yet supported. Use codec="h264" or cfr=false.`
  );
}

Non-blocking: missing -an flag in cfr re-encode

The cfr args specify -c:v libx264 but no explicit -an. The concat output is currently video-only so this is safe, but if a future change introduced audio in the intermediate, ffmpeg would try default audio encoding. Adding -an makes the video-only intent explicit.

Everything else looks good

  • -crf 18 is appropriate for a second-gen re-encode (< 0.5 dB PSNR loss, below perceptual threshold)
  • -preset medium is the right balance for a one-shot stitch step
  • -fps_mode cfr + -r <fps> is the correct flag combo for exact constant frame timing
  • mp4-only guard is correct and well-placed
  • planHash exclusion is correct — chunks render identically regardless of cfr
  • Tests assert both r_frame_rate and avg_frame_rate === "30/1" with ms-precision duration

Magi

The cfr re-encode pass hardcodes `-c:v libx264`. Pairing it with
`codec: "h265"` would silently transcode the h265 chunks to h264.
Detect the encoder discriminant in `meta/encoder.json` and throw a
typed error parallel to the existing non-mp4 format guard, so callers
surface the conflict instead of producing a wrong-codec deliverable.

— Rames Jusso
@jrusso1020
Copy link
Copy Markdown
Collaborator Author

@miguel-heygen — addressed the blocking item in 270b891.

assemble() now reads meta/encoder.json when cfr=true and throws a typed error if the encoder discriminant is libx265-software, parallel to the existing non-mp4 format guard. Error message points the caller at either disabling cfr or rendering with codec: "h264". Added cfr:true rejects h265 chunks with a clear error in assemble.test.ts next to the existing reject-on-non-mp4 test; full assemble suite is 9/9 green locally (lint + format + typecheck clean).

Leaving the non-blocking -an suggestion for a separate follow-up so this commit stays scoped to the blocking fix.

— Rames Jusso

miguel-heygen
miguel-heygen previously approved these changes May 24, 2026
Copy link
Copy Markdown
Collaborator

@miguel-heygen miguel-heygen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review: Magi

Both issues from my first review are addressed:

  1. h265 guard (was blocking) — Fixed. assemble() now reads meta/encoder.json and throws a typed error when cfr=true + libx265-software, preventing silent h264 transcode. The error message is clear and actionable. Test pins it with a plan staged as libx265-software. buildPlanDir helper extended with encoder param — clean.

  2. -an flag (was non-blocking) — Not added, which is fine. The concat intermediate is video-only by construction.

Everything else from the first review still holds: crf 18, preset medium, fps_mode cfr, format guard, planHash exclusion, threading — all solid.

LGTM ✓

Magi

Base automatically changed from fix/distributed-single-chunk-fps-flag to main May 24, 2026 03:50
@jrusso1020 jrusso1020 dismissed miguel-heygen’s stale review May 24, 2026 03:50

The base branch was changed.

Copy link
Copy Markdown
Collaborator

@miguel-heygen miguel-heygen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving after rebase — same verdict as my re-review. h265 guard, format guard, cfr threading, tests all look good. LGTM.

Magi

@jrusso1020 jrusso1020 merged commit 90bf485 into main May 24, 2026
38 checks passed
@jrusso1020 jrusso1020 deleted the feat/distributed-optional-cfr branch May 24, 2026 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants