Skip to content

fix(render): scale timeout for long video encodes#2244

Merged
miguel-heygen merged 1 commit into
mainfrom
fix/compiler-local-script-render
Jul 11, 2026
Merged

fix(render): scale timeout for long video encodes#2244
miguel-heygen merged 1 commit into
mainfrom
fix/compiler-local-script-render

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

What

Scale the video encode timeout to at least four times the composition duration while preserving larger operator overrides.

Why

The existing 600-second total wall-clock timeout kills long disk-frame encodes that are still making progress. Two Windows reports reproduced this on 353-second and 755-second compositions; both completed after manually increasing FFMPEG_ENCODE_TIMEOUT_MS.

How

Derive a video-only engine config in the encode stage with max(configured timeout, duration × 4 seconds). GIF behavior and short-render config identity remain unchanged.

Test plan

  • bun test packages/producer/src/services/render/stages/encodeStage.test.ts --timeout 30000
  • bun run --filter @hyperframes/producer typecheck
  • pre-commit lint, format, tracked-artifact, fallow, and typecheck hooks

@jrusso1020

Copy link
Copy Markdown
Collaborator

@miguel-heygen is this for streaming encoding or no?

streaming encoding in the past has a hard deadline of 600 seconds I think at least that's what we've seen in production and therefore limit streaming encoding to smaller videos and use serial based encoding in production for longer videos to avoid the timeout

@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

@miguel-heygen is this for streaming encoding or no?

streaming encoding in the past has a hard deadline of 600 seconds I think at least that's what we've seen in production and therefore limit streaming encoding to smaller videos and use serial based encoding in production for longer videos to avoid the timeout

@jrusso1020 this type of PRs are being created by Magi based on the cli feedback

@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

@jrusso1020 This is for the separate post-capture disk-frame encode path, not streaming encode.

Streaming encoding still has its existing safeguards unchanged: shouldUseStreamingEncode(...) only selects it for single-worker MP4 renders whose composition duration is at most streamingEncodeMaxDurationSeconds (240 seconds by default). Streaming pipes frames into FFmpeg during capture and skips the standalone Stage 5 encode entirely.

Both reports behind this PR were longer than that gate (353s and 754.8s), so they correctly fell back to disk capture and then hit the separate fixed ffmpegEncodeTimeout = 600000 wall-clock cap during Stage 5. Raising FFMPEG_ENCODE_TIMEOUT_MS allowed both renders to complete, which isolates the issue this PR fixes.

So the production policy you described remains intact: long videos do not use streaming. This PR only gives their standard disk/chunked encode enough time after capture. I'll also make the PR description explicit that streaming selection and the 240s gate are unchanged.

@terencecho terencecho left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Cross-checks

  • Units consistency. job.duration (seconds; test uses 754.8) × 4_000 → ms; ffmpegEncodeTimeout is ms (600_000 = 600s baseline). Consistent. Test expectation 3_019_200 = Math.ceil(754.8 * 4000) ✓.
  • Edge cases via ?? + Math.ceil. undefined → 0 → falls back to configured; NaN → n>configured === false → falls back; negative → < configured → falls back. All safe by construction; no explicit guard needed.
  • Operator override preserved. The Math.max(configured, scaled) shape means an operator override that's already larger than the scaled value wins — matches the PR's "preserve larger operator overrides" property.
  • GIF path unaffected. encodeFramesToGif still uses engineCfg unchanged; scaling only applies to the two video branches (matches PR body claim).
  • CI green — all required checks pass including Windows render + regression shards.

Nits (non-blocking)

  • One point on the curve in the new test (long-video path); a duration=0 / undefined case that asserts falling back to configured default would be worth adding, though the existing "prefers engine config" test likely covers the short-duration default via makeInput().
  • No upper clamp: a 3600s composition yields a 14400s (4h) budget. Fine given the wall-clock is a still-progressing cap, but worth flagging if any host has its own hard limit.
  • Magic constant 4_000 is only documented in the adjacent comment. ENCODE_BUDGET_MULTIPLIER_MS_PER_SEC would age better; not worth churning.

Diff-scope: +30/-2, 2 files, matches title.

— Review by tai (pr-review)

@vanceingalls vanceingalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 🟢 LGTM at 42a3df2

Concurring with @terencecho on units consistency + edge cases (undefined → 0 → falls back, NaN → falls back, negative → falls back) + operator-override preservation via Math.max(configured, scaled). Also endorsing his scope confirmation from Miguel's comment thread: streaming encoding still gates on shouldUseStreamingEncode and only accepts compositions ≤ streamingEncodeMaxDurationSeconds, so both reproducing cases (353s and 754.8s) fell to disk capture where this new timeout applies. GIF branch at encodeStage.ts:268 still uses the raw engineCfg.ffmpegEncodeTimeout, unchanged.

One additional risk beyond Terence's coverage: no upper cap on scaledEncodeTimeout. Math.ceil((job.duration ?? 0) * 4000) — a pathological job.duration (e.g. a corrupted 100000s composition) produces a ~4.6-day timeout, holding a worker for days. Realistically bounded by upstream producer contracts + duration validation, so low real-world risk, but a Math.min(scaled, MAX_ENCODE_TIMEOUT_MS) would be belt-and-suspenders against the pathology. Follow-up worth filing.

Second observation: "4× duration" is a "longer than what we saw yesterday" number, not a p99-derived one. Fine as a CLI-unblock; a follow-up to characterize the actual encode-time-vs-duration curve would let this be tighter.

R1 by Via

@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

is this for streaming encoding or no?\n\nAnswered: no. The timeout scaling is applied only in runEncodeStage to the disk-frame video encoders (encodeFramesFromDir / encodeFramesChunkedConcat). The streaming-fusion path is gated before this stage (the stage docs note it is skipped when streaming is already handled), and the existing streamingEncodeMaxDurationSeconds/600-second streaming deadline remains unchanged. GIF and PNG-sequence paths also retain their existing timeout behavior.

@miguel-heygen miguel-heygen merged commit 9d91c2a into main Jul 11, 2026
46 checks passed
@miguel-heygen miguel-heygen deleted the fix/compiler-local-script-render branch July 11, 2026 22:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants