fix(runtime): tighten audio drift correction to ~80ms for narration#670
Closed
miguel-heygen wants to merge 5 commits intomainfrom
Closed
fix(runtime): tighten audio drift correction to ~80ms for narration#670miguel-heygen wants to merge 5 commits intomainfrom
miguel-heygen wants to merge 5 commits intomainfrom
Conversation
showcase project for the tech article — 17 compositions individually renderable via the HyperFrames producer. 3 flagship HTML-in-Canvas effects (liquid glass, device mockup, shatter) plus 14 GLSL shader transitions organized by category. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
four new compositions for the Three.js shaders article: - vfx-cinematic-camera: dolly zoom, crane shot, orbit, rack focus applied to HTML content in a Three.js scene (16s) - vfx-god-rays: screen-space radial blur god rays through occluding pillars with dust particles and camera orbit (8s) - vfx-particle-field: 8000 GPU particles morphing between text, sphere, torus, and noise states with additive blending (10s) - vfx-ray-march: pure GLSL raymarching through morphing SDF tunnel with fractal folds and octahedron pillars (8s) all render on SwiftShader (no GPU required) at 45-167ms/frame. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
replaces the flat HTML plane with the KhronosGroup DamagedHelmet GLTF model (CC-BY 4.0, 3.6MB). four camera moves orbit, dolly, crane, and rack-focus around the PBR helmet with dramatic three-point lighting. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…se cases Adds a strict sync tier to the runtime-owned media path that catches accumulated drift from repeated pause/play toggling. Previously, drift below 0.5s persisted indefinitely (only the 3s catastrophic valve would correct it). For speech-driven videos where narration, captions, and visual timing need to stay aligned, this caused audible desync. The fix adds three mechanisms: 1. **Strict sync (80ms, consecutive-sample gated)**: when the timeline- to-media offset has stabilized (not growing from initial buffering) and drift exceeds 80ms for 2 consecutive ticks, force a currentTime correction. This mirrors the parent-proxy path's 50ms threshold with consecutive-sample gating (MIRROR_DRIFT_THRESHOLD_SECONDS). 2. **forceSync on transitions**: play/pause toggles, seek, and playback- rate changes set a one-shot flag that forces an immediate sync on the next tick, bypassing all thresholds. This prevents drift from accumulating across pause/play cycles. 3. **Buffering guard**: strict sync only fires when the offset has stabilized (delta < 4ms/tick). During initial buffering, the offset grows ~16ms/tick as the timeline advances while media stays at 0 — this gradual growth is not corrected, preserving the existing behavior of not skipping opening audio. Closes #668 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
forceSyncflagCloses #668
What changed
The runtime-owned media sync (
syncRuntimeMedia) previously only corrected drift above 0.5s (and only on specific events). Stable drift between 80ms–500ms — common after repeated pause/play — persisted indefinitely. For narration-driven videos this caused audible A/V desync.The fix adds three mechanisms:
MIRROR_DRIFT_THRESHOLD_SECONDSstrategy)mediaForceSyncNextTickwhich bypasses all drift thresholds on the next tickTest plan
forceSync corrects any drift above 20ms immediately🤖 Generated with Claude Code