fix(player): resolve iframe media src against iframe baseURI#295
Merged
miguel-heygen merged 1 commit intomainfrom Apr 16, 2026
Merged
fix(player): resolve iframe media src against iframe baseURI#295miguel-heygen merged 1 commit intomainfrom
miguel-heygen merged 1 commit intomainfrom
Conversation
_setupParentMedia scans the iframe for audio[data-start]/video[data-start] and creates parallel media elements in the host document for playback. It was reading the raw src attribute string and assigning it directly to the host-document element, which then resolved relative URLs against the studio root instead of the iframe. A composition using `<audio src="assets/narration.wav">` would silently 404 in preview while rendering fine. Resolve against iframeEl.ownerDocument.baseURI before handing the URL to _createParentMedia. Also prefer the raw src attribute on <source> over the DOM-resolved .src so both codepaths go through the same resolution.
vanceingalls
approved these changes
Apr 16, 2026
Collaborator
Author
Merge activity
|
9 tasks
miguel-heygen
added a commit
that referenced
this pull request
Apr 17, 2026
) ## Summary Fixes the double-voice issue in studio preview where narration plays twice with a drifting offset (measured 23ms → 80ms over a 28s clip). ## Root cause Two audio pipelines were playing the same source in parallel: 1. The iframe runtime played `<audio data-start>` elements via `syncRuntimeMedia` — the intended path. 2. `<hyperframes-player>` also created parent-frame `<audio>` copies on iframe load and auto-played them in response to every runtime `state` message. The existing `_muteIframeMedia` tried to silence the iframe copies via `el.volume = 0`, but `syncRuntimeMedia` re-asserts `el.volume` from `data-volume` every tick, so the mute never held. Studio seeks went through `__player.seek()`, which only updated the iframe timeline; parent copies kept their stale `currentTime` and drift compounded across seeks. Confirmed via agent-browser instrumentation on `factory-series-c-video`: - 6 `volumechange` events per play cycle (mute-fight signature) - Both copies audible at `volume=1`, offset growing 23ms → 80ms - Every seek widened the drift further PR #295 (v0.4.2) actually **made it audible** — before that, parent copies 404'd on the wrong URL and played silently. Fixing the URL exposed the latent double-playback. ## Fix Explicit single-owner audio ownership between `<hyperframes-player>` and the runtime. - **Default ownership is `runtime`**: iframe drives audible playback; parent proxies stay paused and inert. Matches every desktop / studio code path. No parent `play()`, no `volumechange` thrash. - **On `NotAllowedError`** from the runtime's `play()` attempt (autoplay-gated iframes), the runtime posts `media-autoplay-blocked` once. The player promotes to `parent` ownership: sends `set-media-output-muted: true` to the runtime, starts parent proxies, mirrors `currentTime` from state messages with a 150ms correction threshold. Two orthogonal mute channels replace the volume fight: | Channel | Purpose | |---|---| | `set-muted` | User's mute preference (existing, unchanged) | | `set-media-output-muted` | Internal ownership handoff (new) | `syncRuntimeMedia` now accepts `outputMuted` and asserts `el.muted = true` per active tick — sticky against sub-composition media that arrives mid-playback. Uses native `muted` (orthogonal to `volume`) so no other code path can clobber it. ## Why this shape - **Single owner, explicit transition.** No races, no tug-of-war. - **Probes reality, not device class.** We flip on an actual `NotAllowedError`, not on `matchMedia('(pointer: coarse)')` or user-agent sniffing. - **Uses `muted` instead of abusing `volume`.** `muted` is orthogonal to `volume`; `syncRuntimeMedia` doesn't write to it; author / user settings stay intact. - **Parent proxies become a thin mirror.** Under parent ownership, their `currentTime` is slaved to the iframe timeline via state messages — no independent drift. - **Backwards compatible.** Old runtimes without the new bridge action ignore the message; old players without the new message just get the previous behavior. - **Capture engine unaffected** — it bypasses both DOM pipelines and muxes audio from source files. ## Files changed - `packages/core/src/runtime/types.ts` — `set-media-output-muted` action + `media-autoplay-blocked` outbound message types. - `packages/core/src/runtime/state.ts` — `mediaOutputMuted` + `mediaAutoplayBlockedPosted` fields. - `packages/core/src/runtime/bridge.ts` — route new action to `onSetMediaOutputMuted`. - `packages/core/src/runtime/media.ts` — `outputMuted` param asserts `el.muted = true` per tick; `NotAllowedError` detection fires `onAutoplayBlocked`. - `packages/core/src/runtime/init.ts` — wire new bridge handler; coordinate with `set-muted`; post `media-autoplay-blocked` once per session. - `packages/player/src/hyperframes-player.ts` — `_audioOwner` state; delete `_muteIframeMedia`; `_promoteToParentProxy`; mirror parent `currentTime`; gate all parent play/pause/seek on ownership. ## Verified end-to-end with agent-browser on `factory-series-c-video` **Runtime ownership (default — desktop studio):** | | Before | After | |---|---|---| | `PARENT.play()` calls per play cycle | 1 | **0** | | iframe `volumechange` events | 6 | **0** | | Audible streams | 2 (drifting) | **1 (iframe)** | **Parent ownership (simulated autoplay block — direct message):** | | Value | |---|---| | iframe audio | `muted=true`, `volume=1` (untouched) | | parent audio | `muted=false`, `volume=1`, audible | | Parent ↔ iframe `currentTime` offset | ~6 ms steady state | | Offset > 150 ms | corrected by mirror sync | **Mobile path simulated with iPhone 14 emulation + injected `NotAllowedError` from iframe `<audio>.play()`:** Event timeline captured via agent-browser instrumentation: ``` t=0.0 ms IFRAME.play() called ← runtime attempts playback t=0.4 ms IFRAME.play() REJECTED: NotAllowedError ← simulated mobile gate t=0.4 ms →IFRAME bridge set-media-output-muted=true ← player promotes t=0.6 ms PARENT.play() called ← parent proxy starts t=0.8 ms ←IFRAME msg media-autoplay-blocked ← runtime signal t=1.0 ms PARENT.play() resolved ← audible t=1.3 ms IFRAME muted=true, volume=1 ← iframe silenced via native muted ``` Steady state at t=4 s under promoted parent ownership: | Element | currentTime | paused | volume | muted | |---|---|---|---|---| | Parent audio | 4.060 s | false | 1.0 | **false** (audible) | | Iframe audio | 4.068 s | false | 1.0 | **true** (silent) | **Offset: 8 ms**, single audible stream, orthogonal mute channel respected. ## Test plan - [x] `bunx vitest run` under `packages/core` — **467 / 467 pass** (incl. 4 new `media.test.ts` + 2 new `bridge.test.ts`) - [x] `bunx vitest run` under `packages/player` — **23 / 23 pass** (3 rewrites for new contract, 2 new for promotion flow) - [x] `bun run build` — all packages green - [x] Fresh preview + browser repro on `factory-series-c-video`: - [x] Runtime ownership: single audio stream, no drift - [x] Parent ownership promotion via direct `media-autoplay-blocked` message: iframe muted, parent audible - [x] iPhone 14 emulation + injected `NotAllowedError`: full promotion chain verified in ~1 s, 8 ms steady-state offset - [x] No `volumechange` thrash in either ownership mode - [x] One round of QA on a physical iOS / Android device before release — exercises real `NotAllowedError` path (expected behavior identical to simulation above)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_setupParentMediascans the iframe foraudio[data-start]/video[data-start]and creates parallel media elements in the host document (so the studio can scrub audio at sub-frame precision without iframe cross-origin restrictions). It was reading the rawsrcattribute string and assigning it directly to the host-document element, which then resolved relative URLs against the studio root instead of the iframe.Result: a composition like
```html
```
played fine in rendered MP4 output but 404'd silently in the studio preview (parent audio got
src = http://localhost:PORT/assets/narration.wavinstead ofhttp://localhost:PORT/api/projects/<name>/preview/assets/narration.wav).Fix
Resolve the src against `iframeEl.ownerDocument.baseURI` before passing it to `_createParentMedia`. Also read the raw `src` attribute on `` fallbacks so both paths go through the same resolution.
Diff is 2 lines of meaningful change (9 total once you include the comment).
Reproduction
Test plan
bun run --filter=@hyperframes/player test)_setupParentMedia)Notes
No tests added for this path because the existing harness covers only the
audio-srcattribute codepath —_setupParentMediais triggered by an internal probe interval against a live iframe, which the current fixture doesn't build. Happy to add one in a follow-up if reviewers want that coverage before merge.