Skip to content

feat(hdr): z-ordered multi-layer compositing with PQ support#289

Open
vanceingalls wants to merge 1 commit intofeat/hdr-phase-1from
feat/hdr-phase-2
Open

feat(hdr): z-ordered multi-layer compositing with PQ support#289
vanceingalls wants to merge 1 commit intofeat/hdr-phase-1from
feat/hdr-phase-2

Conversation

@vanceingalls
Copy link
Copy Markdown
Collaborator

@vanceingalls vanceingalls commented Apr 16, 2026

Summary

Phase 1 only handled HDR video on the bottom with DOM on top. Real compositions need arbitrary z-interleaving — HDR video between DOM layers, multiple HDR videos at different z-depths, SDR video as background below HDR. This PR adds per-frame z-order analysis and layer-by-layer compositing.

What it does

Per-frame algorithm:

  1. Query stacking orderqueryElementStacking() asks Chrome for every timed element's z-index, bounds, opacity, visibility, and whether it's an HDR video.
  2. Group into layersgroupIntoLayers() walks the z-sorted list. Each time the content type switches (DOM ↔ HDR), a new layer starts. Adjacent DOM elements merge into a single layer (fewer Chrome screenshots).
  3. Composite bottom-to-top — DOM layers get Chrome alpha screenshots (with non-layer elements hidden). HDR layers get native pixel blits from pre-extracted frames. Each layer composites onto the running canvas.

Example z-order:

z=0: SDR background     → DOM layer 0 (1 screenshot)
z=1: HDR video           → HDR layer 1 (native blit)  
z=2: text overlay        → DOM layer 2 (1 screenshot)
z=3: HDR circle video    → HDR layer 3 (native blit)
z=4: logo                → DOM layer 4 (1 screenshot)

This produces 3 screenshots + 2 native blits = 5 layer operations per frame.

Also adds PQ (HDR10) support:

  • buildSrgbToHdrLut("pq") — PQ OETF (SMPTE 2084) maps SDR white to ~203 nits (~58% PQ signal)
  • blitRgba8OverRgb48le() now accepts transfer: "hlg" | "pq" parameter
  • blitRgb48leRegion() — positioned rectangular copy with bounds clipping and optional opacity

Files changed

File What changed
packages/engine/src/utils/layerCompositor.ts NEWgroupIntoLayers(), CompositeLayer type
packages/engine/src/services/videoFrameInjector.ts queryElementStacking() with effective z-index walk, ElementStackingInfo type
packages/engine/src/utils/alphaBlit.ts PQ LUT, blitRgb48leRegion(), transfer parameter
packages/producer/src/services/renderOrchestrator.ts Layer-driven compositing loop replacing simple two-pass

How to test

Render a composition with SDR video below HDR video below text overlays. All three should be visible at their correct z-depths.

Stack position

4 of 6 — Stacked on #288 (two-pass compositing). Generalizes the fixed two-layer model to arbitrary N-layer compositing.

🤖 Generated with Claude Code

Copy link
Copy Markdown
Collaborator Author

vanceingalls commented Apr 16, 2026

@vanceingalls vanceingalls marked this pull request as ready for review April 16, 2026 00:54
@vanceingalls vanceingalls changed the title docs: Phase 2 z-ordered multi-layer compositing design spec feat(hdr): z-ordered multi-layer compositing with PQ support Apr 17, 2026
Per-frame z-order analysis groups elements into DOM and HDR layers,
composited bottom-to-top. Adjacent DOM elements merge into single
screenshots. PQ (HDR10/smpte2084) support via sRGB-to-PQ LUT with
203-nit SDR reference white. queryElementStacking walks DOM for
effective z-index, groupIntoLayers splits on HDR/DOM boundaries.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant