Multi-overlay support for XR_EXT_spatial_workspace (overlay IDs)

[dfattal]

Problem

XrWorkspaceOverlayInfoEXT carries no overlay id, so the runtime keeps exactly one overlay state per session (comp_d3d11_service.cpp — overlay_visible / overlay_xsc / overlay_anchor / pivot / size), overwritten on every xrSetWorkspaceOverlayEXT. The shell's taskbar and launcher each submit an overlay and only coexist because they're never visible at the same time. The new shell toast subsystem (DisplayXR/displayxr-shell-pvt#42) makes this a hard limit — it currently needs a shell-side priority arbiter that hides the taskbar while a toast is shown.

Proposal

Add an overlayId (uint32) to XrWorkspaceOverlayInfoEXT and to the overlay swapchain create info; the runtime keeps a small keyed map of overlays and composites all visible ones (z by id or submission order); bump spec_version. visible=false for an id removes it. The header auto-syncs to displayxr-extensions via CI; the shell bumps its runtime pin.

Benefit

Taskbar + toast + launcher composite simultaneously; retires the implicit mutual-exclusion contract; lets the shell drop the arbiter.

Affected

• Extension header (XR_EXT_spatial_workspace), spec_version bump.
• comp_d3d11_service.cpp — overlay state (single fields → keyed map) + render path (composite all visible overlays).
• Shell shell_taskbar.cpp / shell_launcher.cpp / shell_toast.cpp — one overlay id per overlay.
• Coordinated service+client redeploy required (git-tag match).

Refs DisplayXR/displayxr-shell-pvt#42.

[dfattal]

Runtime side implemented — on branch feat/366-multi-overlay-ids (not yet a PR)

Commit 0a4cb63bc. Builds clean (build_windows.bat build). Leaving it on the branch for local testing before raising the PR.

What changed (runtime)

• XR_EXT_spatial_workspace.h: spec_version 20 → 21; uint32_t overlayId appended to XrWorkspaceOverlayInfoEXT (after stereoSideBySide). Not added to XrWorkspaceOverlaySwapchainCreateInfoEXT — that swapchain is role-less; the id only matters at xrSetWorkspaceOverlayEXT time.
• ipc_workspace_overlay_info grew overlay_id (no proto.json edit — the struct is passed by type and the generated marshalling is sizeof-based).
• comp_d3d11_service.cpp: single overlay_* fields → std::map<uint32_t, overlay_slot> overlays. The runtime composites every live overlay each frame in ascending-id z-order (lower id behind, higher id in front). visible=false or a XR_NULL_HANDLE swapchain erases that id (others untouched). Map capped at 16.

Semantics / z-order

• The runtime keys the map by overlayId and draws ascending → the controller picks ids to control layering. Suggested: taskbar=1, launcher=2, toast=3 (toast on top).
• visible=false now removes an id rather than caching it hidden — so re-show must re-push the overlay's params (already the documented contract: "a subsequent call re-activates").

Backward compatibility (important)

• overlayId == 0 is the default slot. A caller that zero-inits the struct and never sets the field gets exactly the pre-v21 single-overlay behavior (one slot, overwritten each call). So nothing breaks until you opt into multiple ids.
• The IPC client↔service wire growth is hard-protected by the u_git_tag handshake (ipc_client_check_git_tag rejects a mismatch); client + service ship from the same runtime tag.
• The extension-header surface is unversioned and relies on the usual coordinated rebuild — the one hazard is a new DisplayXRClient.dll reading overlayId past a stale shell's smaller struct, so the shell must rebuild against spec 21.

For the shell-side work (displayxr-shell-pvt#42)

You can build + deploy against this branch to test multi-overlay end-to-end now (no PR yet — stay on feat/366-multi-overlay-ids):

1. Build the runtime from this branch:

   scripts\build_windows.bat build
   

   then deploy _package\bin\ (DisplayXRClient.dll, displayxr-service.exe) into C:\Program Files\DisplayXR\Runtime after taskkill of service/shell.
2. Rebuild the shell against the spec-21 header (it auto-syncs to displayxr-extensions only on merge to main; until then point the shell build at this branch's src/external/openxr_includes/openxr/XR_EXT_spatial_workspace.h, or copy it in).
3. Assign distinct overlayIds in shell_taskbar.cpp / shell_launcher.cpp / shell_toast.cpp and drop the priority arbiter that hid the taskbar behind a toast.
4. Verify taskbar + launcher + toast composite simultaneously (screenshot trigger: touch %TEMP%\workspace_screenshot_trigger → %TEMP%\workspace_screenshot.png).

Pending (not done here)

• Runtime PR (after local validation).
• Coupled merge order: merge the runtime PR first → wait for publish-extensions auto-sync → rebuild/merge the shell PR.
• Good occasion to exercise ABI-drift detection across the bundle installer / orchestrator version-gate (stale-shell vs spec-21-runtime is the case to catch).

Leaving this issue open until the shell side lands and the bundle is validated.

[dfattal]

Shell side implemented on displayxr-shell-pvt branch feat/42-voice-toasts (commit 69abd2f), validated on a Leia SR display against this runtime branch — taskbar + toast composite simultaneously.

Approach (two independent gates so the shell survives version skew):

• Build header: #if XR_EXT_spatial_workspace_SPEC_VERSION >= 21 (SHELL_HAVE_MULTI_OVERLAY) — the multi-overlay path compiles out against the current spec-20 displayxr-extensions, so the shell branch builds before this lands (single-slot only). No CI block.
• Live runtime: the shell reads the runtime-reported XR_EXT_spatial_workspace spec_version at init and only activates multi-overlay at >= 21; older runtimes fall back to the single-slot priority arbiter (no clobber regression).

Overlay ids: taskbar=1, launcher=2, toast=3 (matches your suggested z-order). visible=false/NULL-swapchain erase per id; startup+shutdown clear ids 0–3.

Merge order confirmed on our side: merge this runtime PR → wait for publish-extensions auto-sync to bump displayxr-extensions to spec 21 → then the shell PR builds with multi-overlay active in CI.

[dfattal]

Handoff: shell PR is staged — runtime side drives the coupled merge

Shell PR (draft): DisplayXR/displayxr-shell-pvt#45 (feat/42-voice-toasts). It's turnkey — built + validated against this runtime branch, and #if-guarded so it builds against the current spec-20 displayxr-extensions too (multi-overlay just stays dormant until the header syncs).

Sequence for whoever drives from the runtime side:

1. Raise + merge the Multi-overlay support for XR_EXT_spatial_workspace (overlay IDs) #366 runtime PR from feat/366-multi-overlay-ids.
2. Wait for publish-extensions to auto-sync displayxr-extensions → XR_EXT_spatial_workspace_SPEC_VERSION 21.
3. On shell PR Standardize display processor interface across graphics APIs #45: re-run CI (now builds multi-overlay active), flip draft → ready, merge.
4. (Per the issue's pending item) exercise ABI-drift detection in the bundle installer / orchestrator version-gate — stale-shell vs spec-21-runtime is the case to catch.

No shell-side action is blocking; #45 can sit as draft until step 2 completes.

[dfattal]

✅ Coupled merge + full ship complete

Multi-overlay landed end-to-end across the whole stack.

Step	Result
Runtime PR #367 → main	merged 86c6e14e9 (Windows + macOS CI green)
publish-extensions auto-sync	displayxr-extensions now XR_EXT_spatial_workspace_SPEC_VERSION 21 (7455ce99f)
Runtime release	v1.7.0 — release (DisplayXRSetup-1.7.0.1077.exe, DisplayXR-Installer-1.7.0.pkg, test-apps zip)
Shell PR #45	CI re-run against spec-21 header (multi-overlay active, not dormant) → merged 81acf4b7c
Shell release	v1.4.0 → displayxr-shell-releases (DisplayXRShellSetup-1.4.0.10.exe)
versions.json	auto-bumped: runtime v1.7.0 + shell v1.4.0 (mirrored to displayxr-installer)
Bundle	v0.7.0 — release (Windows .exe + macOS .pkg)

ABI / version-drift validated across all three layers

1. Bundle version-gate (assert-versions-in-sync): passed on the matched set in the real publish run; negative test confirmed a stale-shell pin (shell v1.3.1 vs canonical v1.4.0) trips the diff → exit 1 → bundle refused before assembly.
2. Runtime IPC (u_git_tag handshake): client↔service version skew is hard-rejected.
3. Shell spec-version gate: compile-time #if SPEC_VERSION >= 21 + live runtime spec_version check with single-slot fallback → a stale shell against a spec-21 runtime degrades gracefully (no clobber/crash), validated on Leia hardware.

Backward compat holds: overlayId == 0 reproduces the pre-v21 single-overlay behavior, so nothing that didn't opt into ids changed.

Closing — runtime + shell + bundle all shipped and the drift detection is exercised end to end.