feat(XR): WebGPU-compatible XR session + XRGPUBinding plumbing (Phase 1, #18638)#18650
Conversation
RaananW
changed the title
…ible opt-in Phase 1 (#18638) plumbing, PR1 of 3. Adds a webxr.webgpu.d.ts companion with ambient declarations for XRGPUBinding / XRGPUSubImage / XRGPU*LayerInit grounded in the immersive-web/WebXR-WebGPU-Binding explainer + IDL, and documents the inherited GPURequestAdapterOptions.xrCompatible opt-in on WebGPUEngineOptions. No behavior change: xrCompatible already flows through _options to navigator.gpu.requestAdapter(); this only surfaces/documents it. WebGL paths untouched. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…lection Phase 1 (#18638) plumbing, PR2 of 3. Adds the @internal WebGPU implementation of IWebXRGraphicsBinding (wrapping XRGPUBinding) alongside the existing WebGL one, a WebXRGraphicsBindingType.WebGPU enum member, and branches WebXRSessionManager._getGraphicsBinding() on AbstractEngine.isWebGPU so a WebGPU engine gets the XRGPUBinding-backed binding and everything else keeps the XRWebGLBinding-backed one. Introduced-but-unused: per-frame layer/sub-image ops are deferred to later phases. Kept @internal and out of the XR barrels (mirrors webXRGraphicsBinding.ts). The GPUDevice is read from WebGPUEngine._device via a type-only import (mirrors the Phase 0 (engine as ThinEngine)._gl access), so no runtime coupling/side effects. WebGL2 XR behavior is unchanged. Adds unit coverage for the WebGL/WebGPU binding selection, caching, and the uninitialized-session guard. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…only) Phase 1 (#18638) plumbing, PR3 of 3. When the engine is a WebGPU engine: - initializeSessionAsync requests the 'webgpu' feature descriptor as a REQUIRED feature (a WebGPU engine cannot fall back to a WebGL XR session; the native requestSession rejection is left to propagate to the caller, not swallowed). - webXRExperienceHelper skips the baseLayer/XRWebGLLayer path, since a WebGPU-compatible XR session is layers-only per the WebXR/WebGPU binding spec. The WebGPU XRProjectionLayer is added in a later phase, so a WebGPU XR session currently ENTERS with no layer and renders nothing yet — the expected Phase 1 end-state, not a bug. WebGL2 XR behavior is byte-for-byte identical: the WebGL branch leaves the session init object untouched and keeps the baseLayer path. Adds unit coverage: 'webgpu' injected only for WebGPU engines, existing required features preserved without duplicates, WebGL init passed through unchanged, and updateRenderState with neither baseLayer nor layers does not throw. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
RaananW
force-pushed
the
raananw-webgpu-xr-phase1-binding-plumbing
branch
from
be37024 to
fa08272
Compare
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18650/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18650/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18650/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
🟢 Memory Leak Test Results4 passed, 0 leaked out of 4 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (4)
|
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
⚡ Performance Test Results🟢 All performance tests passed — no regressions detected. |
…n layer A layers-only WebGPU XR session attaches no layer in Phase 1, so it receives no requestAnimationFrame callbacks and WebXRState remains ENTERING_XR (never IN_XR, which is gated on the first frame) until the Phase 2 XRProjectionLayer produces the first frame. Documents the hardware-confirmed Phase 1 end-state so it is not mistaken for a regression. Comment-only, no logic change. Refs #18638 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Implements Phase 1 of the WebGPU-for-WebXR effort by adding the minimal plumbing needed for a WebGPU engine to request a WebGPU-compatible XR session (via the "webgpu" required feature) and to construct an XRGPUBinding-backed graphics binding, while preserving byte-for-byte identical behavior for WebGL/WebGL2 XR paths.
Changes:
- Added ambient TypeScript declarations for the WebXR/WebGPU Binding types (
XRGPUBinding,XRGPUSubImage, andXRGPU*LayerInitdictionaries) without modifying the communitywebxr.d.ts. - Introduced an internal
WebXRWebGPUGraphicsBindingand updated session manager binding selection based onengine.isWebGPU. - Gated XR session creation + enter flow for WebGPU (inject required
"webgpu"feature; skipbaseLayer/XRWebGLLayerpath), with unit tests covering the new branching behavior.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| packages/dev/core/test/unit/XR/webXRSessionManager.test.ts | Adds unit tests for WebGPU vs WebGL binding selection, "webgpu" required-feature injection, pass-through behavior for WebGL, and Phase 1 “no layer” render state. |
| packages/dev/core/src/XR/webXRSessionManager.ts | Selects WebGL vs WebGPU graphics binding based on isWebGPU; injects "webgpu" into requiredFeatures only for WebGPU engines. |
| packages/dev/core/src/XR/webXRGraphicsBinding.ts | Adds WebXRGraphicsBindingType.WebGPU and internal WebXRWebGPUGraphicsBinding wrapping new XRGPUBinding(session, device). |
| packages/dev/core/src/XR/webXRExperienceHelper.ts | Skips baseLayer initialization when running on a WebGPU engine (layers-only session path). |
| packages/dev/core/src/LibDeclarations/webxr.webgpu.d.ts | New companion ambient typings for the proposed WebXR/WebGPU binding API surface. |
| packages/dev/core/src/Engines/webgpuEngine.pure.ts | Documents xrCompatible on WebGPUEngineOptions for discoverability (inherited from GPURequestAdapterOptions). |
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
Adds unit coverage for the hardware-observed WebGPU Phase 1 flow where a layers-only session enters but produces no XR frame: - webXRSessionManager: the native session 'end' handler cleans up (nulls the graphics binding, clears inXRSession, notifies onXRSessionEnded) even when no frame ever arrived. - webXRExperienceHelper (new): with a WebGPU engine and no frame, state stays at ENTERING_XR; exitXRAsync() from ENTERING_XR is a clean no-op (does not touch the session manager, state unchanged); and a native session end from ENTERING_XR returns state to NOT_IN_XR with no hang or leak. Refs #18638 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18650/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18650/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18650/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
🟢 Memory Leak Test Results4 passed, 0 leaked out of 4 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (4)
|
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
⚡ Performance Test Results🟢 All performance tests passed — no regressions detected. |
Phase 1 (#18638) — WebGPU-compatible XR session + XRGPUBinding plumbing
Closes #18638 (Phase 1 of the WebGPU-for-WebXR epic #18635 — the epic stays open for Phases 2–5).
Part of the WebGPU-for-WebXR effort (epic #18635). Plumbing only — no per-frame WebGPU XR rendering (Phase 2), no camera/NDC (Phase 3), no feature rewiring (Phase 4). WebGL2 XR behavior is byte-for-byte identical; every WebGPU path is gated behind
AbstractEngine.isWebGPUand is unreachable for WebGL engines.What's in here
1. TS typings + adapter opt-in
packages/dev/core/src/LibDeclarations/webxr.webgpu.d.ts(mirrors the existingwebxr.nativeextensions.d.tssplit so the large community-maintainedwebxr.d.tsstays untouched). DeclaresXRGPUBinding,XRGPUSubImage, and theXRGPU*LayerInitdictionaries. Base XR layer/sub-image types come fromwebxr.d.ts;GPURequestAdapterOptions.xrCompatiblestays inwebgpu.d.ts(not duplicated).xrCompatibleonWebGPUEngineOptions. It is inherited fromGPURequestAdapterOptionsand already flows end-to-end (initAsyncpasses_optionsstraight tonavigator.gpu.requestAdapter()), so this is JSDoc/discoverability only — it notes the flag must be set at adapter-request/engine-construction time (WebGPU has no post-hoc "make XR compatible" step).2.
@internalWebGPU graphics binding + wiringWebXRWebGPUGraphicsBindingimplementing the Phase-0IWebXRGraphicsBindingseam, wrappingnew XRGPUBinding(session, device), plus aWebXRGraphicsBindingType.WebGPUenum member. Minimal/introduced-but-unused — per-frame layer/sub-image ops are deferred to later phases.WebXRSessionManager._getGraphicsBinding()branches onAbstractEngine.isWebGPU: WebGPU engine →XRGPUBinding-backed binding; everything else → the existingXRWebGLBinding-backed one. TheGPUDeviceis read fromWebGPUEngine._devicevia a type-only import + cast (mirrors Phase 0's(engine as ThinEngine)._gl), so there is no runtime coupling/side effect. Kept@internaland out ofindex.ts/pure.ts.3. WebGPU-compatible session gating
initializeSessionAsyncrequests the'webgpu'feature descriptor as a required feature when the engine is WebGPU (a WebGPU engine cannot fall back to a WebGL XR session; the nativerequestSessionrejection is left to propagate to the caller, not swallowed). The WebGL branch leaves theXRSessionInitobject untouched (same reference passed through — asserted in a test).webXRExperienceHelper.enterXRAsyncskips thebaseLayer/XRWebGLLayerpath for WebGPU, since a WebGPU-compatible session is layers-only.4. Doc-only clarification of the ENTERING_XR end-state (see below) — comment-only, no logic change.
✅ Hardware-validated Phase 1 end-state (not a regression)
Validated on Meta Quest Browser with the experimental
XRGPUBindingflag, against this PR's snapshot build. The precise Phase 1 end-state is:This "no layer → no frame" behavior was confirmed both through Babylon and at the raw-browser level: a bare
navigator.xrimmersive-vrsession requested withrequiredFeatures: ['webgpu']is granted'webgpu', yet arequestAnimationFramewith no layer set yields zero callbacks. So the stall is inherent to the layers-only WebGPU-XR design and the WebXR spec, not a bug in this plumbing. ReachingIN_XR/ actually rendering is Phase 2 (projection-layer RTT provider viawrapWebGPUTexture).Exit is clean from
ENTERING_XR: ending the session via its nativeendpath (e.g. the headset system menu) firesonXRSessionEndedregardless ofWebXRState, restoring the framebuffer/render loop/camera and returningWebXRStatetoNOT_IN_XRwith no hang or leak (the@internalbinding is nulled on end). Note: the programmaticexitXRAsync()is a no-op fromENTERING_XR(its pre-existingstate !== IN_XRguard, unchanged by this PR) — a no-op, not a hang; the session remains exitable via the native path.A unit test confirms
updateRenderStatewith neitherbaseLayernorlayersdoes not throw. /cc @RaananW.Spec grounding
Declarations/behavior are taken from the immersive-web/WebXR-WebGPU-Binding explainer + proposed IDL: https://github.com/immersive-web/WebXR-WebGPU-Binding/blob/main/explainer.md — including the
'webgpu'feature descriptor, the layers-only rule (nobaseLayer), and theXRGPUBinding(session, device)shape requiring anxrCompatibleadapter.Constraints honored
isWebGPU..pure.tssplit, no top-level side effects (confirmed: no side-effect-manifest churn on any commit).@internaland out of the barrels;xrCompatibleis an inherited, now-documented option.Testing
tsc -b tsconfig.devPackages.jsonexit 0.npm run format:check✅ ;npm run lint:check✅ (eslint + all 16 tree-shaking checks + side-effects-sync across core/gui/loaders/serializers).vitest run --project=unit -t "XR": 260 passed.webXRSessionManager.test.tscases cover binding selection (WebGL vs WebGPU), binding caching, the uninitialized-session guard,'webgpu'injected only for WebGPU engines (existing required features preserved, no duplicates), WebGL init passed through unchanged (same object ref),updateRenderStatewith no layer not throwing, and the native session-endhandler cleaning up (nulls the graphics binding, clearsinXRSession, notifiesonXRSessionEnded) even when no frame ever arrived. NewwebXRExperienceHelper.test.tscovers the no-frame WebGPU lifecycle: state stays atENTERING_XRwith no frame,exitXRAsync()fromENTERING_XRis a clean no-op (does not touch the session manager, state unchanged), and a native session end fromENTERING_XRreturns state toNOT_IN_XRwith no hang or leak.smartFilterBlocksvitest suite fails to import in this environment (missing generated.block.jsartifacts) — pre-existing, not touched by this PR.