[XR] WebGPU-XR Phase 0: decouple WebXR from WebGL with API-agnostic seams#18645
Merged
Conversation
…XR phase 0) Part of the WebGPU-for-WebXR decoupling (#18637, epic #18635). Behavior- preserving, WebGL-only: no functional change for existing WebGL2 XR. - Promote framebufferDimensionsObject (setter + backing field) from ThinEngine up to AbstractEngine so the XR render loop can drive any engine backend. - Retype WebXRSessionManager._engine and the WebXR RT-provider _engine from Engine to AbstractEngine and drop the "as Engine" casts. - The remaining WebGL _gl access in the RT provider is temporarily narrowed via ThinEngine and relocated to a WebGL-specific provider in phase 0 PR 2. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…sionsObject (#18637) Now that framebufferDimensionsObject lives on AbstractEngine, the spectator-camera reset in webXRExperienceHelper no longer needs to cast the engine to ThinEngine. Remove the cast and the now-unused ThinEngine import. No behavior change. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…18637) Phase 0 of the WebGPU-for-WebXR epic. Behavior-preserving, WebGL-only. Keep WebXRLayerRenderTargetTextureProvider API-agnostic: it no longer imports any WebGL type. It exposes two protected hooks: - _createRenderTargetTextureShell(width, height, multiview): builds the RenderTargetTexture/MultiviewRenderTarget with the right sample count. - _createRenderTargetTextureInternal(...): assembles a RenderTargetTexture from already-wrapped InternalTextures, with no graphics-API types. This is the seam a future GPU backend (XRGPUBinding + wrapWebGPUTexture) uses. Move all WebGL framebuffer/texture wiring into a new WebGL-specific abstract provider, WebXRWebGLRenderTargetTextureProvider, which owns the WebGL-typed _createRenderTargetTexture and _createInternalTexture (WebGLHardwareTexture via engine._gl). The exact original creation order is preserved (framebuffer assigned before setTexture). Repoint all WebGL-backed subclasses to the new base: WebXRWebGLLayerRenderTargetTextureProvider, NativeXRLayerRenderTargetTextureProvider, and WebXRCompositionLayerRenderTargetTextureProvider (and thus the projection-layer provider that extends it). No public API removed or renamed. No WebGPU code added. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ics-binding seam (#18637) Phase 0 of the WebGPU-for-WebXR epic. Behavior-preserving, WebGL-only. WebXRRenderTarget is now generic with WebGL defaults: WebXRRenderTarget<TContext = WebGLRenderingContext, TLayer extends XRLayer = XRWebGLLayer>. Used without type arguments it resolves to the exact previous shape, so this is a strict compile- and runtime-compatible change. A future non-WebGL target can specialize the context/layer types. WebXRManagedOutputCanvas: factor the WebGL context creation (getContext webgl2/webgl) and the XRWebGLLayer construction into overridable protected seams (_createXRCompatibleRenderingContext / _createXRLayer). A future non-WebGL managed output canvas can override them; WebGL behavior is unchanged. Introduce a WebXRGraphicsBinding abstraction (new side-effect-free file) hiding XRWebGLBinding vs a future XRGPUBinding: IWebXRGraphicsBinding + WebXRGraphicsBindingType + a WebGL implementation (WebXRWebGLGraphicsBinding) whose CreateFromEngine factory localizes the WebGL context access. WebXRSessionManager exposes a typed accessor (_getGraphicsBinding) that lazily creates and caches it and resets it on session end. The seam is introduced but not yet consumed; the XR features keep using XRWebGLBinding directly and are migrated in a later phase. No public API removed or renamed. No WebGPU/XRGPUBinding code added. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
RaananW
force-pushed
the
raananw-webgpu-xr-phase0-rtt-split
branch
from
3ca1a38 to
8bc6ac9
Compare
RaananW
force-pushed
the
raananw-webgpu-xr-phase0-binding-seam
branch
from
212399b to
90635fd
Compare
…#18637) The graphics-binding abstraction has no consumer until the features are ported (Phase 4), and its per-operation shape will be designed then. Avoid committing it as public API now: mark IWebXRGraphicsBinding, WebXRWebGLGraphicsBinding and WebXRGraphicsBindingType @internal and drop them from the XR barrels (index.ts / pure.ts). WebXRSessionManager keeps importing them via the direct relative path, so nothing internal breaks and the public API surface is unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
RaananW
changed the base branch from
raananw-webgpu-xr-phase0-rtt-split
to
master
RaananW
changed the title
This was referenced Jul 2, 2026
Collaborator
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
Collaborator
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18645/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18645/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18645/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
Collaborator
Collaborator
🟢 Memory Leak Test Results4 passed, 0 leaked out of 4 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (4)
|
Collaborator
|
WebGL2 visualization test reporter: |
Contributor
There was a problem hiding this comment.
Pull request overview
Refactors the WebXR rendering pipeline to introduce graphics-API-agnostic seams (preparing for a future WebGPU-backed XR path) while preserving current WebGL XR behavior, primarily by generalizing engine typings and splitting WebGL-specific render-target wiring into dedicated abstractions.
Changes:
- Promotes
framebufferDimensionsObjectstorage toAbstractEngineand updates WebXR engine references toAbstractEngine. - Splits XR render-target-texture provider into an API-agnostic base plus a WebGL-specific provider to isolate WebGL framebuffer/texture wiring.
- Generalizes
WebXRRenderTargettyping and introduces internal seams for managed-output-canvas context/layer creation and a (currently unused) graphics-binding abstraction.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/dev/core/src/XR/webXRWebGLRenderTargetTextureProvider.ts | New WebGL-specific XR RTT provider that owns WebGL framebuffer/texture wiring. |
| packages/dev/core/src/XR/webXRWebGLLayer.ts | Switches WebGL layer RTT provider to extend the new WebGL-specific base provider. |
| packages/dev/core/src/XR/webXRTypes.ts | Makes WebXRRenderTarget generic with WebGL-defaulted type parameters. |
| packages/dev/core/src/XR/webXRSessionManager.ts | Retypes engine to AbstractEngine and adds an internal lazy graphics-binding accessor. |
| packages/dev/core/src/XR/webXRRenderTargetTextureProvider.ts | Makes the base RTT provider API-agnostic and adds shell/internal RTT creation hooks. |
| packages/dev/core/src/XR/webXRManagedOutputCanvas.ts | Adds protected seams for XR-compatible context creation and XR layer creation. |
| packages/dev/core/src/XR/webXRGraphicsBinding.ts | Introduces an internal graphics-binding abstraction with a WebGL implementation. |
| packages/dev/core/src/XR/webXRExperienceHelper.ts | Removes ThinEngine cast for framebufferDimensionsObject usage. |
| packages/dev/core/src/XR/native/nativeXRRenderTarget.ts | Repivots native provider to the WebGL-specific provider base. |
| packages/dev/core/src/XR/features/Layers/WebXRCompositionLayer.ts | Repivots composition-layer provider to the WebGL-specific provider base. |
| packages/dev/core/src/Engines/thinEngine.pure.ts | Removes framebufferDimensionsObject storage from ThinEngine (now inherited). |
| packages/dev/core/src/Engines/abstractEngine.pure.ts | Adds framebufferDimensionsObject storage/setter to AbstractEngine. |
Collaborator
Collaborator
🟢 Memory Leak Test Results4 passed, 0 leaked out of 4 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (4)
|
…hase0-binding-seam
Collaborator
|
Visualization tests for WebGPU |
Collaborator
|
WebGL2 visualization test reporter: |
- Guard the WebGL `_gl` access in WebXRWebGLRenderTargetTextureProvider and WebXRWebGLGraphicsBinding.CreateFromEngine so a non-WebGL engine fails with a clear, targeted error instead of a confusing downstream one. - Guard WebXRSessionManager._getGraphicsBinding against a disposed engine and a not-yet-initialized session (also drops the non-null assertion). - Explicitly disallow multiview in the API-agnostic _createRenderTargetTextureInternal hook, since only the single-texture path is wired there today (the WebGL provider still owns the multiview array path). - Fix the framebufferDimensionsObject doc comment moved onto AbstractEngine (no gl fallback at that level). No WebGL2 XR behavior change; guards only fire on paths unreachable in Phase 0. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Member
Author
The last phase of this work is documentation and full tests. It's going to be all in all 5 phases. |
Collaborator
Collaborator
🟢 Memory Leak Test Results4 passed, 0 leaked out of 4 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (4)
|
This was referenced Jul 2, 2026
Merged
Collaborator
|
Visualization tests for WebGPU |
Collaborator
|
WebGL2 visualization test reporter: |
Collaborator
⚡ Performance Test Results🟢 All performance tests passed — no regressions detected. |
Add unit tests locking in the guard contracts introduced by the Phase 0 seams, all using NullEngine (no real XR session / WebGL context required): - WebXRSessionManager._getGraphicsBinding throws before the session is initialized and after the engine is disposed. - WebXRWebGLGraphicsBinding.CreateFromEngine throws for a non-WebGL engine. - WebXRLayerRenderTargetTextureProvider._createRenderTargetTextureInternal throws for multiview (owned by the WebGL provider path). Behavior-preserving: these assert already-shipped guards; no source change. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Collaborator
Collaborator
🟢 Memory Leak Test Results4 passed, 0 leaked out of 4 scenarios 🟢 All memory leak tests passed — no leaks detected. Passed Scenarios (4)
|
Collaborator
|
Visualization tests for WebGPU |
Collaborator
|
WebGL2 visualization test reporter: |
Collaborator
⚡ Performance Test Results🟢 All performance tests passed — no regressions detected. |
RaananW
added a commit
that referenced
this pull request
Jul 3, 2026
… 1, #18638) (#18650) ## Phase 1 (#18638) — WebGPU-compatible XR session + XRGPUBinding plumbing Closes #18638 (Phase 1 of the WebGPU-for-WebXR epic #18635 — the epic stays open for Phases 2–5). Part of the WebGPU-for-WebXR effort (epic #18635). **Plumbing only — no per-frame WebGPU XR rendering (Phase 2), no camera/NDC (Phase 3), no feature rewiring (Phase 4).** WebGL2 XR behavior is byte-for-byte identical; every WebGPU path is gated behind `AbstractEngine.isWebGPU` and is unreachable for WebGL engines. > Consolidated from an initial 3-PR stack into this single PR (kept as focused commits for readability). Supersedes #18651 and #18652. Rebased onto master after Phase 0 (#18645) merged. ### What's in here **1. TS typings + adapter opt-in** - New companion `packages/dev/core/src/LibDeclarations/webxr.webgpu.d.ts` (mirrors the existing `webxr.nativeextensions.d.ts` split so the large community-maintained `webxr.d.ts` stays untouched). Declares `XRGPUBinding`, `XRGPUSubImage`, and the `XRGPU*LayerInit` dictionaries. Base XR layer/sub-image types come from `webxr.d.ts`; `GPURequestAdapterOptions.xrCompatible` stays in `webgpu.d.ts` (not duplicated). - Documented `xrCompatible` on `WebGPUEngineOptions`. It is inherited from `GPURequestAdapterOptions` and already flows end-to-end (`initAsync` passes `_options` straight to `navigator.gpu.requestAdapter()`), so this is JSDoc/discoverability only — it notes the flag must be set at adapter-request/engine-construction time (WebGPU has no post-hoc "make XR compatible" step). **2. `@internal` WebGPU graphics binding + wiring** - `WebXRWebGPUGraphicsBinding` implementing the Phase-0 `IWebXRGraphicsBinding` seam, wrapping `new XRGPUBinding(session, device)`, plus a `WebXRGraphicsBindingType.WebGPU` enum member. Minimal/introduced-but-unused — per-frame layer/sub-image ops are deferred to later phases. - `WebXRSessionManager._getGraphicsBinding()` branches on `AbstractEngine.isWebGPU`: WebGPU engine → `XRGPUBinding`-backed binding; everything else → the existing `XRWebGLBinding`-backed one. The `GPUDevice` is read from `WebGPUEngine._device` via a **type-only** import + cast (mirrors Phase 0's `(engine as ThinEngine)._gl`), so there is no runtime coupling/side effect. Kept `@internal` and out of `index.ts`/`pure.ts`. **3. WebGPU-compatible session gating** - `initializeSessionAsync` requests the `'webgpu'` feature descriptor as a **required** feature when the engine is WebGPU (a WebGPU engine cannot fall back to a WebGL XR session; the native `requestSession` rejection is left to propagate to the caller, not swallowed). The WebGL branch leaves the `XRSessionInit` object untouched (same reference passed through — asserted in a test). - `webXRExperienceHelper.enterXRAsync` skips the `baseLayer`/`XRWebGLLayer` path for WebGPU, since a WebGPU-compatible session is **layers-only**. **4. Doc-only clarification** of the ENTERING_XR end-state (see below) — comment-only, no logic change. ### ✅ Hardware-validated Phase 1 end-state (not a regression) Validated on Meta Quest Browser with the experimental `XRGPUBinding` flag, against this PR's snapshot build. The precise Phase 1 end-state is: > The native WebGPU XR session **enters** — an `xrCompatible` WebGPU engine is created, the `'webgpu'` feature is accepted and appears in `session.enabledFeatures`, `baseLayer` is correctly skipped, and `enterXRAsync` resolves without throwing. **But** because Phase 1 attaches **no layer** (the `XRProjectionLayer` is Phase 2), the session receives **no `requestAnimationFrame` callbacks**, so `onXRFrameObservable` never fires and Babylon's `WebXRState` **stays at `ENTERING_XR` and never reaches `IN_XR`** (which is gated on the first frame). This is the correct Phase 1 result, **not a regression**. This "no layer → no frame" behavior was confirmed both through Babylon and at the **raw-browser level**: a bare `navigator.xr` `immersive-vr` session requested with `requiredFeatures: ['webgpu']` is granted `'webgpu'`, yet a `requestAnimationFrame` with no layer set yields **zero callbacks**. So the stall is inherent to the layers-only WebGPU-XR design and the WebXR spec, not a bug in this plumbing. Reaching `IN_XR` / actually rendering is **Phase 2** (projection-layer RTT provider via `wrapWebGPUTexture`). **Exit is clean from `ENTERING_XR`:** ending the session via its native `end` path (e.g. the headset system menu) fires `onXRSessionEnded` regardless of `WebXRState`, restoring the framebuffer/render loop/camera and returning `WebXRState` to `NOT_IN_XR` with no hang or leak (the `@internal` binding is nulled on end). Note: the programmatic `exitXRAsync()` is a no-op from `ENTERING_XR` (its pre-existing `state !== IN_XR` guard, unchanged by this PR) — a no-op, not a hang; the session remains exitable via the native path. A unit test confirms `updateRenderState` with neither `baseLayer` nor `layers` does not throw. /cc @RaananW. ### Spec grounding Declarations/behavior are taken from the immersive-web/WebXR-WebGPU-Binding explainer + proposed IDL: https://github.com/immersive-web/WebXR-WebGPU-Binding/blob/main/explainer.md — including the `'webgpu'` feature descriptor, the layers-only rule (no `baseLayer`), and the `XRGPUBinding(session, device)` shape requiring an `xrCompatible` adapter. ### Constraints honored - WebGL2 XR byte-for-byte identical; all WebGPU logic gated behind `isWebGPU`. - No `.pure.ts` split, no top-level side effects (confirmed: no side-effect-manifest churn on any commit). - **Zero net public-API additions** — the binding is `@internal` and out of the barrels; `xrCompatible` is an inherited, now-documented option. ### Testing - `tsc -b tsconfig.devPackages.json` exit 0. - `npm run format:check` ✅ ; `npm run lint:check` ✅ (eslint + all 16 tree-shaking checks + side-effects-sync across core/gui/loaders/serializers). - XR unit gate `vitest run --project=unit -t "XR"`: **260 passed**. `webXRSessionManager.test.ts` cases cover binding selection (WebGL vs WebGPU), binding caching, the uninitialized-session guard, `'webgpu'` injected only for WebGPU engines (existing required features preserved, no duplicates), WebGL init passed through unchanged (same object ref), `updateRenderState` with no layer not throwing, and the native session-`end` handler cleaning up (nulls the graphics binding, clears `inXRSession`, notifies `onXRSessionEnded`) even when no frame ever arrived. New `webXRExperienceHelper.test.ts` covers the no-frame WebGPU lifecycle: state stays at `ENTERING_XR` with no frame, `exitXRAsync()` from `ENTERING_XR` is a clean no-op (does not touch the session manager, state unchanged), and a native session end from `ENTERING_XR` returns state to `NOT_IN_XR` with no hang or leak. - **Hardware-validated** on Meta Quest Browser (XRGPUBinding flag) against the PR snapshot — see the end-state section above. - Note: the unrelated `smartFilterBlocks` vitest suite fails to import in this environment (missing generated `.block.js` artifacts) — pre-existing, not touched by this PR. --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WebGPU support for WebXR — Phase 0 (behavior-preserving, WebGL-only refactor)
Part of #18637 (epic #18635).
Phase 0 introduces API-agnostic seams into Babylon WebXR so a future WebGPU-backed XR session (layers-only,
XRGPUBinding,engine.wrapWebGPUTexture) can plug in later. It is a strict no-op for existing WebGL2 XR — no functional change, no WebGPU/XRGPUBindingcode. Public API surface is unchanged (additive/backward-compatible only).1) Engine typing
framebufferDimensionsObject(public setter + protected backing field) fromThinEngineup toAbstractEngine. Additive and backward-compatible:ThinEngine/Enginekeep consuming/overriding it (Engine keeps theonResizenotification; ThinEngine keeps the_glfallback ingetRenderWidth/Height), WebGPU inherits harmless default storage. No new getter — the write-only read semantics are preserved exactly.WebXRSessionManager._engineandWebXRLayerRenderTargetTextureProvider._enginefromEnginetoAbstractEngine; removes thescene.getEngine() as Enginecasts, plus the leftoveras ThinEnginecast inwebXRExperienceHelper.ts.2) RT-provider split
WebXRLayerRenderTargetTextureProvider(base) is now graphics-API-agnostic (no WebGL imports) and exposes two protected hooks:_createRenderTargetTextureShell(width, height, multiview)— builds theRenderTargetTexture/MultiviewRenderTargetwith the correct sample count._createRenderTargetTextureInternal(...)— assembles aRenderTargetTexturefrom already-wrappedInternalTextures with no graphics-API types. This is the seam a future GPU backend uses (wrapWebGPUTexture→_createRenderTargetTextureInternal). Intentionally unused in Phase 0.WebXRWebGLRenderTargetTextureProviderowns the WebGL-typed_createRenderTargetTexture/_createInternalTexture(WebGLHardwareTextureviaengine._gl). The exact original creation order is preserved (framebuffer assigned beforesetTexture).3) Type generalization + graphics-binding seam
WebXRRenderTargetis now generic with WebGL defaults:WebXRRenderTarget<TContext = WebGLRenderingContext, TLayer extends XRLayer = XRWebGLLayer>. Used without type args it resolves to the exact previous shape (compile- and runtime-compatible). Stays public.WebXRManagedOutputCanvascontext/layer creation factored into overridable protected seams_createXRCompatibleRenderingContext()and_createXRLayer(); WebGL behavior is byte-for-byte unchanged. Kept offAbstractEngine's public surface deliberately (WebGPU XR is layers-only and may never use a managed-output baseLayer).WebXRGraphicsBindingabstraction (IWebXRGraphicsBinding+WebXRGraphicsBindingType+ WebGL implWebXRWebGLGraphicsBinding) hidingXRWebGLBindingvs a futureXRGPUBinding, plusWebXRSessionManager._getGraphicsBinding(). Marked@internaland not barrel-exported — introduced but unconsumed until Phase 4 proves the per-op shape, so it stays off the public API surface.Deliberate non-changes / scope
_createRenderTargetTextureInternalare introduced but not yet consumed. XR features (WebXRLayers,WebXRRawCameraAccess,WebXRDepthSensing,WebXRLightEstimation,WebXRSpaceWarp) keep constructingXRWebGLBindingdirectly — porting them is Phase 4.WebXRLayerWrapper/WebXRLayerTypewere already API-agnostic; thelayerType == "XRWebGLLayer"foveation guard stays for behavior preservation._colorTextureArray/_depthStencilTextureArray) still lives in the WebGL_createRenderTargetTexture; the GPU-agnostic_createRenderTargetTextureInternalonly handles the single-InternalTexturepath today. Equivalent GPU path is tracked for Phase 2 ([WebGPU-XR][Phase 2] WebGPU projection layer + render target provider #18640).Validation
tsc -b tsconfig.devpackages.json✅format:check(prettier) ✅ /lint:check(eslint + ratchets) ✅check:treeshaking(16 checks) ✅ /check:side-effects-sync(all 4 packages) ✅@internal;framebufferDimensionsObjectis additive onAbstractEngine;WebXRRenderTargetgenerics are default-compatible).AbstractEngineretype, context seam yields a realWebGL2RenderingContext). In-headset Quest Browser WebGL2 XR smoke is the final gate before merge.