Proposal: DisplayXR for Autodesk Maya (in-viewport Viewport 2.0 plug-in, Leia SR tested)

[joehillthunder]

Maya has no OpenXR mode, so a native plug-in is the only route. It's also the cleanest fit for the in-viewport texture model in the architecture direction.

Code: joehillthunder/displayxr-maya @ 13917cf (private, Apache-2.0 + NOTICE; internals in src/coupling.h). Prototype: Windows, Maya 2027, Viewport 2.0 on D3D11. Leia SR is the tested display.

What we built

• Rendering: a Viewport 2.0 render override draws one scene render per eye from Maya's own camera. Maya's UI, selection highlight and manipulators are drawn by Maya itself.
• Framing: the runtime's view rig gets Maya's framing. The vertical FOV is the camera's port FOV, and the glass is the camera's centre of interest (or a chosen focus distance).
• Depth:
  • The fit runs every few frames on what's in view. The nearest and farthest visible shapes (meshes, NURBS, and plug-in shapes such as MayaUSD proxies) fill the comfort budget: 0.14 % of panel width in front, 0.42 % behind.
  • Live Depth (0–100 %) and Focus controls, with "3D active" / "3D paused" / "Exit 3D".
• Display values: panel size and nominal viewing distance come from XR_DXR_display_info. The comfort budget is still a constant in the plug-in, because display info doesn't publish one (Display info: publish the comfort budget (front/behind disparity limits), e.g. layout.comfort {front_frac, behind_frac} #2388).
• Content round trip: MayaUSD references Y-up cm USD directly. An "Image → 3D" menu item generates an asset and references it, and the depth fit frames it with no extra work.
• Measured on the tested display: per-eye images are correct. The framed asset is analytically 0.140 % front / 0.076–0.106 % behind, inside the budget. Pixel measurement is coarse (±1 eye px ≈ 0.05 %).

How it presents today, and the target

• Today: a plug-in-owned, click-through window over the viewport panel (win32 window binding). That window must run on its own thread. If Maya's main thread owns it, Present waits on a thread that is itself blocked in xrWaitFrame/xrEndFrame; details are in a comment on Hosted D3D11 app hangs in the cross-adapter DXGI flip-present (dGPU render → iGPU-driven panel), with the repaint thread stuck behind c->mutex — looks like a crash and logs nothing #1000.
• Target: texture mode. VP2 already hands us both eye textures. The plug-in would give them, plus the viewport's screen rect, to the runtime, and the vendor plug-in would weave into Maya's own viewport. That removes the overlay window and its input caveats.

What Maya needs from the shared host library

• Session lifecycle with no app-owned window (texture + screen rect), including resize, dock/undock, panel maximise and multiple monitors.
• Display values: panel size, nominal viewer, and the comfort budget (Display info: publish the comfort budget (front/behind disparity limits), e.g. layout.comfort {front_frac, behind_frac} #2388).
• A device-agnostic hand-off: D3D11 on Windows today, OpenGL on Linux and Metal on macOS, which is where Maya's VP2 goes on those platforms.
• A shared depth/focus control channel, so the host-side slider and the app agree.

Open

• OpenGL / Linux: needs Linux: run OpenGL (XR_KHR_opengl_enable) legacy apps — VTK-based ParaView / 3D Slicer, Blender VR #2354, or GL → Vulkan interop inside the plug-in.
• Metal / macOS: needs a vendor plug-in, or sim-display for development.

Pending:

• a seated depth check;
• a manual mouse test (tumble, track, dolly, select, manipulators) with 3D on;
• side-by-side viewing with another weaving app;
• the F11 round trip.

🤖 Generated with Claude Code

[joehillthunder]

Update: Maya weaves inside its own viewport on the released v2.55.0

The plug-in now uses the ADR-060 texture target (XR_DXR_win32_window_binding v9). The woven image is drawn by Maya into its own Viewport 2.0 panel, with no overlay window. Code: joehillthunder/displayxr-maya @ 579b811 (private).

Results (installed runtime v2.55.0):

• sim-display, windowless: the woven image is drawn into Maya's viewport, and the target rect equals the viewport port size (no DPI mismatch).
• Leia SR plug-in 2.18.3 with LeiaSR 1.38, anchor fallback: the fix(d3d11): texture target anchor fallback phases a sub-rect canvas where the app draws it (ADR-060) #2406 path phases the viewport sub-rect, and the viewport target is byte-identical to the runtime's woven rect. About 120 frames/s, with no stall from anchoring on Maya's UI-thread window. That's the stall we saw with the old overlay (Hosted D3D11 app hangs in the cross-adapter DXGI flip-present (dGPU render → iGPU-driven panel), with the repaint thread stuck behind c->mutex — looks like a crash and logs nothing #1000).
• Full panel: switches to window mode and back cleanly.

Two integration notes:

• The runtime's woven output is BGRA. Drawing it into an RGBA host target needs a Load() shader blit, and SINGLETHREADED context-state handling on single-threaded devices.
• Anchoring on the host's own top-level window didn't stall in texture mode.

Pending:

• a seated check;
• the weave phase while the window is dragged;
• Leia's 2D drop under Maya's tooltips and menus;
• the depth −/+ steps and Reset;
• a gamepad;
• Send to Omniverse.

🤖 Generated with Claude Code

[joehillthunder]

Final update (2026-10-27)

Since this proposal was posted, the plug-in draws the 3D inside Maya's own viewport, and it runs side by side with another 3D app on the panel. Repo: joehillthunder/displayxr-maya (private), commits 579b811, 076bcb4 and c03eead.

What's new

• Texture mode (579b811): the woven 3D is drawn in Maya's own Viewport 2.0 viewport, through the v9 texture target on runtime v2.55.0. No click-through window over the viewport is needed, and Maya stays responsive while 3D is active.
• Maya's framing at the glass (c03eead, a workaround for XrCameraRigDXR: with an app-framed verticalFov wider than canvas-true, the eyes land far above the app camera and the frustum is entirely below the horizon #2422): each eye uses the canvas's true vertical field of view, and the rig is pulled back so the framing matches Maya's at the glass. The narrower field of view crops the horizon a little.
• Comfortable near plane: each eye's near plane is set to max(n, n + its pull-back along the view axis), so geometry close to the camera no longer jumps out of the comfort budget.
• Accurate depth: the comfort range is measured by clipping each shape's box exactly against the view frustum, instead of estimating from bounding spheres.
• Import GLB… (076bcb4): Maya has no glTF importer, so this menu item converts a .glb to USD and references it through a USD proxy shape. It can keep the file's own size or fit it to a size in metres, and it needs no API key. The same path imports image→3D results (Proposal: generated 3D content in DisplayXR apps (image→3D asset bridge with comfort-budget placement) #2391).

Measured on the Leia SR panel (runtime v2.55.0, texture mode, a textured CC0 forest scene)

• Glass at 5.20 m; the scene spans 2.89 m to 21.41 m.
• Parallax: 0.140 % in front and 0.133 % behind, against a budget of 0.14 % / 0.42 %. At 50 % depth the parallax halves.
• Side by side: Maya in 3D in its viewport ran next to Nuke (Proposal: DisplayXR for Foundry Nuke (3D in Nuke's own Viewer via the texture target; NDK node on the shared host core, Leia SR tested) #2433) in 3D in its Viewer, on the panel at the same time. Both looked right, and Maya stayed responsive.

Still open

• A seated viewing check of Maya on its own.
• Colour: Maya's ACES view transform isn't applied to the eye images yet, so the 3D looks brighter and less saturated than the 2D viewport.
• Xbox-controller navigation, Send to Omniverse, and a real plug-in icon.
• Platforms: only Windows (D3D11) is built; the OpenGL (Linux) and Metal (macOS) paths aren't.
• Runtime items: XrCameraRigDXR: with an app-framed verticalFov wider than canvas-true, the eyes land far above the app camera and the frustum is entirely below the horizon #2422 (canvas field of view) and Atlas capture: the per-exe trigger writes nothing for texture-target (client-presents) sessions; window mode works #2421 (atlas capture in texture mode). Once both land, the framing workaround can come out.

🤖 Generated with Claude Code