Proposal: DisplayXR for Autodesk MotionBuilder (in its own viewer via the texture target; an in-process adapter on an app-neutral host core)

[joehillthunder]

MotionBuilder has no OpenXR mode, so it runs through a thin Open Reality SDK plug-in:

• the plug-in renders the two eyes with MotionBuilder's own renderer;
• a small app-neutral host core runs the OpenXR session;
• the woven frame is drawn back, pixel-exact, into MotionBuilder's viewer through XR_DXR_win32_window_binding v9 (texture target).

Nothing covers the app: its menus, tooltips, manipulators and mouse stay its own. Status text reads "3D active".

Code: joehillthunder/displayxr-motionbuilder @ d53d68f (private, Apache-2.0 + NOTICE; internals in src/core/coupling.h and src/mobu/coupling.h).

What it does

1. On each redraw of MotionBuilder's viewer (after-render hook), it renders both eyes, with one helper camera per eye.
2. MotionBuilder cameras have no off-axis projection. So each eye renders a symmetric frustum containing the off-axis one, then crops the eye's rect, with one scale for both eyes and an identical crop size.
3. The host core runs the session with a shared D3D11 texture and the texture-target binding. The reference window is MotionBuilder's main window; the rect is the viewer widget, published from PySide6.
4. The woven frame is opened in MotionBuilder's GL context with WGL_NV_DX_interop2 and drawn 1:1 into the viewer.
5. A dedicated window stays as the fallback, and as a "Full panel" button (MotionBuilder keeps F11 and its other hotkeys).
6. Depth −/+ in 25 % steps, plus Reset. 100 % fills the comfort budget for what the camera sees; going above that needs an explicit opt-in.
7. Send to Omniverse (USD) in Kit's layout (Y-up, cm, /World defaultPrim, UsdPreviewSurface), using MotionBuilder's bundled OpenUSD 0.25.11.

Display values come from display info (panel size, nominal viewer). The comfort budget is still a constant (0.14 % front / 0.42 % behind of panel width), pending #2388.

Measured

• Sim-display atlas: the character at the glass measured +0.034 % of panel width (−0.03 % predicted from the runtime's eyes), with 0.03 px vertical error.
• Leia SR panel, runtime v2.55.0 with LeiaSR 1.38: anchor-fallback state 0x2, with the canvas woven at the viewer rect (fix(d3d11): texture target anchor fallback phases a sub-rect canvas where the app draws it (ADR-060) #2406). Windowless phasing needs LeiaSR ≥ 1.39.
• Concurrency: running at the same time as another weaving app, both kept weaving.

What a shared host library would need, from this adapter

1. The app-neutral core as a seed. Its C API (dxrh_start, dxrh_start_in_viewport, dxrh_set_target, dxrh_texture_info); eye hand-off with a serial, so frames are decoupled from the host's render rate; input and pad; and the shared slider's comfort clamp. It's shared with the Nuke adapter and reusable by Houdini.
2. A GL hand-off of the woven texture (interop open/lock), not only D3D11.
3. Self-pacing for texture sessions (xrWaitFrame doesn't pace a session that submits no layers (about 8,700 frames/s before the first image) #2386).
4. A per-toolkit rect publisher: Qt widget → device-pixel rect in the top-level client area.
5. Guidance that adapters read back the renderer's actual view and projection every frame. That read-back is what caught this app's bugs.

Runtime findings

• Texture-target sessions run unpaced too; see the comment on xrWaitFrame doesn't pace a session that submits no layers (about 8,700 frames/s before the first image) #2386.
• The per-exe atlas trigger doesn't capture texture-target sessions: Atlas capture: the per-exe trigger writes nothing for texture-target (client-presents) sessions; window mode works #2421.
• A question about eye placement on app-framed rigs: XrCameraRigDXR: with an app-framed verticalFov wider than canvas-true, the eyes land far above the app camera and the frustum is entirely below the horizon #2422.

Open

• Framing comes out closer than the app camera's (XrCameraRigDXR: with an app-framed verticalFov wider than canvas-true, the eyes land far above the app camera and the frustum is entirely below the horizon #2422).
• The eye camera label shows in the eyes.
• The helper cameras mark the scene as modified.
• Still to check: the export in USD Composer, running beside Kit, a physical pad, and a seated comfort check after the framing fix.

🤖 Generated with Claude Code

[joehillthunder]

Final update (2026-10-26)

Seated check. On the Leia SR panel, with 3D inside MotionBuilder's own viewer (texture target, runtime v2.55.0, LeiaSR 1.38 → anchor fallback 0x2, woven at the viewer rect): "comfortable, looks good."

Fixed since the post (all measured on sim-display; code joehillthunder/displayxr-motionbuilder @ 7865b3c, private):

1. Stereo pair.
   • MotionBuilder applies FBModel::SetMatrix at the next scene evaluation. So a single eye camera re-posed between the two renders drew the right eye from the left eye's pose. The adapter now uses one camera per eye: pose both, evaluate, then render. Both eyes share one render scale.
   • GetBoundingBox returns world-space bounds, and transforming them again put the content range 2× too far.
   • Every frame, the adapter now reads back the view, projection and viewport it actually rendered with.
2. Framing (XrCameraRigDXR: with an app-framed verticalFov wider than canvas-true, the eyes land far above the app camera and the frustum is entirely below the horizon #2422).
   • The app camera's vertical FOV no longer goes into XrCameraRigDXR.verticalFov. The rig stays canvas-true (vFOV from the window's physical height).
   • The host core moves the rig back along the view axis until the window frames the app camera's height at the glass; glass, near and far move with it.
   • It then turns the rig up about the glass point by atan(viewer y / viewer z) from display info, so the nominal viewer's line through the window centre is the app's view axis. The 3D now frames what MotionBuilder's camera frames, with true viewing angles.
   • In the core self test, the spheres measure −0.090 / −0.002 / +0.169 % (budget 0.14 / 0.42 %). In MotionBuilder, the character is fully framed at +0.176 %, with 0.07 px vertical error.
3. Clean eyes. The eyes no longer show the camera name or axis gizmo. The tool panel moves itself off the viewer while 3D runs, because an overlapping window drops the panel to 2D.

Shared host core. The Nuke adapter has adopted the same core byte-for-byte (apart from app names), including the framing fix and a shutdown fix from the Houdini adapter (request exit, end on STOPPING). That supports the shared-host-library direction: one app-neutral core, with thin per-app adapters.

Still open:

• a ~0.2 % residual between the measured disparity and the disparity predicted from the runtime's eyes;
• the helper cameras mark the scene as modified;
• Send to Omniverse not yet opened in USD Composer, and not yet run beside Kit;
• the pad untested with a physical pad.

Related: #2421 (atlas trigger in texture mode), #2422 (rig framing), #2386 (pacing).

🤖 Generated with Claude Code

[joehillthunder]

Update: fly-through comfort

The seated check on the new demo set found that flying through objects was uncomfortable. The cause: the framing pull-back (the #2422 workaround) also moved the eyes' near plane back, so the eyes kept drawing what the app camera had already flown past, far in front of the comfort limit.

Fix (commit f017940):

• The eye near plane now follows the pull-back, to the app camera's own near plane (3389 cm for a 3399 cm pull-back). Nothing behind the app camera is drawn.
• Content depth now clips bounds to the view frustum exactly, instead of sampling. Samples can miss a wall that fills the view.

Measured (sim-display, depth 100 %, 12 scripted camera stops: toward the wall, inside a barrier, between crates, then 60 → 20 → 5 cm from a brick wall):

• Front: ≤ 0.140 % at every stop; 0.086 % inside the barrier, where the near distance clamps to the camera's 10 cm near plane.
• Behind: ≤ 0.420 % at every stop.

For other adapters: any adapter that pulls the eyes back to keep the app's framing must move the eye near plane with them, and should test with a scripted fly-through. A seated recheck on the panel is next.

🤖 Generated with Claude Code

[joehillthunder]

Hosting-readiness check: pass (2026-10-27)

Leia SR panel, installed runtime v2.55.0 with the leia-sr plug-in 2.18.3, default launcher:

• the runtime loaded from Program Files;
• 3D woven in MotionBuilder's own viewer through the texture target (6,300+ frames).

Fly-through: a scripted run of 12 stops through the demo set (inside a barrier, between crates, up to a brick wall), with the eye near plane following the framing pull-back and exact frustum depth:

• front ≤ 0.140 % at every stop (0.092 % inside the barrier);
• behind ≤ 0.420 %.

Exit: clean. The session went STOPPING → destroyed, the panel switched back to 2D, and the weaver was released, with no errors.

Not exercised in this unattended run:

• the Depth −/+/Reset buttons (verified on 2026-10-26);
• the physical pad.

The ~0.2 % measured-vs-predicted residual stays open until a released runtime includes #2426.

🤖 Generated with Claude Code