Proposal: DisplayXR for Autodesk VRED (automotive visualization, Leia SR tested)

[joehillthunder]

Proposal: DisplayXR for Autodesk VRED (automotive visualization, Leia SR tested)

I've built a thin adapter that runs Autodesk VRED's built-in OpenXR mode on DisplayXR, so VRED scenes show in glasses-free 3D on a DisplayXR display. It's tested on a Leia SR laptop. I'd like the maintainers to review the approach, and to add VRED to DisplayXR's compatible applications: hosting as DisplayXR/displayxr-vred (experimental), and a displayxr.org listing after a tagged release, as proposed for Omniverse (#2331), 3D Slicer (#2352) and ParaView (#2360).

Code: joehillthunder/displayxr-vred (private for now, Apache-2.0 + NOTICE). @dfattal is invited as a collaborator; personal repos can only grant write.

Why VRED

VRED is a widely used tool for automotive and industrial design review: real-scale, physically based car and product models. Designers already review those models in headsets. A desk-sized glasses-free display gives the same depth during everyday desk work, with VRED's UI readable on the same screen and nothing on anyone's head.

Approach: a thin adapter, dedicated 3D window

This follows the architecture direction (texture mode or a dedicated 3D window, plus a shared host library with a thin per-app adapter):

• VRED Design has no scripting. It has no Python, Terminal or Script Editor, and script plugins never load. (VRED Pro has Python.) Nothing can draw a woven texture into VRED's viewport, so VRED falls in the dedicated 3D window class.
• No VRED plug-in. VRED ships its own OpenXR loader, which honours XR_RUNTIME_JSON, and has an OpenXR HMD display mode (OpenGL renderer). The launcher selects DisplayXR for this process only, sets DXR_LEGACY_CAMERA_RIG from a JSON file, starts VRED, and switches it to View › Display › OpenXR HMD through UI Automation. VRED then runs as a hosted legacy app in the runtime's own window. The shared depth slider is docked in VRED's menu bar.
• Mouse navigation stays VRED's. A drag in VRED's own render viewport orbits the camera, and the 3D view follows. VRED's camera remains the source of truth.
• Camera rig (legacy). VRED scenes are real scale, and VRED puts the eye 1.7 m above the floor. The rig is: convergence 5 m, horizontal FOV 45°, ipdFactor 0.5, pitch −15°, pullback 1.5 m (Proposal: legacy camera profile pitchDegrees + pullbackMeters (starting framing) #2334). That frames the whole demo car. The true-geometry rig the Slicer and ParaView adapters use needs a per-frame scene scale, which VRED Design can't run. Panel size and nominal distance come from display info.
• The overlay embed (Proposal: embed hosted windows in the app's own window (DXR_HOSTED_WINDOW, viewer inset, F11) #2332, viewport rect from VRED's GL surface) was verified to fit VRED's viewport exactly. It's kept as a test-only switch (DXR_VRED_OVERLAY=1) and won't be developed further.

Tested

VRED Design 2027.1 (Windows) on DisplayXR 2.53.0 (dev build with #2332 / #2350 / #2353), with the Leia SR plug-in and a Leia SR panel (3840 × 2400). Demo: VRED's Automotive_Genesis.vpb.

• Session. The session is hosted legacy, GL native compositor, 1920 × 2400 per view, and reaches FOCUSED automatically from the launcher. It renders at about 25 fps in the dedicated window. In overlay-embed mode it ran at about 3 fps, a second reason to drop that path for VRED.
• Framing. Captures show the whole car centred (capture_frame, post-compose).
• Mouse. A posted left drag on VRED's viewport rotated the car in the 3D view.
• Not measured yet:
  • Parallax against the comfort budget. The atlas trigger isn't consumed on VRED's GL path. Next I'll try capture_frame on sim-display.
  • Gamepad / controller mapping of VRED's VR actions (teleport, menu) via Proposal: gamepad EMULATE_PROFILE=touch — pad-driven oculus/touch_controller hands #2333 Touch emulation.
  • Concurrent weaving next to other apps, and the F11 round trip in dedicated-window mode.
• Platforms. Windows only: VRED is Windows-only.
• Omniverse. VRED 2027 has native USD import and export, so scenes can go to Omniverse without a connector. The Kit round trip isn't checked yet.

Runtime-side findings (for discussion)

1. Locked desktop. At the Windows lock screen VRED creates the instance, session and swapchains but never calls xrPollEvent / xrBeginSession, so the session stays READY. Other legacy apps keep rendering while locked. This looks like VRED-side behaviour, but it's worth knowing for unattended tests.
2. Atlas trigger on GL. %TEMP%\displayxr_atlas_trigger.VREDDesign wasn't consumed on VRED's GL path; capture_frame works.
3. External apps and the shared host library. VRED can only be driven from outside the process. The shared host library should therefore also cover apps whose own OpenXR session is the only hook: a runtime-owned dedicated 3D window with the app's camera as the source of truth, not only in-process adapters.

Asks

• A review of the approach.
• Adding VRED to DisplayXR's compatible applications (hosting as DisplayXR/displayxr-vred, experimental, me as maintainer; a displayxr.org listing after a tagged release).
• Guidance on how out-of-process apps like VRED fit the coming texture-mode / dedicated-window contract.

[joehillthunder]

Final update: VRED (2026-10-25)

Code: joehillthunder/displayxr-vred @ 26eb508 (private, Apache-2.0 + NOTICE; COUPLING.md marks every VRED and runtime internal as "not a contract").

Target class: dedicated 3D window. VRED Design has no scripting, so nothing can draw a woven texture into its viewport.

• VRED's own OpenXR HMD mode runs as a hosted legacy app in a runtime-owned 3D window.
• The adapter is thin: a launcher, a rig JSON, and a UI Automation switch to View › Display › OpenXR HMD.
• The overlay embed (Proposal: embed hosted windows in the app's own window (DXR_HOSTED_WINDOW, viewer inset, F11) #2332) stays as a test-only switch.
• The shared host library must also accept an external OpenXR app, where the app's own session is the only hook. It can't assume an in-process adapter for every app.

Measured on the tested display's simulator (projection atlas, Automotive_Genesis.vpb):

• Rig: convergence 6.5 m, 45° horizontal FOV, ipdFactor 0.7, pitch −15° / pullback 1.5 m (Proposal: legacy camera profile pitchDegrees + pullbackMeters (starting framing) #2334).
• Result: −0.081 % of view width in front and +0.404 % behind (p99), inside the 0.14 % / 0.42 % comfort budget.
• The shared depth control (DXR_HostedDepthRel) works in the dedicated window: 50 % gives −0.046 / +0.193 %.
• F11 full-panel round trip: works.
• Frame rate: about 25 fps in the dedicated window, against about 3 fps through the overlay embed.

Input:

• VRED's desktop mouse navigation drives the 3D camera.
• VRED suggests bindings for Touch, Vive, Index and WMR controllers, but not khr/simple_controller, while the stock gamepad provider's emulated hands report khr/simple_controller. A gamepad therefore needs Touch emulation (Proposal: gamepad EMULATE_PROFILE=touch — pad-driven oculus/touch_controller hands #2333).
• That's untested: the dev layer was refused by the trust gate, and no pad was available.

Named gaps:

• The rig constants are scene-scale values, because VRED Design can't set a per-frame scene scale. ipdFactor should come from a comfort budget in display info (Display info: publish the comfort budget (front/behind disparity limits), e.g. layout.comfort {front_frac, behind_frac} #2388).
• VRED renders only on an unlocked desktop. That's on VRED's side: it never calls xrBeginSession while the desktop is locked.
• The VRED → USD → Kit route (VRED 2027 native USD export) is untested.

Pending:

• a seated check;
• the gamepad path;
• the Omniverse round trip.

🤖 Generated with Claude Code

[dfattal]

Thanks for the update. Putting each app into one of the two target classes (its own viewport, or a dedicated 3D window), with an explicit list of what it needs from a shared host library, is the right framing. It is the input we're using to design that library.

The overlay PRs (#2332, #2350, #2353) stay as measurement rigs, as you say. We'll post the host-library API here for review when there's a first draft.

[joehillthunder]

Hosting-readiness check (2026-10-27)

VRED Design 2027.1 on the installed runtime v2.55.0, default launcher (commit af17fe1):

• 3D active on the leia-sr plug-in in the runtime's dedicated window, with the rig accepted.
• Seated check clean at ipdFactor 0.05: "much better".
• Exit: clean, except one forced close after VRED hung on a menu.

Findings:

• Unknown rig keys: a released runtime drops the whole camera profile if it contains any unknown key, such as the unreleased Proposal: legacy camera profile pitchDegrees + pullbackMeters (starting framing) #2334 pitchDegrees / pullbackMeters. The launcher now strips them unless the dev runtime is selected. Filed with the runtime.
• sim-display vs the panel: rigs tuned on sim-display were about 10× too strong on the panel. Filed with the runtime.
• Zooming: a fixed per-session rig lets zoom bring content in front of the glass (Focus on camera rigs: convergence isn't a pure glass shift, and the comfort clamp assumes content at infinity #2385).
• Python: a probe plug-in in VRED Design's own Scripts folder did not load. User Python doesn't run in Design by any route we found, which confirms the dedicated-window class.

Pending: gamepad input, and a CC0 backstreet demo set (built, not yet imported).

🤖 Generated with Claude Code

[joehillthunder]

Seated check on the released runtime: Pass (2026-10-29)

VRED Design 2027.1 on the installed runtime v2.57.1 with the Leia SR plug-in 2.19.1. Scenes: the Genesis sample and a CC0 street set. Rig: horizontal FOV 65°, ipdFactor 0.12, mid-tone sky dome (no pure black).

Check	Result	Conditions
Framing (subject fully in view at a middle size)	Pass	whole car in view at FOV 65° (45° was too close)
Depth, % of panel width	Pass	p5 −0.07 % in front (budget 0.14 %), p50 +0.31 %, p95 +0.37 % behind (budget 0.42 %)
Seated check at the tested display (Leia SR)	Pass	the rig above
Clean exit	Pass	lens back to 2D, window destroyed, no save

Notes:

• The p1 / p99 tails reach −0.18 % / +0.85 %; probably matcher noise on repeating brick textures, not yet checked.
• Frame rate not measured.
• Placement: VRED is a legacy OpenXR app (no plug-ins in Design), so its 3D shows in the runtime's dedicated window beside VRED, without a snap-back key.
• Known, low priority: VRED's own desktop mirror shows stretched bands above and below its viewport. The panel output isn't affected.