Proposal: DisplayXR for Adobe Substance 3D Painter (dedicated 3D window rendered by Painter's own viewport renderer, Leia SR tested)

[joehillthunder]

Substance 3D Painter has no OpenXR mode, and its plug-in API (Python 3.13 / PySide6) has no render or draw hook into its Vulkan viewport. So this integration uses the dedicated-3D-window class from the target-class comment on #2331:

• A Painter plug-in opens a 3D window beside Painter on the shared app-neutral host core, the same dxrh_* core as MotionBuilder (Proposal: DisplayXR for Autodesk MotionBuilder (in its own viewer via the texture target; an in-process adapter on an app-neutral host core) #2423), Nuke (Proposal: DisplayXR for Foundry Nuke (3D in Nuke's own Viewer via the texture target; NDK node on the shared host core, Leia SR tested) #2433) and Houdini (Proposal: DisplayXR for SideFX Houdini (HDK adapter on the shared host core, dedicated 3D window; Leia SR tested) #2432).
• Painter's own camera drives it.

Code: joehillthunder/displayxr-substance-painter @ fd646d1 (private, Apache-2.0 + NOTICE; every not-a-contract item in coupling.py; cgltf MIT in third_party).

What it does

• The eyes are rendered by Painter itself, with its own viewport renderer. For each eye, in one main-thread call, the plug-in:

  1. moves the live camera to the eye, aimed at the centre of that eye's off-axis window, with a symmetric frustum that just holds it (1.5× supersampled);
  2. renders offscreen;
  3. restores the camera.

  A small DLL then reprojects each render into the eye's off-axis image (an exact homography) and submits the pair. The offscreen render uses a private function, kept behind the coupling module.

• Live while painting: pairs re-render only when the eyes, the camera or the paint change (about 70 ms per pair here), so strokes show up in 3D as you paint.

• Depth:

  • The glass sits where the front and behind comfort limits meet for the surface the camera actually sees (mesh points through a coarse z-buffer), and the host core applies the budget.
  • Depth −/+ in 25 % steps and Reset, with an explicit "Allow depth above comfort" opt-in.

• Input: mouse orbit, pan, dolly and frame in the 3D window, and an XInput pad, drive Painter's camera. Painter's own navigation drives the 3D.

• Send to Omniverse (USD): Painter's glTF export written as Kit-layout USD (Y-up, cm, UsdPreviewSurface with base/ORM/normal, relative textures).

Measured

Tested display: Leia SR panel, runtime v2.55.0, Painter 12.2.0, Windows 11.

• Seated check: clean edges.
• The shipped default is 100 % (0.14 % front / 0.42 % behind, solved by the host core). For one textured asset, the seated viewer chose 150 % (0.20 % / 0.63 %), which sits behind the explicit above-comfort opt-in and isn't the default.

What would make this native

• The minimum: a per-eye off-axis render into an app-owned texture from the plug-in API.
• Better: a viewport draw-back hook plus the viewport rect, so the woven image can be drawn inside Painter's own viewport through the texture target (XR_DXR_win32_window_binding v9), as Maya (Proposal: DisplayXR for Autodesk Maya (in-viewport Viewport 2.0 plug-in, Leia SR tested) #2390), MotionBuilder and Nuke do.
• Runtime side: the comfort budget in display info (Display info: publish the comfort budget (front/behind disparity limits), e.g. layout.comfort {front_frac, behind_frac} #2388).

Runtime finding

In-process sessions can silently fall back to sim-display, because the vendor plug-in's dependencies aren't found from some host processes: #2436.

Pending

• Opening the USD export in USD Composer.
• Running beside another 3D app.
• A stroke-to-3D latency measurement.
• Confirming the viewport-toolbar flicker is gone; repaints are now held during pair renders.

🤖 Generated with Claude Code

[joehillthunder]

Hosting-readiness check: pass (2026-10-27)

Installed runtime v2.55.0 with Leia SR plug-in 2.18.3, the default launcher, and a clean environment:

• "3D active". The log shows loaded from: C:\Program Files\DisplayXR\Runtime\DisplayXRClient.dll, active plug-in: id=leia-sr and System head device: 'Leia 3D Display'. There's no sim-display fallback: the plug-in preloads the vendor dependency, a workaround for In-process runtime: leia-sr plug-in fails to load in some host processes (dependency not on the search path) and silently falls back to sim-display #2436.
• Depth: −/+ in 25 % steps, capped at 100 % without the opt-in. The default 100 % = 0.14 % front / 0.42 % behind.
• Exit: clean (session destroyed, window closed).
• Fixed during the check: a fresh launch fell back silently to a 2D preview, because the viewport-freeze code held a stale Qt widget. The check was re-run from a fresh launch on the fixed code.

🤖 Generated with Claude Code

[joehillthunder]

Final update: complete, nothing pending (2026-10-29)

• Send to Omniverse (USD): checked in Omniverse Kit 110. No composition errors; defaultPrim /World, Y-up, metersPerUnit 0.01; UsdPreviewSurface base colour, ORM and normal map all resolve; normals are OpenGL as exported. The project now stands on y = 0 (a translate on its root; the mesh is as painted, 69f6524). Rendered in Kit RTX with no errors.
• Hosting readiness: pass on the installed runtime (comment above). In-process runtime: leia-sr plug-in fails to load in some host processes (dependency not on the search path) and silently falls back to sim-display #2436 is closed; the plug-in keeps its preload until vendor dependencies load from the vendor directory.
• Depth: shipped default 100 % of the comfort budget; more only behind the explicit "Allow depth above comfort" opt-in.

Nothing else is open for Substance 3D Painter on our side. What would make it native is unchanged: a per-eye off-axis render into an app-owned texture, or a viewport draw-back hook plus the viewport rect for the texture target.