Proposal: DisplayXR for Adobe Substance 3D Sampler (live material in a dedicated 3D window, Leia SR tested)

[joehillthunder]

Follows the outline of Discussion #2397.

Summary

Substance 3D Sampler turns photos and scans into PBR materials. A Sampler plug-in,
DisplayXR, live-exports the selected material, and a small DisplayXR window beside
Sampler shows it as a real-size relief swatch in glasses-free 3D. The window follows
edits: change a parameter in Sampler and the 3D view updates a few seconds later.
The same export also feeds USD (Omniverse) and glb (Substance 3D Modeler).

Sampler's plug-in API has no viewport hook, so this is a dedicated 3D window rather
than 3D inside Sampler's own viewport. The last section lists the hook that would
allow the in-viewport path.

What it does

• Panel plug-in (Python + QML, Sampler 6.0.4): Show in 3D, Live (follow
  edits), Export full resolution (maps + .sbsar).
• Live loop: a hash of the layer stack and parameter values is polled every
  1.5 s. On a change, the material is exported (PNG, 2K, 16-bit height/normal) into an
  inbox folder and swapped in atomically, with a provenance sidecar carrying the
  real size.
• 3D window (viewer/sampler3d.py): a DisplayXR texture-target window, so it
  weaves windowed beside Sampler and other 3D apps. It renders relief from the height
  map with a key light and shadow, using the material view from the Substance 3D
  Designer integration.
  • Mouse orbit, pan and zoom; double-click to reframe.
  • Depth −/+ in 25 % steps and Reset; 100 % is the comfort budget.
  • F11 for the full panel, Esc back.
  • The gamepad drives the view only while the window is in front.
• USD / glb: the measured-materials library (Proposal: measured materials at true size in host 3D views (capture device → USD/MDL → Omniverse + Substance 3D Modeler) #2419) converts the same export into a live USD
  stage (UsdPreviewSurface + OmniPBR) and a glb.
• Status text says "3D active".

Measurements

• Panel (Leia SR, 2026-10-26): 3D active in the window beside Sampler; tracked
  eye images differ. One live edit (Geometry Details 0.5 → 1.0) took 13 s to export at
  2K, and the window reloaded under 0.5 s after the export landed. Sampler and the
  window both stayed responsive.
• Depth budget: a 180 cm swatch with 2 cm relief comes out at −0.14 % front and
  +0.16 % behind, within budget with no push-back.
• Sim-display: about 60 fps; Depth 50 % halves parallax as expected.
• Omniverse: headless Kit 110 check passes: 0 composition errors, all six textures
  resolve, OmniPBR binds, DirectX normals render the right way up. USD Composer in 3D
  reloaded a live re-export in about 1 s.
• Known issue: Sampler lost its GPU device twice while Kit RTX was in 3D on the
  same 16 GB GPU. The second loss hung Kit as well; displayxr-service survived. Sampler
  beside the 3D window above ran without problems.

What a shared host library needs

• A small, Qt-free renderer binding that a host can import as a package: load
  material maps from files, set relief and real size, and present into a
  caller-owned window rectangle. This window reuses the Designer integration's
  bindings as-is; a shared library would remove the cross-repo import.
• A shared depth control (−/+ in 25 % steps, Reset), so every dedicated window looks
  and behaves the same.
• A helper that states the normal-map convention, or measures it from the maps.
  In Sampler it's an app preference, not an export option.

Open items

• Viewport hook (the in-viewport path): Sampler would need to get and set the
  viewport camera, including an off-axis projection, render it into a caller-owned
  shared texture (D3D11 or Vulkan), and report the viewport's screen rectangle.
  Sampler's viewport is Hydra (hdEclair) on Vulkan; its Python API reaches assets,
  layers, filters and export only.
• Seated comfort rating on the panel: pending.
• Eye images render at window size (1890 × 1015 per eye on the panel); full-panel
  resolution needs F11.
• macOS: blocked, as there's no Leia runtime there.
• Material scanner capture into Sampler needs an Enterprise/Teams licence: not tested.

Notes for plug-in authors (Sampler 6.0.4)

• Call the API on the UI thread only: worker-thread calls deadlocked, and
  wait_for_computation() deadlocks on the UI thread. Step jobs as generators on a
  Qt timer, and finish exports through an ExportController subclass.
• Asset.resource doesn't change on parameter edits; hash the layer stack instead.
• Open a real project (pass the .ssa on the command line) before asset calls.
• Sampler hot-reloads a plug-in when its files change; edit with Sampler closed.

Repo: displayxr-substance-sampler (Apache-2.0). Tested with Sampler 6.0.4 and
DisplayXR runtime v2.55.0.

🤖 Generated with Claude Code

[joehillthunder]

Seated check (2026-10-27): comfortable at the default depth (100 %). The hosting-readiness check on the installed runtime follows.

🤖 Generated with Claude Code

[joehillthunder]

Readiness on the released runtime: Pass (2026-10-28)

Installed runtime v2.57.1 with the Leia SR plug-in 2.19.1, started from the default launcher (Launch-Sampler3D.cmd).

Check	Result	Conditions
Runtime and plug-in	Pass	loaded from C:\Program Files\DisplayXR\Runtime\DisplayXRClient.dll, active plug-in: id=leia-sr; installed gamepad plug-in only
"3D active" in its own window (texture target)	Pass	windowed
Depth at 100 %	Pass	front 0.14 % / behind 0.12 % of panel width (budget 0.14 % / 0.42 %)
Depth control	Pass	−/+ in 25 % steps from 0 to 200 %, plus Reset; above 100 % is past the comfort budget, and the status shows the %
3D on/off	Pass	switches the Leia SR lens to 2D and back; minimizing the window or the full panel also returns the lens to 2D
Seated check at the tested display (Leia SR)	Pass	100 % depth
Clean exit	Pass	the lens returns to 2D and the log ends