Proposal: measured materials at true size in host 3D views (capture device → USD/MDL → Omniverse + Substance 3D Modeler)

[joehillthunder]

A photometric material scanner, driven by a material-authoring app (Substance 3D Sampler), produces PBR map sets (basecolor / normal / roughness / metallic / height / opacity / AO, plus anisotropy and sheen) or .sbsar. The scanner has no 3D view of its own, so we treat it as content, the same as generated 3D (#2391). A small stdlib-only library turns each export into host content, and existing DisplayXR 3D windows show it.

Device-neutral by design. Several commercial photometric material scanners exist, and the library only needs a PBR map set (or .sbsar) plus the capture size. So any scanner whose software exports those works the same way. The tested device is one of these scanners, used through Substance 3D Sampler.

Code: private repo, commit c8c5cc2 (Apache-2.0 + NOTICE; no third-party assets bundled). Tests: 10/10.

What it does

• Real size is required. Without it, 1:1 inspection on the glass and the comfort budget mean nothing. The size comes from, in order:

  1. an explicit argument;
  2. a sidecar;
  3. pixels ÷ the scanner's capture density;
  4. an assumed capture area, flagged in the output.

• USD:

  • Y-up cm, one defaultPrim, bindings inside it, relative textures.
  • Two render contexts: UsdPreviewSurface (portable) and OmniPBR MDL (RTX).
  • DirectX-convention normals are handled in both.
  • Provenance is in customData plus a sidecar.

• Inspection shapes as a variant set: a drape (shallow folds), a flat swatch, and a 3×3 tile for checking seams, all with real-size UVs. They're shallow on purpose: the desk-display comfort band is thin, so there's no sphere. All three are predicted to fit the budget at 30 cm with no camera push-back; a panel measurement is pending.

• Live: a watched drop folder. Each capture is versioned and the live stage is swapped in atomically, so hosts reload on mtime and never see a half-written stage.

• Omniverse Kit: a thin extension that can:

  • load captures and follow live ones;
  • switch the inspection shape;
  • "frame for 3D": camera 60° down, focus on the material so it sits on the glass, through the existing Omniverse integration (Proposal: DisplayXR for NVIDIA Omniverse (Kit extension, Leia SR tested) #2331);
  • run a light sweep, so view-dependent response can be judged.

  Verified headless on Kit 110.4: the MDL binds, the 30 cm drape is at true size, and live reload works.

• Substance 3D Modeler (Proposal: DisplayXR for Adobe Substance 3D Modeler (closed legacy OpenXR app, dedicated 3D window, Leia SR tested) #2394): version 1.22 has no .sbsar or MaterialX import, so the material goes in as a textured glb/usda through Import mesh. In the glb, roughness and metalness are constants.

Target class

None of its own; see the target-class comment. It rides on the hosts' own viewport or dedicated 3D window. From the host library it needs:

• display-info reads (panel size, nominal viewer);
• the comfort budget in display info (Display info: publish the comfort budget (front/behind disparity limits), e.g. layout.comfort {front_frac, behind_frac} #2388). Until then the budget is a named constant.

Status

Tested with a synthetic material only.

Pending:

• side-by-side viewing on the panel (Kit plus Modeler), with a seated check;
• the Modeler USD → Kit round trip;
• end-to-end capture on the device, with the authoring app's real export presets.

🤖 Generated with Claude Code

[joehillthunder]

Update: more scanner formats (device-neutral in code)

The library now reads exports from several material scanners, not just the one we tested. Commit 9cda62f (private): stdlib only, 21 tests (11 new), synthetic fixtures only. No vendor SDKs, sample scans or vendor files are included.

Source	Status	Layout
Substance 3D Sampler PBR map folder	supported	assumed (Substance naming; real export presets still to check)
.sbsar	supported when sbsrender is installed	confirmed
U3M 1.1 (Vizoo's open Unified 3D Material: .u3m + textures)	supported; real size from image width/height in mm and dpi	confirmed from the open spec and JSON schema (github.com/vizoogmbh/u3m)
U3M 1.0	supported; size inferred from pixels ÷ dpi	confirmed from the spec example
U3MA (zipped U3M)	supported; paths checked, escaping paths refused	confirmed
Vizoo xTex PBR maps	supported	_BASE / _NRM / _ROUGH / _MTL / _ALPHA confirmed; _DISP / _SPEC assumed
X-Rite Pantora PBR export (maps + .axf in one folder)	supported through the generic reader; the AxF hash and its representation go into provenance	naming assumed (no public export docs found)
AxF (.axf) on its own	recognised but refused, with the way forward: export PBR from Pantora, or use the AxF → MDL route. Decoding it would need X-Rite's AxF SDK	HDF5 signature confirmed; representation (SVBRDF / CarPaint / FactorizedBTF) detected heuristically
AxF car paint	unsupported as PBR; flagged	heuristic

How it works:

• Detection: the source format is detected automatically and recorded in provenance.
• Real size, in this order: an explicit argument, then a sidecar, then the file's own size (U3M), then pixels ÷ the device's capture density, then the device's capture area (flagged as assumed). Density presets come from public datasheets for each device class.
• U3M mapping:
  • Base colour, normal (OpenGL convention, per the spec), roughness, metalness, alpha and displacement are mapped.
  • Anisotropy, sheen, specular, clearcoat, transmission and subsurface are kept and noted.
  • The spec's custom section is never read, because the spec reserves it for its owner.

Pending:

• a real export from each device (to confirm the assumed naming);
• the panel checks listed above.

🤖 Generated with Claude Code

[joehillthunder]

Update: on the panel, side by side

USD Composer (docked 3D) and Substance 3D Modeler (embedded) ran side by side with captured materials, each in its own window, on the tested display. Composer live-followed the USD + OmniPBR stage; Modeler took the .glb through Import mesh.

What we learned:

• A normal map alone gives no stereo depth. A flat swatch with a strong normal map didn't look 3D. Stereo needs real geometry.
• So the library now builds the relief: a new --relief-cm option displaces the inspection shapes by the scan's height map. A rock scan with 8 cm relief on a 30 cm swatch, seen from 35° down, looked good on the panel. Composer's lit scene read much better than an unlit viewer.
• Depth needs fitting to the content. For imported content, Modeler's hand-set rig strength was too flat (0.1) and needed 0.3. That's more evidence that depth should be fitted from content bounds plus display info (Display info: publish the comfort budget (front/behind disparity limits), e.g. layout.comfort {front_frac, behind_frac} #2388).
• Window overlap on Leia SR: the SR runtime disables the lens and stops weaving whenever another process's window overlaps the 3D area, so a terminal on top showed the flat stereo image. SR logs it per app under C:\ProgramData\Simulated Reality\Applications\<app>\Log. Integrators should keep the 3D area unobstructed, or expect a fallback to 2D.
• Two overlay sessions at once: with two apps weaving at once on the overlay embed, the second app (Kit) fell back to the unwoven stereo image, including on F11. Either app alone was fine. That fits the single shared weaver direction, so it's input for the host library rather than an overlay bug.

Code: commit 908528c. It adds:

• relief geometry from the height map;
• emissive output in the glb and in OmniPBR;
• OpenGL-convention normals preferred;
• re-framing when the size changes;
• Enter resets to the optimised view.

23 tests pass. Demo materials were CC0 scans used locally only; none are in the repo.

Pending: a seated comfort measurement at the fitted strength, the Modeler USD → Kit round trip, and real scanner exports.

🤖 Generated with Claude Code

[joehillthunder]

Update: Substance 3D Sampler export confirmed

A real Substance 3D Sampler 6.0.4 export, made from a CC0 photo, confirms the Sampler row of the format table:

• Maps: the default preset writes _BaseColor / _Normal / _Height / _Roughness / _Metallic / _AmbientOcclusion, as PNG and EXR.
• Normal convention: its _Normal is DirectX convention (measured).
• Detection: when the file name doesn't say, the converter now measures the normal convention from the height and normal maps together (green channel against the height gradient). Checked on ambientCG GL/DX pairs: 6/6 correct.

Format table status: Sampler confirmed; Pantora export naming still assumed. Commit e82a3c2 (private).

For integrators: never assume an exporter's normal-map convention. Measure it, or state it per export.

🤖 Generated with Claude Code

[joehillthunder]

Final update: lossless height maps, glTF-ready normals, status

Since the Sampler update:

• Height maps for relief, all read with the standard library:

  • PNG (8/16-bit);
  • TIFF (uncompressed / LZW / Deflate / PackBits; 8/16/32-bit and float);
  • OpenEXR scanline (NONE / RLE / ZIPS / ZIP / PIZ; half / float / uint). PIZ is a port of OpenEXR's decoder and takes about 6 s for a 2K map.

  Lossy or tiled EXR (PXR24 / B44 / DWA) is refused with "re-export ZIP or PIZ, or as 16-bit PNG/TIFF". Float maps are normalised to their own range. Commits 5ee8de7, 58ee70a.

• The glTF output flips DirectX normals to OpenGL, because glTF expects OpenGL normals.

  • A default Sampler PNG export now goes straight to mesh-only importers with correct shading. The flip is checked on a real export, measured against the height map.
  • The glb's normal map is capped at 2048 px to keep the file small; the USD keeps full resolution.
  • EXR and JPEG normals can't be flipped in the glb, so use the PNG set for mesh tools. Commit 927b878.

• Sampler's normal format is an application preference, not an export-preset option (default DirectX), and its Python export API has no switch for it. So measuring the convention from the maps is the dependable route, not a fallback.

• 41 tests pass, on synthetic fixtures only. No vendor files or scans are in the repo.

Status:

• Omniverse (Composer) is the reference host. Live reload, relief, and night / inspect lighting were seen on the panel.
• Substance 3D Modeler takes the glb or usda as reference geometry. Its Import mesh file dialog is being re-checked, because a .glb didn't appear in the dialog in one run.

Still open:

• a real export from a second scanner family (its naming is still assumed);
• the Modeler USD → Kit round trip;
• a measured seated comfort number at the fitted strength;
• two apps weaving at once (host library; see Display info: publish the comfort budget (front/behind disparity limits), e.g. layout.comfort {front_frac, behind_frac} #2388).

For integrators:

• Treat a material scan as content for the 3D views hosts already have, not as a new viewer.
• Give it real size, and real relief from the height map.
• Measure the normal convention from the maps instead of assuming it.

🤖 Generated with Claude Code

[joehillthunder]

Update: seat-tested on the panel

Status: seat tested (2026-10-28) on a Leia SR panel, using the installed DisplayXR 2.55.0 runtime (Program Files, leia-sr plug-in). Seated check: Pass.

What was run: USD Composer with the material extension, in the default desk view (35° down, 30 cm swatch with relief from the height map). The run covered:

• a rock scan with an emissive glow map;
• a live publish of a real Substance 3D Sampler export (rock wall, 5 cm relief);
• a live publish of a bark scan.

Results:

• 3D active: 3D started 37 s after launch, with the 3D window docked over the viewport.
• Weaving: Leia SR weaves in the window, and its log shows no occlusion.
• Exit: Composer closes cleanly in about 1 s, and the weaver is released.
• Live update: a new export appears in 3D under 1 s after it's published (conversion takes about 15 s).
• Framing: the whole swatch is in view at middle size, and depth stays inside the comfort budget.

Fix found during the run: a material loaded before the 3D session started was framed only in the 2D viewport, so the 3D view opened at the bridge's default distance until the user re-framed it. The extension now re-frames by itself about 1.5 s after 3D starts, so no manual step is needed. Commit b6ff063 (private). 41 tests pass.

For integrators: in Kit, frame content once the 3D session has started, not only when it loads. Check the framing with a woven atlas capture taken after that re-frame.

Open notes (not blocking):

• Night lighting: the night look wasn't rated on the panel.
• Steeper views (≥ 50°): these come out too close. The bridge's frame-to-fit uses the full view height, not the shorter docked window (Omniverse bridge, Proposal: DisplayXR for NVIDIA Omniverse (Kit extension, Leia SR tested) #2331).
• Pending: a second scanner family's real export.

🤖 Generated with Claude Code