Skip to content

3.20.0

Choose a tag to compare

@fernandotonon fernandotonon released this 10 Jul 21:59

Skinning v2 — ML auto-skinning, geodesic voxel binding, and dual-quaternion display

This release ships the complete Skinning v2 epic (#819): a new default ML skinner, a production-grade geodesic binding algorithm, weight post-processing, a dual-quaternion viewport display, and an evaluation suite — plus the UX and stability work that landed alongside it.

SkinTokens ML skin weights — the new default (#831)

One click (or one command) computes artist-grade skin weights with SkinTokens/TokenRig (VAST-AI, MIT), an autoregressive transformer on a Qwen3-0.6B backbone, exported to ONNX and run fully locally:

  • Default on every surface: the GUI Skinning dialog, qtmesh skin (--algo skintokens), MCP compute_skin_weights, and the rig→skin chain. Falls back to Geodesic Voxel automatically when the models aren't available (headless/CI unaffected).
  • Raw ML predictions are post-processed with a geodesic localisation pass, cutting cross-limb bleed from 0.74 to 0.05.
  • Models (~2.3 GB, five ONNX graphs) download on first use; also published standalone at QtMeshEditor-skintokens-onnx.

SkinTokens weights preview
The full AI pipeline on an unrigged OBJ: UniRig predicts the 46-bone skeleton, SkinTokens paints the weights — the heat-map shows the smooth hip falloff (6,968 verts, geodesic bleed 0.002, no template and no manual steps).

Geodesic Voxel Binding + weight post-passes (#829)

The Maya-style Geodesic Voxel bind (Dionne & de Lasa, SCA 2013), implemented natively with zero new dependencies: distances travel through the mesh interior, so a hand posed near a thigh can never steal leg weights — cross-limb bleed is impossible by construction. Works on non-watertight, self-intersecting, and multi-component meshes; planes/cloth fall back safely. Every algorithm now runs the same post-pass pipeline (Laplacian weight smoothing with manual weights as constraints, prune + renormalize, bleed reporting).

Rig → skin, end to end, all AI (#831)

Generate a mesh from a photo (TripoSR/SG) → Generate Rig (AI) (UniRig skeleton prediction) → SkinTokens weights — no manual steps. This release also fixes the chain around it:

UniRig progress

  • UniRig actually rigs now: the token detokenizer rejected real decodes on an exact-coordinate-bin parent match and always fell back to the template; it now resolves branch parents to the nearest earlier joint.
  • The ML skinner runs on a worker thread with a progress bar and Cancel — a multi-minute skin no longer freezes the UI. Rig progress shows the live token count instead of a bogus "/ 2048" cap.
  • Viewport-freeze fixes: a rigged-but-not-yet-skinned mesh was unrenderable (Ogre asserts every frame without blend data) and one render exception permanently killed the render loop. Rigs now get an instant placeholder bind that the real weights replace, and the render loop skips bad frames, surfaces the first error in the status bar, and only stops after 300 consecutive failures.
  • ONNX inference is capped at cores−1 threads with spinning disabled, so decodes can't starve rendering.

Rigging panel

Dual-quaternion skinning display (#830, fixed by #835)

Animation mode → Skinning → Display: Linear | Dual Quaternion (and MCP set_skinning_display) previews DQS deformation — the fix for linear blending's "candy-wrapper" volume collapse on twists. Display-only: exported weights are unchanged.

Linear (LBS) — necks in at the blend zone Dual Quaternion — full cross-section
LBS DQS

Two deep bugs fixed on the way: the hardware-skinning factory teardown double-freed Ogre's built-in and crashed test teardowns (#830), and Ogre's cached per-entity hardware-animation decision made toggled entities vanish entirely — the toggle now busts that cache (#833/#835), with a pixel-level regression test.

Skinning evaluation suite (#830, #834)

  • qtmesh skin --evaluate — influence histogram, Laplacian smoothness, geodesic bleed on existing weights.
  • qtmesh skin ours.fbx --compare reference.fbx — per-vertex weight diff against a reference-skinned copy (position-matched vertices, name-matched bones).
  • Hermetic CI fixtures assert the quality bar (90° elbow volume ≥ 0.9, zero bleed on proximity fixtures); the protocol, a recorded reference run, and the LBS/DQS comparison live in docs/SKINNING_QUALITY.md.

Hugging Face: dedicated repos for every converted model (#832)

Every AI model the editor uses now has a standalone HF repo with a proper model card, license tag, and I/O contract — the aggregate QtMeshEditor-models repo remains what the app downloads from: skintokens · unirig · triposr · triposg · pbrify · realesrgan · u2net · smolvlm — plus the in-house rmib-inbetween, mesh-segmentation, and t2m.

Merged PRs

  • #829 — Skinning v2 Slices A+B: Geodesic Voxel Binding (new default at the time) + weight post-pass pipeline
  • #830 — Slices C/D/E: DQS display toggle, evaluation suite, UniRig-skin-head decision record
  • #831 — SkinTokens ML skin-weight prediction (now the default) + async skinning, rig placeholder bind, render-loop resilience, UniRig detokenizer fix
  • #832 — dedicated HF mirror repos + sync script
  • #834 — skinning-quality docs: recorded reference run + forearm-twist captures
  • #835 — DQS hardware-animation cache fix (#833)