Skip to content

Choose a tag to compare

@fernandotonon fernandotonon released this 28 Jun 05:20
dfb238b

AI Mesh Part Segmentation (#410)

Predicts a semantic part label (head / torso / left+right arm / left+right leg) per vertex and per face — the fourth ONNX consumer in the AI-assist epic.

Highlights

  • Edit Mode → "Select by Part (AI)" — select a whole body part with one click. Worker-threaded with a progress bar (incl. first-use model download) and cancel.
  • Three-tier dispatch: (1) rig-prior — for skinned meshes, label each vertex by the part of the bone it's most-weighted to (exact, and handles non-human anatomy: ears/snout→head, tail→torso, paws→leg); (2) ONNX PointNet++ model for unrigged meshes; (3) deterministic geometric fallback (always available, no model/ONNX needed).
  • CLI: qtmesh segment <file> [--json] [--no-model] [--up-axis x|y|z] — per-part vertex/face counts or full label arrays.
  • MCP: segment_mesh tool — returns the full label map (per-part counts + per-face label array) as JSON, with an up_axis arg.
  • Up-axis correctness: X/Z-up meshes are remapped to the model's +Y-up training frame before inference (and in the nearest-point scatter), so they aren't mislabelled.

Continual-training loop

  • qtmesh segment <mesh> --dump-training-data out.json mines exact per-vertex labels from any rigged mesh's bone weights — every rigged asset becomes a free, perfectly-labelled training sample.
  • scripts/mine-training-data.sh + scripts/fetch-training-rigs.sh assemble + mine a CC0/CC-BY corpus; export-meshseg-onnx.py --real-data <dir> mixes it into the synthetic training set.

Model: trained on synthetic + CC0-mined permissively-licensed data (ShapeNet-Part / PartNet / Mixamo excluded — they don't clear the redistribution bar). Hosted on Hugging Face, downloads on first use.

Preview

Captura de Tela 2026-06-28 às 02 57 27