Skip to content

3.18.0 — TripoSG image-to-3D + AI texturing

Choose a tag to compare

View all tags
@fernandotonon fernandotonon released this 07 Jul 22:57
3.18.0
7765d5f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Image → 3D: TripoSG backend + AI texturing

This release adds TripoSG as a second, higher-fidelity image-to-3D geometry backend alongside TripoSR, plus a describe-then-generate AI texturing path for the geometry it produces.

TripoSG geometry (new backend)

  • TripoSG (VAST-AI-Research, MIT code + MIT weights) — a 1.5B rectified-flow DiT over an SDF VAE, giving noticeably higher-fidelity geometry than the TripoSR fast path. Runs entirely locally via ONNX Runtime (no torch), models download on first use.
  • Selectable next to TripoSR on the CLI, MCP, and GUI. --flow-steps and --guidance knobs; fp32 tier.
  • Generated meshes are scaled to a workable size and oriented to face the camera.

AI texturing (describe-then-generate)

  • A local vision model (SmolVLM, Apache-2.0) captions the input image in the background the moment it's selected — shown live under the thumbnail.
  • The caption drives a multi-view depth-ControlNet bake (front + back), projected onto the mesh's UVs, with optional PBR (normal + roughness) map synthesis and Real-ESRGAN upscaling. Requires the optional Stable Diffusion build + a loaded model.
  • No SD model → the mesh ships with a clean neutral material.

Robustness

  • Fixed a long-standing bug where meshes over 65 535 vertices tore apart on import/export (16-bit → 32-bit index selection). Affects any large imported asset, not just generated ones.
  • Numerous image-to-3D stability fixes: memory footprint, UV-unwrap speed (decimate-to-budget + single-threaded xatlas), correct normals/orientation, and per-step progress feedback.

Surfaces

  • CLI: qtmesh generate3d image.png --backend triposg [--flow-steps 25] [--guidance 7] -o out.glb
  • MCP: generate_mesh_from_image with backend / flow_steps / guidance
  • GUI: Object Mode → AI: Image → 3D — backend + model pickers, per-stage checkboxes, live progress, background caption

License

Everything shipping is permissive and safe for commercial use + redistribution (TripoSG MIT, DINOv2 / SmolVLM / U²-Net Apache-2.0, xatlas / meshoptimizer MIT). No non-commercial or field-of-use-restricted components.

Full details: PR #794.

Assets 13
Loading