Skip to content

v0.43.0: Shap-E text-to-3D + per-family dispatch

Choose a tag to compare

View all tags
@queelius queelius released this 10 May 22:08
· 549 commits to main since this release
v0.43.0
c8c18ad

Summary

Adds ShapERuntime (text-to-3D via diffusers.ShapEPipeline) to the existing 3d/generation modality so the curated shap-e entry actually works end-to-end. Refactors the 3D HF resolver into a per-family backend_path dispatcher establishing the extension point for v0.44.0+ runtimes (Wonder3D, TRELLIS, Hunyuan3D-2).

Phase 1 of 4 for task #108. The remaining three runtimes will land in their own releases.

What works after this release

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
# Note: OpenAI SDK doesn't have a 3D endpoint, so use raw HTTP:
import requests
r = requests.post(
    "http://localhost:8000/v1/3d/generations",
    json={"model": "shap-e", "prompt": "a chair shaped like an avocado"},
)
glb_b64 = r.json()["data"][0]["b64_json"]

OR via the muse Python client:

from muse.modalities.model_3d_generation import Generation3DClient

client = Generation3DClient()
result = client.from_text(model="shap-e", prompt="a small cube")
# result.glb_bytes contains the GLB-encoded mesh

Curated entry honesty

The TRELLIS / Wonder3D / Hunyuan3D-2 curated entries have descriptions updated to call out the release where their runtime arrives:

  • wonder3d-base: runtime arrives in v0.44.0
  • trellis-text-image: runtime arrives in v0.45.0
  • hunyuan3d-2: runtime arrives in v0.46.0

These entries continue to fall through to TripoSRRuntime as a fallback (pre-v0.43.0 behavior). After their dedicated runtime releases ship, muse pull for these entries will produce working models.

Architecture additions

  • _runtime_path_for(repo_id) in model_3d_generation/hf.py: per-family extension point. v0.44.0+ adds one constant plus one dispatch branch per family.
  • _pip_extras_for(runtime_path): per-runtime pip dependency tuple.

Tests

  • 6 new dispatch tests in tests/modalities/model_3d_generation/test_hf_per_family_dispatch.py.
  • 10 new ShapERuntime unit tests in tests/modalities/model_3d_generation/runtimes/test_shape_e.py.
  • 1 new slow e2e test (skips without GPU + cached weights).
  • 5 new opt-in integration tests against a real muse server.
  • Plus a latent-bug fix to an existing test that incorrectly asserted Shap-E supported image-to-3D.

Full fast lane: 3023 passing (was 3003).

Migration / breaking changes

None.

  • TripoSR continues to work unchanged.
  • Catalog format unchanged; new manifests just have a different backend_path for Shap-E URIs.
  • Wire format / route protocol unchanged.

Out of scope

  • Wonder3D / TRELLIS / Hunyuan3D-2 runtimes (separate releases v0.44.0 to v0.46.0).
  • Shap-E radiance-field output (mesh + GLB only).
  • Shap-E image-to-3D variant (openai/shap-e-img2img, not curated).
  • Bundling Shap-E (GPU-required; bundled models must be CPU-friendly).
Assets 2
Loading