Skip to content

[Task] text-to-3d model support #292

@DingmaomaoBJTU

Description

@DingmaomaoBJTU

Overview

Text-to-3D models generate 3D geometry directly from a natural language description. TRELLIS supports text conditioning via the same SLAT pipeline as its image variant; shap-e (OpenAI) generates implicit neural representations from text; BrickGPT specializes in generating LEGO-like brick structures from text prompts.

Agent Scenarios

  • Creative content agent: generate 3D scene elements (characters, props, environments) from a designer's text description for prototyping
  • Game / metaverse asset agent: instantly produce game-ready geometry from narrative prompts without a 3D artist in the loop
  • Education / toy design agent: use BrickGPT to generate brick-by-brick building instructions from a text description of a structure
  • Concept visualization agent: turn a product brief into a rough 3D mockup for early-stage design review

ModelKit Integration

Models must pass the full wmk pipeline on all EPs:

wmk config → wmk build (ONNX export) → wmk perf → wmk eval

Acceptance Criteria

  • microsoft/TRELLIS-text-xlarge
  • microsoft/TRELLIS-text-large
  • openai/shap-e
  • AvaLovelace/BrickGPT

Metadata

Metadata

Assignees

No one assigned

    Labels

    No fields configured for Feature.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions