AI-powered development tools for NVIDIA Omniverse, including Chat USD and Model Context Protocol (MCP) servers for Kit, USD, OmniUI, and Isaac Sim development.
This repository provides two types of AI development tools:
Chat USD is an AI assistant that runs inside NVIDIA Omniverse Kit, enabling natural language interaction with USD scenes.
Capabilities:
- USD Code Generation: Generate and execute USD Python code from natural language
- Asset Search: Find USD assets using conversational queries
- Scene Analysis: Get information about your current USD scene
- Real-time Editing: Modify scenes interactively through conversation
- Custom Agents: Extensible architecture for adding specialized capabilities
Four standalone Model Context Protocol servers that integrate with AI coding assistants like Claude, Cursor, and other MCP-compatible tools.
Provides USD/OpenUSD API assistance with 7 specialized tools:
- Browse USD modules and classes
- Get detailed class, module, and method API documentation
- Search code examples with semantic search
- Search USD knowledge base
Comprehensive Kit development assistance with 12 specialized tools:
- Search 400+ Kit extensions
- Explore extension APIs and dependencies
- Find code examples and test patterns
- Access Kit documentation and settings
OmniUI development assistance with 10 specialized tools:
- Browse UI classes, modules, and methods
- Get widget documentation, examples, and instructions
- Search code examples and window-pattern examples
- Look up styling docs and scene-system APIs
Isaac Sim development assistance with 5 specialized tools:
- Search Isaac Sim extensions and APIs
- Find code examples with semantic search
- Access Isaac Sim documentation and best practices
- Search Isaac Sim configuration settings
Build and run in Omniverse Kit:
# Windows
build.bat -r
_build\windows-x86_64\release\omni.app.chat_usd.bat
# Linux
./build.sh -r
./_build/linux-x86_64/release/omni.app.chat_usd.shEach MCP server can be run locally with Docker or Python. See the individual server documentation:
Quick Start (Docker - Recommended):
# Set your NVIDIA API key first
export NVIDIA_API_KEY=your_api_key_here # Linux/macOS
# or
set NVIDIA_API_KEY=your_api_key_here # Windows
# Build wheels and start all MCP servers
cd source/mcp
./build-wheels.sh all # or build-wheels.bat all on Windows
docker compose -f docker-compose.ngc.yaml up --buildThis uses NVIDIA's cloud-hosted embedder and reranker services via your API key. No local GPUs required.
Local GPU Deployment (Advanced):
For running embedder/reranker models locally on your own GPUs (requires 2 NVIDIA GPUs):
export NGC_API_KEY=your_ngc_api_key # For pulling NIM container images
export NVIDIA_API_KEY=your_nvidia_api_key
cd source/mcp
./build-wheels.sh all
docker compose -f docker-compose.local.yaml up --buildSee Local Deployment Guide for details.
Requirements:
- NVIDIA API Key (for embeddings, reranking, and LLM access)
- Docker (recommended for deployment)
This project is currently not accepting contributions.