Skip to content

jaguarcode/UnrealMasterAI

BranchesTags

Repository files navigation

Unreal Master Agent

License: MIT Node.js Unreal Engine Tests MCP Tools LLM Hosts

An autonomous AI agent that gives Claude Code bidirectional control over Unreal Engine internals — manipulating Blueprints at the graph level, generating Slate UI code, triggering Live Coding compilation, and self-healing from compile errors without manual intervention.

What This Does

You describe what you want in natural language. Claude Code calls the right tools in sequence. Unreal Engine executes the changes in real time.

Example:

"Add a PrintString node to BP_TestActor connected after BeginPlay, set the message to 'Hello World', then compile."

The agent serializes the Blueprint to JSON, creates the node, connects the exec pins, sets the default value, and triggers Live Coding — reporting any compile errors back for self-healing if needed.

Core capabilities:

  • Blueprint graph serialization (UEdGraph → structured JSON AST)
  • Dynamic node creation and pin connection via C++ UE APIs
  • Slate UI code generation with RAG-assisted template retrieval
  • Live Coding compilation trigger and compile log capture
  • Self-healing loop: parse compile errors → apply fix → retry (max 3 iterations)
  • Human-in-the-loop approval gate for destructive operations
  • Actor spawning, deletion, property editing, transform control
  • Material creation, parameter setting (scalar/vector), texture assignment
  • Level management (create, open, save, sublevels, world settings)
  • Asset pipeline (import, export, create, duplicate, rename, delete, references)
  • Animation tools (montages, blend spaces, skeleton info)
  • Mesh operations (LOD, materials, collision generation)
  • DataTable CRUD operations
  • Project introspection (structure, plugins, settings, class hierarchy, dependency graph)
  • Build pipeline (lightmaps, content cooking, map check)
  • Source control integration (status, checkout, diff)
  • Gameplay systems (input actions, game mode)
  • Python script execution bridge (166 scripts) for extensible UE automation
  • Sequencer/cinematics, AI/navigation, widget/UMG tools
  • Texture, Niagara VFX, audio, landscape pipelines
  • Physics, world partition, foliage, curves, PCG, geometry script tools
  • Workflow templates (character, UI, level, multiplayer, inventory, dialogue)
  • Analysis tools (Blueprint complexity, asset health, performance, conventions)
  • Context intelligence (auto-gather project context, tool manifest, workflow chains)
  • Workflow learning system (learn from Epic docs, intent matching with UE synonym expansion)
  • Error resolution learning (capture troubleshooting outcomes, replay fixes for similar errors)
  • Outcome-weighted recommendations (proven workflows rank higher automatically)
  • 89 workflow patterns (21 builtin from Epic docs + 68 community-seeded), 25 error resolution patterns
  • Proactive tool recommendations based on workflow step adjacency analysis
  • Analytics dashboard for workflow coverage, tool usage, and error resolution metrics

Multi-LLM Support:

Works with Claude Code, Claude Desktop, Cursor, Windsurf, and VS Code + GitHub Copilot — any MCP-compatible AI editor.

Extensibility:

  • Custom MCP tools: drop .ts/.js files in mcp-server/custom-tools/ for automatic discovery and registration
  • Tool hooks: register pre/post execution callbacks via ToolHookManager for logging, validation, or transformation
  • Custom Python scripts: place scripts in Content/Python/uma_custom/ and invoke them with python-customExecute
  • Auto-registration architecture: each domain exposes a ToolModule via its own index.ts — adding a new domain requires no changes to server.ts
  • Workflow Marketplace: standardized JSON sharing format, context-exportWorkflow/context-importWorkflow tools, CLI import from file or URL, 10 community workflow templates

Architecture Overview

The system uses a 4-layer architecture with clean separation between reasoning, bridging, execution, and engine APIs.

Claude Code (Layer 1)
    │ stdio / JSON-RPC
MCP Bridge Server — Node.js/TypeScript (Layer 2)
    │ WebSocket
UE Agent Plugin — C++ (Layer 3)
    │ Direct C++ API calls
Engine APIs: UEdGraph, Slate, ILiveCodingModule (Layer 4)

Key design decisions:

  • UE is the WebSocket client — Node.js listens, UE connects. This uses UE's stable FWebSocketsModule client rather than a nonexistent server API.
  • All UE operations run on the GameThread — WebSocket callbacks dispatch via AsyncTask(ENamedThreads::GameThread).
  • TryCreateConnection always, never MakeLinkTo — ensures polymorphic pin type propagation.
  • Self-healing is Claude's responsibility — the server provides atomic tools; Claude orchestrates retry logic.

See ARCHITECTURE.md for the full architecture document including data flow diagrams, threading model, safety architecture, and all ADRs.

Prerequisites

  • Node.js 20+
  • npm 10+
  • TypeScript 5.5+ (installed via devDependencies)
  • Unreal Engine 5.4 - 5.7
  • Python Editor Script Plugin enabled in UE (Edit → Plugins → Scripting) — required for Python automation (166 scripts)
  • Claude Code (latest) with MCP support

Quick Start

1. Install dependencies

cd mcp-server
npm install

2. Configure for your AI editor

npx unreal-master-mcp-server init --host=claude      # Claude Desktop (default)
npx unreal-master-mcp-server init --host=cursor      # Cursor
npx unreal-master-mcp-server init --host=windsurf    # Windsurf
npx unreal-master-mcp-server init --host=vscode      # VS Code + Copilot
npx unreal-master-mcp-server init --host=claude-code # Claude Code CLI

Note: Only one AI host can connect to a UE instance at a time. Each host spawns its own MCP server process which binds WebSocket port 9877. Close the current host before switching to another. See the Integration Guides for details.

Or manually add to .claude/mcp.json (project root):

{
  "mcpServers": {
    "unreal-master": {
      "command": "node",
      "args": ["mcp-server/dist/index.js"],
      "env": {
        "UE_WS_PORT": "9877"
      }
    }
  }
}

Or, if installed globally via npm:

{
  "mcpServers": {
    "unreal-master": {
      "command": "npx",
      "args": ["-y", "unreal-master-mcp-server"],
      "env": {
        "UE_WS_PORT": "9877"
      }
    }
  }
}

3. Build the MCP server

cd mcp-server
npm run build

4. Enable the UE Plugin

Quick install (copies the plugin automatically):

./scripts/install-plugin.sh /path/to/YourProject

Manual install:

Copy or symlink UnrealMasterAgent/ into your Unreal Engine project's Plugins/ directory, then enable it in the .uproject file:

{
  "Plugins": [{ "Name": "UnrealMasterAgent", "Enabled": true }]
}

Rebuild the project from source.

5. Enable Python Editor Script Plugin

The Python automation layer (166 scripts) requires the Python Editor Script Plugin:

  1. Open your UE project → Edit → Plugins
  2. Search for "Python Editor Script Plugin" under Scripting
  3. Enable it and restart the editor

Without this plugin, all non-Python tools (Blueprints, actors, materials, etc.) still work. Only Python-based automation will be unavailable.

See docs/setup-guide.md for verification steps and troubleshooting.

6. Start the agent

Open your Unreal Engine project (the plugin auto-connects to the MCP server on startup), then launch Claude Code in the project root. The MCP server starts automatically.

Verify the connection:

> editor.ping

Expected response: { "status": "ok", "ueVersion": "5.4.x" }

Development mode (watch + auto-rebuild)

cd mcp-server
npm run dev

Project Structure

Unreal Master/
├── ARCHITECTURE.md          Architecture decisions and system design
├── README.md                This file
├── AGENTS.md                AI agent guidance for this codebase
├── package.json             Workspace root
│
├── mcp-server/              Layer 2: Node.js/TypeScript MCP bridge
│   ├── src/
│   │   ├── index.ts         Entry point (McpServerBootstrap)
│   │   ├── server.ts        McpServer configuration (auto-registers tools)
│   │   ├── tools/           188 MCP tool handlers across 37 domains
│   │   │   ├── editor/      Editor queries (ping, list-actors, etc.)
│   │   │   ├── blueprint/   Blueprint graph manipulation
│   │   │   ├── compilation/ Live Coding trigger and status
│   │   │   ├── file/        File read/write/search
│   │   │   ├── slate/       Slate UI generation
│   │   │   ├── chat/        In-editor chat
│   │   │   ├── actor/       Actor spawn, delete, properties, transform
│   │   │   ├── material/    Material create, parameters, textures
│   │   │   ├── mesh/        Mesh info, LOD, materials, collision
│   │   │   ├── level/       Level create, open, save, sublevels
│   │   │   ├── asset/       Asset import, export, create, duplicate
│   │   │   ├── animation/   Montages, blend spaces, skeleton info
│   │   │   ├── content/     Asset listing, search, details, validation
│   │   │   ├── datatable/   DataTable CRUD operations
│   │   │   ├── build/       Lightmaps, content cooking, map check
│   │   │   ├── project/     Project structure, plugins, settings
│   │   │   ├── gameplay/    Input actions, game mode
│   │   │   ├── python/      Python script execution bridge
│   │   │   ├── sourcecontrol/ Source control status, checkout, diff
│   │   │   ├── debug/       Console commands, logs, performance
│   │   │   ├── sequencer/   Cinematics sequences, tracks, keyframes
│   │   │   ├── ai/          Behavior trees, blackboards, nav mesh, EQS
│   │   │   ├── widget/      UMG widget creation and editing
│   │   │   ├── texture/     Texture import, compression, render targets
│   │   │   ├── niagara/     Niagara VFX systems, emitters, parameters
│   │   │   ├── audio/       Audio import, sound cues, MetaSound
│   │   │   ├── landscape/   Landscape creation, heightmaps, materials
│   │   │   ├── physics/     Physics assets, profiles, constraints
│   │   │   ├── worldpartition/ World partition, data layers, HLOD
│   │   │   ├── foliage/     Foliage types, density, properties
│   │   │   ├── curve/       Curve creation, keyframes
│   │   │   ├── pcg/         PCG graphs, nodes, connections
│   │   │   ├── geoscript/   Geometry script operations
│   │   │   ├── workflow/    Workflow templates (character, UI, level)
│   │   │   ├── analyze/     Blueprint complexity, asset health, perf
│   │   │   ├── refactor/    Rename chain with reference updates
│   │   │   └── context/     Auto-gather, tool manifest, chains, workflow learning, error learning
│   │   ├── transport/       WebSocket bridge and codec
│   │   ├── state/           Cache store and safety gate
│   │   └── observability/   Tracing (LangSmith/Langfuse)
│   ├── package.json
│   ├── tsconfig.json
│   ├── vitest.config.ts
│   ├── tests/             Test suites
│   └── docs/              Development guides
│
├── UnrealMasterAgent/               Layer 3: C++ UE plugin
│   ├── Content/Python/uma/  Python scripts for UE automation (166 scripts)
│   └── Source/
│       ├── UnrealMasterAgent/        Main module
│       │   ├── Safety/               UMAApprovalGate
│       │   ├── FileOps/              UMAFileOperations
│       │   ├── Editor/               UMAEditorSubsystem, SUMAChatPanel
│       │   ├── Blueprint/            Serializer, Manipulator
│       │   └── Python/               Python script execution bridge
│       └── UnrealMasterAgentTests/   Automation test module
│
├── TestProject/             UE5 test project for development
│   ├── Source/UMATestProject/  C++ gameplay classes (PatrollingActor, etc.)
│   └── Plugins/UnrealMasterAgent/  Symlinked/copied plugin with Python scripts
│
└── docs/
    ├── api-reference/
    │   └── mcp-tools.md            Complete MCP tool API reference
    ├── coding-conventions/
    │   └── README.md               TypeScript + C++ coding conventions
    ├── slate-templates/            Slate UI RAG templates (7 templates)
    ├── setup-guide.md              Installation and configuration guide
    ├── websocket-protocol.md       WebSocket protocol specification
    ├── safety-architecture.md      Safety system and approval gate docs
    └── AGENTS.md                   AI agent guidance for docs

Development Workflow

Two-Track Development

Work proceeds on two parallel tracks that integrate at milestones:

Track Focus Primary Language
Track A MCP Bridge Server TypeScript
Track B UE Agent Plugin C++

Both tracks use TDD from the start.

MCP Server Development

# Run tests
cd mcp-server && npm test

# Run tests in watch mode
cd mcp-server && npm run test:watch

# Type check
cd mcp-server && npm run typecheck

Test files live at tests/unit/**/*.test.ts and tests/integration/**/*.test.ts.

UE Plugin Testing

Run automation tests headlessly:

UnrealEditor-Cmd YourProject.uproject \
  -ExecCmds="Automation RunTests UnrealMaster" \
  -unattended -nopause -nullrhi

stdout / stderr Discipline

The MCP Bridge Server communicates with Claude Code over stdout using JSON-RPC. Never use console.log() in the server — it will corrupt the stream. All debug output must go to stderr via process.stderr.write() or a logger configured to use stderr.

Technology Stack

Component Technology Version
LLM Host Claude Code (Sonnet / Opus) Latest
MCP SDK @modelcontextprotocol/sdk ^1.12.0
Bridge Server Node.js + TypeScript Node 20+, TS 5.5+
WebSocket ws ^8.18.0
Validation zod ^3.23.0
Test Runner vitest ^2.0.0
UE Plugin C++ UE 5.4
Transport WebSocket RFC 6455
Observability LangSmith / Langfuse Latest

Contributing

  1. All code changes must start with a failing test (TDD).
  2. TryCreateConnection only — never MakeLinkTo in Blueprint manipulation code.
  3. Never write to stdout in the MCP server — use stderr for all logs.
  4. Every destructive operation must pass through SafetyGate.
  5. Run npm run typecheck && npm test before committing.
  6. Follow the two-track (MCP server / UE plugin) branch strategy.

License

MIT License — see LICENSE file for details.

For the full architecture specification, threading model, data flow diagrams, and all Architecture Decision Records, see ARCHITECTURE.md.

Troubleshooting

Connection Issues

Symptom Cause Fix
WebSocket connection failed UE plugin not running or wrong port Verify UE Editor is open with the plugin enabled. Check UE_WS_PORT matches in both MCP config and UE plugin settings (default: 9877).
editor-ping returns no response Plugin not connected Restart UE Editor. Check Output Log for "UnrealMasterAgent: WebSocket connected" message.
Tool not found MCP server not started Ensure .claude/mcp.json is configured and Claude Code restarted.
Python script errors PythonScriptPlugin not enabled Enable "Python Editor Script Plugin" in Edit → Plugins → Scripting. Restart the editor.
EADDRINUSE error Port already in use Another instance is running on port 9877. Kill it or change UE_WS_PORT to a different port.

Build Issues

Symptom Cause Fix
tsc compilation errors Missing dependencies Run cd mcp-server && npm install
Plugin compile errors Missing UE dependencies Ensure PythonScriptPlugin is enabled in your .uproject

Disclaimer

Unreal Engine is a trademark of Epic Games, Inc. This project is not affiliated with or endorsed by Epic Games.

About

Unreal Master AI agent that gives Claude Code bidirectional control over Unreal Engine internals — manipulating Blueprints at the graph level, generating Slate UI code, triggering Live Coding compilation, and self-healing from compile errors without manual intervention.

Topics

unrealengine ai-agents claude-code

Resources

Readme

License

MIT license

Code of conduct

Code of conduct

Contributing

Contributing

Security policy

Security policy
Activity

Stars

3 stars

Watchers

0 watching

Forks

2 forks
Report repository

Releases 4

v0.5.0 Latest
Mar 12, 2026
+ 3 releases

Packages

 
 
 

Contributors

Languages