A Home Assistant custom integration that connects to remote AG-UI Protocol compatible AI agents, enabling framework-agnostic smart home control.
AG-UI Agent provides a framework-agnostic abstraction layer that enables integration of various agentic frameworks into the Home Assistant smart home environment. The AG-UI protocol serves as a standardized event-driven streaming interface between any AG-UI compatible agent backend and Home Assistant's frontend tool execution layer.
Key Benefits:
- Framework Agnostic: Connect to any AG-UI-compatible agent backend
- Standardized Protocol: Any AG-UI-compatible agent works with the same client code
- Frontend Tool Execution: Home Assistant retains control of tool execution (security, validation)
- Remote Agent Support: Connect to agents running anywhere via SSE endpoints
- Future-Proof: As new agent frameworks emerge, they can be integrated via AG-UI adapters
Why AG-UI?
The AG-UI protocol was designed specifically for connecting AI agents to user interfaces while keeping tool execution on the frontend. This separation is critical for Home Assistant because:
- Tools need access to the HA runtime (entity states, service calls, user context)
- Security-sensitive operations require frontend validation
- Local tool execution avoids sending HA credentials to remote agents
┌─────────────────────────────────────────────────────────────────────────────┐
│ Home Assistant (Frontend) │
│ ┌───────────────────────────────────────────────────────────────────────┐ │
│ │ AGUIAgentConversationEntity │ │
│ │ • Owns Home Assistant context (entities, services, user) │ │
│ │ • Executes ALL tools locally via HA LLM API │ │
│ │ • Translates HA tools to AG-UI format │ │
│ └────────────────────────────┬──────────────────────────────────────────┘ │
│ │ │
│ ┌────────────────────────────▼──────────────────────────────────────────┐ │
│ │ AGUIClient │ │
│ │ • Framework-agnostic event processor │ │
│ │ • Handles AG-UI protocol (RUN_*, TEXT_*, TOOL_CALL_* events) │ │
│ │ • Works identically with ANY AG-UI-compatible backend │ │
│ └────────────────────────────┬──────────────────────────────────────────┘ │
└───────────────────────────────┼─────────────────────────────────────────────┘
│
┌─────────────────────┼─────────────────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────────┐ ┌─────────────────────┐ ┌─────────────────────────┐
│ LangGraph Agent │ │ CrewAI Agent │ │ Custom Agent Server │
│ │ │ │ │ │
│ ag_ui_langgraph │ │ ag_ui_crewai │ │ Your AG-UI server │
│ adapter │ │ adapter │ │ │
└─────────────────────┘ └─────────────────────┘ └─────────────────────────┘
The AGUIClient doesn't know or care which framework powers the agent. It only understands AG-UI events.
- Open HACS in Home Assistant
- Click on "Integrations"
- Click the three dots menu and select "Custom repositories"
- Add this repository URL and select "Integration" as the category
- Find "AG-UI Agent" and install it
- Restart Home Assistant
- Copy the
custom_components/agui_agentdirectory to your Home Assistantconfig/custom_componentsdirectory - Restart Home Assistant
- Go to Settings → Devices & Services → Add Integration
- Search for "AG-UI Agent"
- Enter your AG-UI endpoint URL (e.g.,
http://your-agent-server:8005/agent) - Configure timeout settings as needed
| Option | Type | Default | Description |
|---|---|---|---|
agui_endpoint |
string | required | AG-UI agent endpoint URL |
timeout |
int | 120 |
Request timeout in seconds |
AG-UI Agent works with any backend that implements the AG-UI protocol:
- LangGraph via ag-ui-langgraph
- CrewAI via AG-UI adapter
- Google Agent Development Kit (ADK) via AG-UI adapter
- Custom agents - implement the AG-UI SSE protocol
Any AG-UI-compatible agent backend must:
- Accept
RunAgentInput(messages, tools, context) via POST request - Emit events via SSE stream following the AG-UI protocol
- Request tool calls via
TOOL_CALL_*events (not execute them) - Handle tool results injected by the frontend
The AG-UI Protocol specification defines the complete event schema.
1. User: "Turn on the kitchen light"
2. AGUIClient sends RunAgentInput:
{
"thread_id": "01HXY...",
"messages": [{"role": "user", "content": "Turn on..."}],
"tools": [{"name": "HassTurnOn", ...}],
"context": {"user_id": "...", "language": "en"}
}
3. Agent emits events via SSE:
→ RUN_STARTED
→ TEXT_MESSAGE_START (messageId: "msg-1")
→ TEXT_MESSAGE_CONTENT (delta: "I'll turn on the kitchen light.")
→ TEXT_MESSAGE_END
→ TOOL_CALL_START (toolCallId: "tc-1", toolCallName: "HassTurnOn")
→ TOOL_CALL_ARGS (delta: '{"domain": ["light"], "name": "Kitchen"}')
→ TOOL_CALL_END
→ RUN_FINISHED
4. AGUIClient:
- Accumulates text: "I'll turn on the kitchen light."
- On TOOL_CALL_END, executes HassTurnOn via HA LLM API
- Returns ConversationResult with speech response
# configuration.yaml
logger:
default: warning
logs:
custom_components.agui_agent: debugDEBUG - Tool call started: HassTurnOn (tc-1)
DEBUG - Tool HassTurnOn executed: success
DEBUG - Text message ended
DEBUG - Agent run finished
| Symptom | Likely Cause | Solution |
|---|---|---|
| No response | Agent endpoint unreachable | Check URL and network connectivity |
| Timeout errors | Agent taking too long | Increase timeout setting |
| Tool not found | Tool not in HA LLM API | Verify tool is available in Home Assistant |
| Connection refused | Agent server not running | Start your AG-UI agent server |
- AG-UI Protocol: >= 0.1.10
- Home Assistant: 2025.2+
- Python: 3.13.2+
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.