Home Assistant Natural Language Admin Agent

A powerful tool (CLI + Web Interface) that allows you to administer your Home Assistant instance using natural language via LLM (OpenAI GPT or Ollama). Create automations, manage devices, and control your smart home with simple conversational commands.

Features

Core Capabilities

• 🤖 Natural Language Interface: Control everything with conversational commands
• 🎨 Beautiful Web UI: Modern, responsive interface with real-time updates
• 💬 CLI Tool: Terminal-based interface for power users
• 🔐 Secure: Token-based authentication with Home Assistant

Home Assistant Management

• 📊 Query configuration, entities, states, and statistics
• 💡 Control devices and services (lights, switches, climate, etc.)
• 🏠 Manage areas, entities, and device registry
• 🤖 Create/Edit/Delete Automations via natural language
• 📁 File-based Config Management via SSH (optional)
• 🔄 Reload configurations and automations

AI & LLM

• Support for both OpenAI (GPT-4, GPT-3.5) and local Ollama models
• Function calling for precise Home Assistant operations
• Interactive conversation with context retention
• Safe execution with LLM validation

Prerequisites

• Python 3.11+
• Home Assistant instance (running at 10.1.10.20 or your IP)
• Either:
  • OpenAI API key (for GPT-4/GPT-3.5)
  • Ollama installed locally (for local LLM)

Setup

1. Create a Home Assistant Long-Lived Access Token

1. Log into your Home Assistant instance at http://10.1.10.20:8123
2. Go to your Profile (click your username in the bottom left)
3. Scroll down to "Long-Lived Access Tokens"
4. Click "Create Token"
5. Give it a name (e.g., "HA Admin Agent")
6. Copy the generated token

2. Install Dependencies

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

3. Configure Environment

Copy the example environment file and add your credentials:

cp .env.example .env

Edit .env file:

For OpenAI (GPT-4):

HA_URL=http://10.1.10.20:8123
HA_TOKEN=your_home_assistant_token_here

OPENAI_API_KEY=sk-your-openai-api-key-here
LLM_PROVIDER=openai
LLM_MODEL=gpt-4-turbo-preview

For Ollama (local LLM):

HA_URL=http://10.1.10.20:8123
HA_TOKEN=your_home_assistant_token_here

LLM_PROVIDER=ollama
OLLAMA_URL=http://localhost:11434
LLM_MODEL=llama3.1:8b

4. (Optional) Setup Ollama

If using Ollama for local LLM:

# Install Ollama from https://ollama.ai
# Then pull a model:
ollama pull llama3.1:8b

Usage

Web Interface (Recommended)

Start the web server:

python web_server.py
# Or with the virtual environment:
venv/bin/python web_server.py

Then open your browser to: http://localhost:5000

The web interface provides:

• Real-time chat with the AI agent
• Dashboard with system stats (entities, areas, automations)
• Beautiful, responsive design
• WebSocket-based instant responses
• Example commands to get started

CLI Interface

For terminal-based interaction:

python main.py
# Or with the virtual environment:
venv/bin/python main.py

Example Commands

Natural language examples you can try:

Device Control

"Show me all the lights in my home"
"Turn on the living room light"
"Turn off all lights in the bedroom"
"Set the thermostat to 72 degrees"
"What's the current temperature in the bedroom?"

Areas & Entities

"Create a new area called 'Home Office'"
"List all areas in my home"
"What entities are in the kitchen?"
"Rename the 'light.lamp_1' entity to 'Desk Lamp'"
"Move the bedroom lamp to the Living Room area"

Automations

"Create an automation to turn on the porch light at sunset"
"Show me all my automations"
"Create an automation that turns off all lights at 11 PM"
"Disable the 'Good Morning' automation"
"What automations are currently running?"

System Info

"Show me the Home Assistant configuration"
"What version of Home Assistant am I running?"
"How many entities do I have?"
"List all available services"

Special Commands

• reset - Clear conversation history
• exit or quit - Exit the agent

Available Agent Tools

The AI agent has access to these Home Assistant operations:

• Configuration: Get config, check config validity, reload
• States: Get all entities, get/set entity states
• Services: Call any Home Assistant service
• Areas: List, create, update, delete areas
• Devices: List all devices
• Entity Registry: List and update entity settings
• Automations: List, create, update, delete, enable/disable, trigger
• File Management: Read/write config files via SSH (optional)
• System: Reload config, check config, restart (use with caution)

Architecture

web_server.py    # Flask web server with WebSocket support
main.py          # CLI interface
agent.py         # Agent orchestration with tool calling (25+ tools)
config.py        # Configuration management (.env)
ha_client.py     # Home Assistant REST API client
llm_client.py    # OpenAI/Ollama LLM client wrapper
file_manager.py  # SSH/SFTP for config file management
templates/       # Web UI HTML templates
static/          # JavaScript and CSS assets

How It Works

1. User Input: Natural language command via web or CLI
2. LLM Processing: Command is sent to OpenAI/Ollama with available tools
3. Function Calling: LLM decides which Home Assistant operations to execute
4. Execution: Agent calls the appropriate HA API endpoints or file operations
5. Response: Results are formatted and returned to the user

Security Notes

• Keep your .env file secure and never commit it to version control
• The Home Assistant token has full access to your instance
• Review destructive operations before confirming
• Consider using a dedicated HA user/token with limited permissions

Troubleshooting

Connection refused to Home Assistant:

• Verify HA is running at the configured IP
• Check firewall rules allow connections from your Mac

OpenAI API errors:

• Verify your API key is valid
• Check your OpenAI account has credits

Ollama errors:

• Ensure Ollama is running: ollama serve
• Verify the model is pulled: ollama list

Tool/Function calling not working:

• Make sure you're using a model that supports function calling (GPT-4, GPT-3.5-turbo, or compatible Ollama models)

Contributing

Feel free to extend the agent with additional tools and capabilities!