Home

🏠 GPT Home Developer Documentation

Welcome to the GPT Home developer wiki! This documentation provides in-depth technical details for developers who want to understand, extend, or contribute to the project.

📚 Documentation Index

Section	Description
🏗️ Architecture	System design, component interactions, and data flow
🤖 Agent System	LangGraph agent, configuration, and state management
🔧 Tools Reference	All available tools: weather, Spotify, lights, calendar, alarms
🧠 Memory System	LangMem integration, semantic/episodic memory, PostgreSQL storage
🌐 API Reference	FastAPI endpoints, SSE streaming, and REST API
🔌 Hardware Setup	Raspberry Pi configuration, I2C display, audio setup
⚙️ Configuration	Environment variables, settings.json, LiteLLM providers
💻 Development Guide	Local development, Docker builds, contributing

---

🎯 Quick Overview

GPT Home is a smart home voice assistant built on the Raspberry Pi using AI models via LiteLLM. The system uses:

• LangGraph for agent orchestration and workflow management
• LangMem for persistent long-term memory
• PostgreSQL + pgvector for semantic search and storage
• FastAPI for the REST API and web interface
• React frontend for settings and monitoring

┌─────────────────────────────────────────────────────────────────────┐
│                         GPT Home System                             │
├─────────────────────────────────────────────────────────────────────┤
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────────────────┐  │
│  │   Voice     │    │   Agent     │    │      Integrations       │  │
│  │   Input     │──▶│  (LangGraph)│───▶│  Spotify │ Hue │ CalDAV │  │
│  │ (Mic/STT)   │    │             │    └─────────────────────────┘  │
│  └─────────────┘    └──────┬──────┘                                 │
│                            │                                        │
│                     ┌──────▼──────┐                                 │
│                     │   Memory    │                                 │
│                     │  (LangMem)  │                                 │
│                     └──────┬──────┘                                 │
│                            │                                        │
│                     ┌──────▼──────┐                                 │
│                     │ PostgreSQL  │                                 │
│                     │  + pgvector │                                 │
│                     └─────────────┘                                 │
└─────────────────────────────────────────────────────────────────────┘

🗂️ Project Structure

gpt-home/
├── src/
│   ├── app.py              # Main application entry point
│   ├── common.py           # Shared utilities (display, TTS, speech recognition)
│   ├── backend.py          # FastAPI application and routes
│   ├── routes.py           # Action router (main agent interface)
│   ├── actions.py          # Legacy action functions (weather, Spotify, etc.)
│   ├── audio_capture.py    # Real-time audio capture for waveform visualization
│   ├── audio_activity.py   # Unified audio activity detection (VAD, WebRTC)
│   ├── settings.json       # Local settings fallback (primary: PostgreSQL)
│   ├── agent/              # LangGraph agent implementation
│   │   ├── core.py         # GPTHomeAgent class
│   │   ├── config.py       # AgentConfig with builder pattern
│   │   └── state.py        # Agent state schema
│   ├── display/            # Display management (HDMI/SPI/I2C)
│   │   ├── base.py         # BaseDisplay class and DisplayMode enum
│   │   ├── detection.py    # Auto-detect connected displays
│   │   ├── factory.py      # Display factory pattern
│   │   ├── manager.py      # DisplayManager singleton
│   │   ├── animations.py   # Animation utilities
│   │   ├── integration.py  # Tool context parsing for display
│   │   └── drivers/        # Display driver implementations
│   │       ├── kmsdrm.py   # KMS/DRM driver (full displays)
│   │       └── i2c.py      # I2C text display
│   ├── memory/             # Memory management
│   │   ├── manager.py      # MemoryManager facade
│   │   └── store.py        # Memory store factory
│   ├── tools/              # LangChain tools
│   │   ├── registry.py     # Tool registration system
│   │   ├── weather.py      # Weather tool (OpenWeather/Open-Meteo)
│   │   ├── spotify.py      # Spotify control
│   │   ├── lights.py       # Philips Hue control
│   │   ├── calendar.py     # CalDAV calendar/tasks
│   │   └── alarm.py        # Alarms and reminders
│   ├── waveform/           # Waveform visualization (Mediator/Observer pattern)
│   │   ├── mediator.py     # WaveformMediator singleton
│   │   ├── strategies.py   # Rendering strategies (voice-gated, always-on)
│   │   └── observers.py    # Display observers
│   └── frontend/           # React web interface
├── docker-compose.yml      # Service orchestration
├── Dockerfile              # Container build
└── .env                    # Environment configuration

🔄 Request Flow

User speaks "Computer, what's the weather?"
          │
          ▼
    ┌───────────┐
    │  listen() │  Speech Recognition (SpeechRecognition)
    └─────┬─────┘
          │
          ▼
    ┌───────────────┐
    │ action_router │  Routes to LangGraph agent
    └───────┬───────┘
            │
            ▼
    ┌───────────────────┐
    │  GPTHomeAgent     │
    │  ├─ Search Memory │  Check relevant context
    │  ├─ Select Tool   │  weather_tool selected
    │  └─ Execute       │  Fetch weather data
    └─────────┬─────────┘
              │
              ▼
    ┌─────────────────┐
    │ Background Task │  Extract & store new memories
    └─────────────────┘
              │
              ▼
    ┌───────────┐
    │  speak()  │  Text-to-Speech output
    └───────────┘

🚀 Getting Started for Developers

1. Clone the repository

   git clone https://github.com/judahpaul16/gpt-home.git
   cd gpt-home

2. Set up environment

   cp .env.example .env
   # Edit .env with your API keys

3. Run with Docker

   docker compose up -d

4. View logs

   docker compose logs -f app

See the Development Guide for detailed local development instructions.

---

📖 Next Steps

• Understand the Architecture for system design
• Learn about the Agent System powering the AI
• Explore available Tools
• Configure the system in Configuration