SideQuest

An AI Narrator powered by coordinated Claude agents, so everyone gets to play.

SideQuest runs a council of specialized AI agents — narrator, combat, NPC, and world state — that collectively act as your Narrator. You type actions; the system routes them to the right agent and returns narration. Swap in a genre pack to change the entire tone, rules, and setting.

Quick Start

# Install
pip install -e .

# Start the game server (default genre: low_fantasy)
sidequest --server

# Start with a different genre
sidequest --server --genre mutant_wasteland

# Open the React client
cd client && npm run dev
# Then visit http://localhost:5173

Requires Python 3.11+ and a working Claude CLI installation.

How It Works

graph TD
    Browser[React Client] -->|WebSocket| Server[GameServer]
    Server --> Orchestrator

    subgraph "Agent Council"
        Orchestrator --> IntentRouter[Intent Router]
        IntentRouter -->|exploration, examine| Narrator
        IntentRouter -->|combat| Combat
        IntentRouter -->|dialogue, persuasion| NPC
        Narrator -->|JSON patch| WorldState[World State]
        Combat -->|JSON patch| WorldState
        NPC -->|JSON patch| WorldState
        WorldState -->|updated GameState| Orchestrator
    end

    subgraph "Prompt Assembly"
        PromptComposer[Prompt Composer] -->|system prompts| Orchestrator
        GenrePack -->|rules, tone, lore| PromptComposer
        SOUL[SOUL.md] -->|guiding principles| PromptComposer
    end

    subgraph "Output Pipeline"
        Narrator -->|narrative text| SceneInterpreter[Scene Interpreter]
        Combat -->|narrative text| SceneInterpreter
        NPC -->|narrative text| SceneInterpreter
        SceneInterpreter -->|StageCues| Renderer
        Renderer --> ImageGen[Flux Image Generation]
        MusicDir[Music Director] -->|mood selection| AudioMixer[Audio Mixer]
    end

    subgraph "Voice Pipeline"
        VoiceRouter[Voice Router] --> Kokoro[Kokoro TTS]
        VoiceRouter --> Piper[Piper TTS — fallback]
        Kokoro --> TTSService[TTS Service]
        Piper --> TTSService
    end

    subgraph "Client"
        Browser --> WhisperSTT[Whisper STT — local]
        Browser --> WebAudio[Web Audio API]
        Browser --> WebRTC[WebRTC Voice Chat]
    end

    TTSService -->|streaming audio| Server
    Server -->|streaming chunks| Browser

Loading

The Orchestrator receives player input and uses the Intent Router to dispatch it to the right agent. Each agent runs as an interactive Claude session with a system prompt assembled by the Prompt Composer — which layers genre pack rules, SOUL.md principles, and world state into a structured, attention-aware prompt.

After each action, the World State agent produces a JSON patch to keep the shared GameState consistent. The Scene Interpreter extracts structured stage cues from narrative text, which the Renderer distributes to image generation, audio, and voice pipelines. Results stream back to the React client over WebSocket.

Agent	Handles
Narrator	Exploration, description, story progression
Combat	Attacks, spells, tactical decisions
NPC	Dialogue, persuasion, social interaction
World State	State tracking, consistency enforcement
Intent Router	Classifies player input and selects the right agent
Music Director	Selects background music based on scene mood

Genre Packs

Genre packs are YAML directories that configure tone, rules, NPCs, character creation, audio, and visuals. The default pack is low_fantasy — a gritty, low-magic world where steel matters more than spells.

genre_packs/low_fantasy/
├── pack.yaml            # Name, version, description
├── archetypes.yaml      # Character archetypes and classes
├── char_creation.yaml   # Scene-driven character creation
├── lore.yaml            # World lore, history, factions
├── rules.yaml           # Game mechanics and stat generation
├── prompts.yaml         # Agent prompt extensions
├── theme.yaml           # Color schemes and UI styling
├── inventory.yaml       # Item tables and equipment
├── progression.yaml     # Leveling and advancement
├── audio.yaml           # Music and sound configuration
├── visual_style.yaml    # Image generation style directives
└── voice_presets.yaml   # TTS voice configuration per archetype

To create a new genre, add a directory under genre_packs/ following the same structure. See docs/genre-packs.md for the full format and a step-by-step creation guide.

React Client

The game UI is a React web client (client/) that connects to the server over WebSocket:

• Narrative view — streaming narration with inline scene art
• Party panel — live HP bars and character stats
• Character sheet — detailed stats, inventory, and progression
• Map overlay — world map and known regions
• Audio controls — now-playing indicator, volume, mute
• Voice input — local Whisper STT via Transformers.js + WebGPU
• Voice chat — WebRTC peer-to-peer between players

Multiplayer

Multiple players connect to the same game server over WebSocket. The system batches actions with an adaptive timer (scales with player count) and resolves them together. Each player can receive a different version of narration based on their character's perception — a blinded character hears different details than one who can see.

Project Structure

sidequest/
├── agents/              # AI agents (narrator, combat, npc, world_state, intent_router, music_director)
├── audio/               # Audio pipeline (mixing, queue, loudness normalization)
├── comms/               # Inter-agent communication (channel)
├── game/                # Game logic — character, state, session, chase, persistence, progression
├── genre/               # Genre pack loader, Pydantic models, lore system
├── media/               # Image generation (Flux workers, caching, subject extraction)
├── renderer/            # Render pipeline — stage cues, beat filtering, stale detection
├── server/              # WebSocket server, TTS service, collect window, turn modes
├── voice/               # Voice/TTS pipeline (Kokoro, Piper fallback, effects, streaming)
├── main.py              # CLI entry point
├── orchestrator.py      # Agent coordination and game loop
├── prompt_composer.py   # Three-tier prompt assembly with genre/SOUL integration
├── scene_interpreter.py # Narrative-to-stage-cue extraction
└── soul.py              # SOUL.md parser for guiding principles
client/                  # React web client (TypeScript, Vite, Tailwind, shadcn)
genre_packs/             # Genre pack data (YAML)
docs/                    # Documentation

Documentation

Doc	Audience	Description
How to Play	Players	Getting started, the world contract, commands, genre showcase
Features	Players / Prospective users	Full feature overview
Architecture	Developers	System design, component diagrams, data flow
Genre Packs	Creators / Developers	Genre pack format and creation guide
Genre Pack Audio	Creators	Audio configuration for genre packs

Development

python -m pytest                 # Run tests
python -m pytest -k "pattern"    # Run specific tests
ruff check .                     # Lint
python -m build                  # Build package
cd client && npm run dev         # Start React client dev server

License

MIT