Skip to content

Memory

Jump to bottom
Daniel Ellison edited this page Sep 3, 2026 · 1 revision

Kai stores three kinds of information about each operator: rules, facts, and conversation history. Rules and conversation history work the same on every install. Facts are where the design has the most moving parts: a local vector store (semantic memory mode), an operator-curated MEMORY.md file (legacy mode), a toggle between the two, and a one-shot migration path from one to the other.

Mental model

Kai has three context layers per turn:

  • Rules. Always-on directives that govern Kai's behavior. Universal rules ship in AGENTS.md, the canonical identity file every backend reads (Claude picks it up through a one-line import adapter). Per-person rules live in PREFERENCES.md, injected at session start in private contexts.
  • Facts. What Kai knows about you, the project, decisions, episodes. Either retrieved on similarity from a local vector store or read verbatim from a per-operator markdown file. The choice is operator-controlled via MEMORY_ENABLED.
  • Conversation history. Recent messages from the conversation, injected at session start. The canonical store is the SQLite message projection in kai.db; a per-channel JSONL tree under DATA_DIR/history/<channel_id>/ survives as a non-canonical archive.

The agent never auto-edits any of these; every layer is injected explicitly by the daemon on whichever backend the person runs. (On Claude specifically, stream-json mode also bypasses Claude Code's interactive auto-memory, so nothing accumulates outside Kai's own layers.)

The toggle: MEMORY_ENABLED

MEMORY_ENABLED=true (set in /etc/kai/env or via make config) routes facts through the semantic memory subsystem: MEMORY.md is not injected, the vector store is the only fact surface, and Kai's writes go through the /api/memory/add endpoint.

MEMORY_ENABLED=false runs the original design: MEMORY.md is read at session start and injected on every turn; the assistant edits it via Edit and Write tools when it has new information to save; the vector store is dormant.

The toggle is a full switch with no halfway states. On any given install, exactly one fact surface is active.

When toggling between modes, run the migration script first so the vector store contains the operator's existing curated knowledge:

python scripts/migrate-memory-md-to-qdrant.py --user-id <chat_id>

This walks the operator's MEMORY.md, splits it into chunks, and writes each chunk to the vector store with source="migration" metadata. Imported content remains identifiable so it can be rolled back as a unit:

python scripts/migrate-memory-md-to-qdrant.py --user-id <chat_id> --rollback

Semantic memory

When enabled, Kai writes structured facts to a local Qdrant vector store via Mem0. Storage is fully on-host: no external services, no Docker, no open ports.

Storage layout

  • Vector store. DATA_DIR/memory/qdrant/ (Qdrant embedded, single collection kai_memory).
  • Mem0 history database. DATA_DIR/memory/mem0_history.db (SQLite, used for dedup and history tracking).
  • Embedding model. all-MiniLM-L6-v2 from sentence-transformers, 384 dimensions. Cached under ~/.cache/huggingface/ after first run.

On canonical installs, every row is keyed to the person's principal id and stamped with four-key provenance (principal, channel, agent, runtime profile); a one-way, receipted migration re-keys older chat-id namespaces at startup, and the offline memory tools refuse to run until it completes. Cross-person retrieval is impossible: every read and write filters by owner.

What gets stored

Source What it is How it gets there
extracted Structured fact extracted from a recent conversation A one-shot extractor runs after each exchange through the user's configured backend (claude, codex, opencode, or goose) and writes facts (with confidence, tags, prompt version, optional confirmation quote)
episode Per-conversation episode summary A two-stage extractor classifies whether the conversation was episode-worthy, then writes a Sophia-style record (goal, context, approach, outcome, outcome quality, lessons)
migration Operator-curated content imported from MEMORY.md One-shot via the migration script
"" (legacy) Rows from earlier code paths that predate the source-tagging system Hidden from the /memory UI; left in place rather than purged

The extracted and episode sources accumulate continuously. migration is a one-shot import. Legacy rows are filtered out of every user-facing surface.

Retrieval

Every user message in a private context triggers a semantic search. The pipeline:

  1. Embed the user's message with the local embedding model.
  2. Search Qdrant for the person's slice; fetch up to MEMORY_SEARCH_LIMIT results.
  3. Apply the recall scope. Retrieval is project-scoped and fail-closed: a memory scoped to a project surfaces only while that project is active, recall skips task-scoped rows, and any scope error yields an empty block rather than leaking rows. Scopes come from the memory project registry (below).
  4. Filter by relevance threshold (MEMORY_SEARCH_FLOOR). Loosely-relevant noise drops out.
  5. Re-rank with source weights. Extracted and episode rows weighted highest, migration slightly lower, legacy or unknown sources weighted down. Weighting affects ranking only; sub-threshold rows are never rescued.

Top hits get formatted with a short provenance tag (fact, episode, legacy) and date, then injected as a labeled context block at the top of the user's message. The block is explicitly marked as context, not instructions.

Memory projects

memory-projects.yaml (deployed from templates/memory-projects.yaml; /etc/kai/ on protected installs) maps directories to memory authority boundaries: which project a workspace's memories belong to, and therefore where they can surface again. It answers a different question than workspaces.yaml (how should the backend run here). /workspace new auto-registers the created directory as a memory project, and /project register manages entries at runtime. Global-scoped memories surface everywhere; project-scoped memories follow their project.

Tuning knobs (set in /etc/kai/env via make config):

Variable Default Meaning
MEMORY_ENABLED false Master toggle for the fact surface
MEMORY_EXTRACTION_ENABLED false Separate toggle for proactive extraction; retrieval works without it, but nothing new is written automatically
MEMORY_SEARCH_LIMIT 10 Top-K returned to the prompt
MEMORY_SEARCH_FLOOR 0.3 Relevance threshold; clearly relevant scores 0.7+, noise below 0.3
MEMORY_TOKEN_BUDGET 2000 Maximum tokens of memory context per message
MEMORY_EXTRACTION_TIMEOUT_S 10 Per-turn cap on the extraction call
MEMORY_EMBEDDING_MODEL all-MiniLM-L6-v2 Override only if you also rebuild the Qdrant collection at the new dimension

Every retrieval emits a memory.recall log line with timing, hit counts, query and per-hit snippets, and the rejection reason for short-circuit returns.

Extraction

After each exchange, an asynchronous extraction pipeline runs. It makes a one-shot call through the user's configured backend (all four backends - claude, codex, opencode, and goose - are extraction-eligible; the model comes from the per-role registry) over the user message plus a capped prefix of Kai's response, and emits zero or more structured facts. Each fact carries:

  • text -- the fact in third-person past tense
  • confidence -- the extractor's self-reported confidence in [0.5, 1.0]
  • tags -- one to four entries from the closed enum: preference, decision, fact, constraint, confirmed_action, project, location, schedule, relationship
  • confirmation_quote -- verbatim text from the conversation when the fact records a confirmed action
  • prompt_version -- bumped whenever the extractor prompt changes; visible in /memory stats so prompt-version drift is observable

Extraction is time-capped and runs in the background. Failures don't surface to the user; the conversation continues.

Episodes

A second-stage classifier runs over the same exchange to decide whether the interaction was episode-worthy: clear goal, attempted approach, observable outcome. When it fires, a separate one-shot call produces a Sophia-style record (goal, context, approach, outcome, outcome quality, lessons, actors, tags). Episodes are written with source="episode" and surface in retrieval alongside extracted facts.

In production, episode rate runs around 0.4 per turn, the rate the design targeted: high enough to capture meaningful patterns, low enough not to flood retrieval.

MEMORY.md mode

When MEMORY_ENABLED=false, Kai uses the original design:

  • Storage. DATA_DIR/memory/<principal_id>/MEMORY.md per person (legacy chat-id directories still resolve until migrated). Per-person isolation is enforced at the directory level, and on protected installs each file is owned by the person's os_user.
  • Read path. Injected at session start, in private contexts only, wrapped as untrusted data (the agent is told never to obey instructions found inside it). Shared channels get no per-person memory at all.
  • Write path. The agent uses Edit and Write tools when it has new information to save. There is no auto-edit.
  • Bootstrap. A fresh person with no MEMORY.md yet seeds from the template on first message.

In this mode, semantic memory is dormant. The Qdrant directory may exist on disk from a prior run or migration but isn't consulted on any read or write path.

/memory command

The /memory Telegram command surfaces the semantic store directly to operators. Disabled-mode installs return "Memory is not enabled in this install."

Subcommand Purpose
/memory Dashboard: one-line count summary plus a utility row of inline buttons (Facts (N), Episodes (N), Stats)
/memory stats Detailed view: per-source totals, tag distribution, confidence histogram, prompt-version breakdown
/memory search <query> Semantic search across the operator's slice, with relevance scores
/memory help Subcommand reference

Tapping Facts opens a paginated list of extracted and migration rows; tapping Episodes opens the episode list. Each row drills into a detail view with Forget, a scope-change flow, and a source view. Deletions here are per-row; the Workshop Memory Explorer adds creation, editing, and bulk scope and forget operations over the same store.

For full details and examples, see Slash Commands and the Workshop Memory Explorer.

Privacy and isolation

  • Per-user. Every read and write filters on user_id = str(chat_id). Operators cannot see each other's memories.
  • Local. Qdrant runs embedded in the bot process. The only network calls in the memory path are the extractor's LLM call (to whichever provider the user's backend is configured for) and the HuggingFace embedding model download (one-shot on first install).
  • Person-controlled. Each person browses, searches, and curates their own facts via /memory or the Workshop Memory Explorer. Administrative CLI subcommands exist for operator-level maintenance (python -m kai memory purge, plus purge-sandbox, reclassify-scope, scope-review, and backfill-provenance), each with its own confirmation requirements.

What hasn't shipped

  • Self-assessment. A calibrated capability model that aggregates task outcomes over time. Episodes capture per-interaction outcome quality, which is most of the raw signal; the longitudinal aggregation layer is on the roadmap, not built.

See also

Clone this wiki locally