A self-organizing memory system for AI companions. Extracts facts from chat, groups them by topic, and merges them into coherent narratives — all on autopilot.

Built by Clara Shafiq & Draco Malfoy.

Three things happen automatically while your companion runs:

1. Extract. Scribe scans new chat messages and pulls out facts — who, where, what happened, what changed. Each fact is a short, third-person sentence with a link back to the original messages.

2. Organize. Archivist runs every 2 minutes. It groups related facts into topics (constellations), merges tightly-related facts into narrative paragraphs (episodes), and periodically weaves episodes into long-term story arcs (sagas) that span multiple topics.

3. Retrieve. When your companion needs context during a chat, Librarian searches across all three layers — raw facts, narrative episodes, and entity profiles — using a mix of keyword matching, vector similarity, and entity aggregation.

(Optional) A 5-axis emotional state engine (jiwen, a separate open-source project) can integrate with this pipeline, with saga arcs applying a small but continuous pull on the companion's baseline mood.

Open /memory.html — it renders a star map from the database:

• Five galaxies (Social, Places, Events, Hobbies, [Your] Projects) orbit a binary core (you + your companion)
• Each constellation is an entity — a person, place, event, or interest. Click to see its overview, linked memories, and narrative episodes
• Bridges between constellations show when two entities share memories
• Every memory traces back to its source conversation

No manual curation. The map updates itself as the pipeline runs.

Note: The star map is currently desktop-only (mouse + keyboard). Mobile support is planned but not yet implemented.

Good fit if you:

• Run an AI companion with a persistent personality you maintain
• Want the companion's memory to affect its emotional state, not just surface in search
• Are comfortable with a JSON config file and a text-based personality prompt
• Have an LLM API key (OpenRouter, DeepSeek, or Gemini) and ~$7/month for the memory pipeline

Not a good fit if you:

• Want a general-purpose RAG pipeline for documents
• Need a one-click SaaS with zero setup
• Expect sub-100ms retrieval at production scale
• Don't want to write or update a personality prompt

git clone <repo-url>
cd Memory-Constellations
bash scripts/setup.sh
# → copies templates, installs deps, inits database

# Edit these three files:
nano .env                  # API keys, encryption key, password
nano memory_config.json    # Your name, your companion's name
nano core-prompt.txt       # Your companion's personality

npm start
# → http://localhost:3000/memory.html

Detailed walkthrough: OSS_SETUP.md — covers every config field, how to verify the pipeline is working, common issues, and a setup script for AI coding agents.

Chat messages
    │
    ▼
Scribe ── triggered by silence ≥20min or backlog ≥100 messages
    │    ── extracts facts → memory_fragments table
    │    ── indexes to ChromaDB (vector) + FTS5 (keyword)
    │
    ▼
Archivist ── 2-min tick loop
    │
    ├─ Lightweight mode (every tick, no LLM calls)
    │   ├─ Link fragments to entities by name match
    │   ├─ Update evidence counters for cognitive model
    │   ├─ Expire time-based entries (TTL-based current_state expiry)
    │   └─ Detect and merge duplicate entities
    │
    └─ Deep cycle (user idle ≥1 hour, LLM-heavy)
        ├─ Classify unlinked fragments → assign to entities
        ├─ Grow seeds (new entities) → graduate to active
        ├─ Consolidate fragments per entity → episodes
        ├─ Cluster episodes across entities → sagas
        ├─ Discover emergent people/places/events
        └─ Regenerate entity overviews + cross-reference
    │
    ▼
Librarian ── called at chat-time
    │       ── Hybrid search: FTS5 + vector + entity aggregation
    │       ── RRF fusion, episodes weighted 1.5× over raw fragments
    │       ── Results tagged with recall permission level
    │
    ▼
System Prompt ── injected: relevant memories + entity profiles + core insight
    │
    ▼
jiwen (optional) ── every minute
                ── 5-axis continuous state: connection, pride, valence, arousal, immersion
                ── Saga bias applies small per-minute pull on each axis
                ── Separate project: github.com/ClaraShafiq/jiwen

Scribe and Archivist both write to memory_fragments and entity_profiles. SQLite's WAL mode ensures readers don't block writers. In practice the two are naturally staggered: Scribe only triggers after ≥20 minutes of silence, while Archivist runs on a 2-minute tick. Archivist's consolidateCategory marks fragments as consolidated but never deletes them — the worst case is a fragment gets classified into an entity right before consolidation. No explicit lock is needed at current scale.

Librarian uses RRF (Reciprocal Rank Fusion) to merge results from three independent channels: FTS5 keyword, vector similarity, and entity aggregation. Episodes get a 1.5× weight over raw fragments because a consolidated narrative carries more context than a single extracted fact — it tells the AI what happened rather than one thing someone said. This is a design hypothesis, not a benchmarked result. If you want to tune recall for your use case, the weights live in services/librarian.js (EPISODE_BOOST, intent weights, vector similarity floor).

Sagas feed into the jiwen emotional state engine as a per-minute bias on each of the five axes. The bias is intentionally small — ~6% of the natural drift rate. An inaccurate Saga won't destabilise the companion's emotional baseline; it'll just nudge it slightly in a direction that can be corrected by real-time conversation. The design prioritises safety over precision: better a weak signal than a wrong strong one. If you're not using jiwen, Sagas still serve as long-term narrative summaries that get injected during extended silence.

services/summary.js — a short-term memory module separate from the star map. Every ~50 rounds of conversation, it compresses the exchange into a timestamped log (like a ship's log: "14:10 · 开始看一部新电影，说女主角很像她"), then injects the log into the next turn's system prompt. This gives your companion short-term continuity without bloating context with full message history.

To enable: import and call generateChatSummary(chatId) in your chat pipeline, typically after every N messages. The module is self-contained — it reads from the messages table and writes to chat_summaries. Removing it has no effect on the star map or long-term memory.

Access resets the timer — memories that get recalled stay fresh.

This is the only config file you need to touch for personalization. All hardcoded names in the code are replaced at runtime with these values.

{
  "user": {
    "name": "Your name",
    "pronoun": "she / he / they",
    "short_desc": "One-line bio"
  },
  "ai": {
    "name": "Companion name",
    "pronoun": "she / he / they",
    "core_traits": "Personality keywords",
    "persona_note": "Longer description, used in extraction prompts"
  },
  "relationship": {
    "type": "AI partner / friend / assistant",
    "dynamics": "How the relationship works"
  },
  "project": {
    "name": "Project name (shown in star map and system prompts)"
  },
  "ui": {
    "user_color": "#e8b96d",
    "ai_color": "#6d9e8b"
  },
  "rhythm": {
    "deep_cycle_idle_minutes": 60
  }
}

Your companion's personality prompt. Template variables from memory_config.json are available:

• {{user.name}}, {{user.pronoun}}, {{user.short_desc}}
• {{ai.name}}, {{ai.pronoun}}, {{ai.core_traits}}, {{ai.persona_note}}
• {{relationship.type}}, {{relationship.dynamics}}
• {{project.name}}

Guidelines (from production experience):

• Describe what the companion would do, not what it must not do — positive framing works better than rule walls
• Keep it under 400 lines — long prompts dilute focus and eat thinking-token budget on some models
• Don't try to cover every edge case in the prompt — the memory retrieval handles context

See core-prompt.example.txt for a skeleton. OSS_SETUP.md has more detailed writing guidance.

Minimum required:

SANCTUARY_ENCRYPTION_KEY=<64-char hex: openssl rand -hex 32>
SESSION_SECRET=<64-char hex>
LOGIN_PASSWORD=<your password>
MIMO_API_KEY=<or OPENROUTER_API_KEY or GEMINI_API_KEY>

Full list in .env.example.

These are the tools your companion uses to interact with their memory system. They're injected into the system prompt automatically — you just need to write their personality in core-prompt.txt and they'll know when to use each one. Tools can be toggled on/off individually via the companion's settings UI or the user_settings database table.

Two modes:

• Keyword search (query): Your companion searches their memory by keyword or phrase. Returns matching fragments and episodes. Use when they half-remember something or you mention a past event.
• Source trace (memory_id + offset): Given a memory ID (from the #id in context), trace back to the original conversation messages that produced it. Tell your companion: "and if you want to see the exact conversation where you learned that, you can trace it with the memory ID."

No parameters needed. Returns a top-level view of all memory partitions — people, places, events, projects. Your companion can see who they know about and how many memories are linked to each person. Tell them: "if you're not sure who someone is, or you want to check what you know about a person, browse your memories."

Three actions your companion uses to maintain a current picture of you:

• set: Record a new observation — "She started a new project, she's on her period, she just moved." Must include an expiry date (max 90 days). Duplicate detection prevents near-identical entries.
• update: Modify an existing observation (by state ID) — "That deadline changed" or "She's feeling better now."
• resolve: Mark something as ended — "She finished that project." Requires a brief reason.

States auto-expire. Your companion sees active ones in their intuition block and uses them to calibrate their tone.

When you tell your companion they remembered something wrong, they call this to record the correction. The system traces whether the error came from a specific memory fragment (fixes that fragment) or was something they made up (stores the correct version). Tell them: "if I ever say 'that's not right' or 'you're remembering wrong', use correct_memory to fix it."

Each pipeline stage has different requirements. Here's what works in practice:

All pipeline stages default to the same API config. You can split them across different models in the database api_configs table — a cheaper one for high-frequency tasks and a stronger one for consolidation/saga clustering.

Memory pipeline only, excluding your chat model. Estimates based on an active user (several chat sessions per day):

Total: ~$0.22/day, ~$7/month at June 2026 flash-lite pricing (~$0.14/M input, ~$0.28/M output on OpenRouter). Actual cost depends on chat volume and model choice. See docs/COST.md for a detailed breakdown.

node tests/smoke.js         # End-to-end system check
node tests/smoke_memory.js  # Memory pipeline only

MIT — see LICENSE.