-
Notifications
You must be signed in to change notification settings - Fork 1
Design Decisions
This page explains the "why" behind the major architectural choices. The project went through three iterations before landing on the current design.
Previous iterations used Neo4j (graph DB) and Qdrant (vector DB). They were abandoned.
The problem with databases:
- Docker containers for 3+ services (PostgreSQL, Qdrant, Neo4j)
- Container orchestration, health checks, startup ordering
- External API calls for embeddings (cost per write)
- Maintenance burden far exceeded the benefit for a single-user tool
Why JSON works here:
- Human-readable — inspect or edit with any text editor
- Version controllable — meaningful git diffs
- No dependencies — no services to manage
- LLM-native — Claude reads JSON fluently, no transformation needed
- Portable — copy the file to backup or share
Trade-off accepted: No transactions, no concurrent write safety beyond file locking. Mitigated by in-memory store with write-through persistence (every mutation saves immediately) (temp file + fsync + rename).
The previous iteration had sophisticated retrieval: inverted indexes, TF-IDF scoring, Steiner tree path-finding for "surprising connections." It didn't work well enough.
The insight: Even clever retrieval algorithms can't fix poor storage. If you store verbose, uncompressed knowledge, no search algorithm reliably surfaces what matters.
Current approach:
- The LLM compresses knowledge at capture time — it's best at distillation during creation, not during search
- Store only what truly matters (curated by AI)
- Load the entire active graph into context every session (~4000 tokens per level, configurable via
KG_MAX_TOKENS) - No query language, no retrieval algorithms — just
kg_read()→ full context
Why this scales: Auto-compaction keeps the active graph under the token limit. Archived nodes stay on disk with memory traces (edges) pointing to them. When needed, kg_read(cwd, id) retrieves their full content and promotes them back to active.
MCP supports Streamable HTTP with stateful sessions and server-sent events. The server was tested with stateful mode. It failed.
What happened:
Stateful mode:
Client → POST / (initialize)
Server → Response with mcp-session-id: abc123
Client → POST / (next request)
❌ No mcp-session-id header sent
Server → ERROR
Claude Code's MCP client simply doesn't preserve session IDs between requests. This isn't a bug in the plugin — it's how the client works as of late 2025.
Current approach:
- Stateless HTTP: each MCP request is independent
- Session tracking done at application level — the first
kg_read(cwd)call registers a session and returns asession_idthat Claude passes back in subsequent tool arguments (the old standalonekg_register_sessiontool was absorbed intokg_readin 0.9.0) - WebSocket exists separately for the visual editor (browser clients handle it natively)
Two transports coexist:
- MCP tools: explicit sync via
kg_sync()(polling) - Visual editor: implicit updates via WebSocket (push)
- Same underlying store, different transport needs
Not all knowledge has the same scope or lifetime.
User level — "I tend to over-engineer error handling" or "pytest fixtures > manual setup." These apply across all projects and are personal to the developer.
Project level — "Auth module requires session handler init first" or "Rate limiting uses Redis sliding window." These are codebase-specific and potentially shareable with a team.
Separating them means:
- Project graph can be git-committed and shared (if desired)
- User graph stays private, always
- Switching projects doesn't lose personal learnings
- Different compaction rates — project graphs can be larger if needed
You can create an edge like src/auth.py --requires--> src/session.py without creating nodes for those files. This is deliberate.
Rationale: Most file references don't need a node. A node is for capturing an insight or concept. The edge itself encodes the relationship. Creating a node for every file would bloat the graph with low-value entries.
Edges pointing to archived nodes become memory traces — visible hints that related knowledge exists somewhere. This is a feature, not a bug.
Nodes are scored for archival using: 0.33 × recency_pct + 0.66 × connectedness_pct.
Why percentiles instead of raw values:
- Raw values have different scales (timestamps vs. edge counts)
- Percentile ranking normalizes everything to 0.0–1.0
- A node that's average on both dimensions gets ~0.5
Why weighted sum, not product:
- Product would zero out a well-connected node with stale recency — unfair to durable foundational concepts
- Weighted sum lets high connectedness compensate for low recency
Why connectedness is weighted 0.66:
- It's the most objective structural signal — how many active nodes depend on or reference this one
- In-degree (others point here) weighted higher (×0.66) than out-degree (this points elsewhere, ×0.33): incoming edges indicate the node is a dependency; outgoing edges indicate it's a consumer
- Only edges to currently active nodes count — edges to archived/orphaned nodes don't protect a node
Why richness was dropped:
- Content length is a proxy for effort, not value — a crisp 80-char gist is better than a verbose one
- It rewarded verbosity and was easy to game
Why recency tracks reads, not just writes:
-
_last_read_tsis stamped on everykg_read(id=...)call - A node that's frequently consulted but rarely rewritten stays fresh in the scorer
Newly created nodes are protected from archival for a configurable period (see KG_GRACE_PERIOD_DAYS). This prevents the "capture then immediately archive" problem where compaction runs right after a batch of new nodes.
The grace period is based on _created_ts only — subsequent updates and reads do not reset it. The grace period gives knowledge time to accumulate connections (edges) that will protect it longer-term. Once the grace period expires, the node competes on its merits permanently.
When compaction archives a node, it may displace a node that was well-connected at archive time. If that archived node has since accumulated more edges than a newly-archived candidate, it deserves to come back.
After each archiving pass, a resurrection pass re-scores archived and active nodes in a unified pool. Any pre-existing archived node that outscores a just-archived node by ≥0.05 is swapped back to active. The margin prevents thrashing on near-equal scores. Resurrection only runs during compaction (graph over limit) — not as a background promotion, which would allow well-connected old nodes to surface constantly.
Claude Code has two built-in memory mechanisms that interact poorly with the knowledge graph:
Built-in auto-memory (~/.claude/projects/*/memory/) — runs in parallel, captures its own entries. The result is two competing memory systems with no coordination. Nodes appear in both, instructions can contradict, and every session loads context from both sources. Disabling auto-memory makes the knowledge graph the single, intentional source of truth.
Project-level CLAUDE.md files — Claude Code loads all CLAUDE.md files it finds (global + per-project). Each one adds to the instruction set. When multiple files exist, instructions tend to contradict or duplicate. The knowledge graph is designed to carry project-specific knowledge (architecture decisions, patterns, non-obvious constraints) precisely so project-level config files aren't needed. One global ~/.claude/CLAUDE.md containing the KG template is the recommended setup.
The design goal: one memory system, one instruction source. Everything else is noise.
The first iteration used Gemini embeddings (3072-dimensional vectors) for semantic retrieval. Abandoned because:
- Cost per write — External API call for every memory operation
- Latency — Network round-trip for embedding generation
- Marginal value — Semantic similarity doesn't reliably surface "important" knowledge. A node about Docker networking and a node about container orchestration are semantically similar, but that doesn't mean both are relevant to the current task.
- The active graph is small — At ~4000 tokens per level, Claude can scan the entire thing in milliseconds. No retrieval algorithm needed.
For larger graphs (10k+ tokens), embeddings might become valuable again. But at current scale, direct loading beats semantic search.
Three persistent memory systems exist for Claude Code. Understanding how they differ clarifies where KG Memory sits and what trade-offs it makes.
| Aspect | KG Memory | Claude Code Auto-Memory | MemPalace |
|---|---|---|---|
| Structure | Graph — nodes + typed edges | Flat markdown files + MEMORY.md index | Spatial hierarchy — wings/rooms/halls |
| Granularity | Compressed zettelkasten-style insights (gist + notes + touches) | Short markdown files with frontmatter | Verbatim transcripts, never summarized |
| Backend | MCP server, in-memory + write-through JSON | Plain filesystem | ChromaDB (vector) + SQLite (graph triples) |
| Relationships | First-class — explicit labeled edges between nodes | None — flat list | Implicit via spatial co-location; cross-wing tunnels |
| KG Memory | Auto-Memory | MemPalace | |
|---|---|---|---|
| Architecture / code patterns | Yes — core use case | Explicitly excluded | Yes |
| User preferences / profile | Yes (user-level nodes) | Yes | Yes |
| Decisions + rationale | Yes (nodes + edges) | Yes | Yes |
| Debugging insights | Yes | Excluded ("fix is in the code") | Yes |
| Verbatim conversations | No — compressed to insights | No — short summaries | Yes — raw transcripts |
| File paths, git history | Yes (touches on nodes) | Explicitly excluded | Yes |
| Cross-project wisdom | Yes (user level) | Partially (user + feedback types) | Yes (tunnels across wings) |
| KG Memory | Auto-Memory | MemPalace | |
|---|---|---|---|
| Primary method |
kg_read loads full graph; kg_search for full-text |
MEMORY.md index loaded into context every conversation | Semantic search with structural filtering (wing/room/hall) |
| Filtering | By level (user/project), node ID, full-text | By memory type | Hierarchical: wing → room → hall narrows search space |
| Context cost | Graph overview loaded; full nodes read on demand | Entire MEMORY.md always in context (capped at 200 lines) | Relevant verbatim chunks loaded from ChromaDB |
| Recall benchmark | Not benchmarked | Not benchmarked | 96.6% on LongMemEval |
| KG Memory | Auto-Memory | MemPalace | |
|---|---|---|---|
| Multi-session sync | Yes — shared server, kg_sync() for coordination |
No | No |
| Subagent support | Yes — shared server, sync after writes | No | No |
| Compaction | Automatic — scores by recency/connectedness (+ resurrection pass) | Manual — user/agent must update or delete | None — grows unbounded |
| Staleness handling | Recency factor in scoring; archived nodes preserved | Instructions say "verify before acting on memory" | No explicit mechanism |
| Self-evolution | Explicitly designed — plugin captures improvements to its own approach | No | No |
KG Memory — Compressed meaning + relationships. Value is in connections between concepts: "nodes gain meaning through edges." Zettelkasten for AI. Trade-off: high signal density but lossy — requires good judgment on what to compress and what's worth an edge.
Auto-Memory — Lightweight persistence. Simple key-value notes, minimal overhead. Deliberately narrow scope (excludes code patterns, architecture, debugging). Trade-off: low maintenance but shallow — no relationships, no architecture knowledge.
MemPalace — Verbatim fidelity + spatial organization. Never summarize; structure compensates for volume. Ancient memory palace metaphor applied to AI. Trade-off: high recall accuracy but high storage cost; raw text scales poorly without the supporting infrastructure.
The key split: MemPalace bets that raw data + good search = recall. KG Memory bets that compressed insights + explicit connections = understanding. Auto-Memory bets that simplicity + minimal scope = good enough for most cases.