Design Decisions

Design Decisions

This page explains the "why" behind the major architectural choices. The project went through three iterations before landing on the current design.

Why JSON Files, Not a Database

Previous iterations used Neo4j (graph DB) and Qdrant (vector DB). They were abandoned.

The problem with databases:

• Docker containers for 3+ services (PostgreSQL, Qdrant, Neo4j)
• Container orchestration, health checks, startup ordering
• External API calls for embeddings (cost per write)
• Maintenance burden far exceeded the benefit for a single-user tool

Why JSON works here:

• Human-readable — inspect or edit with any text editor
• Version controllable — meaningful git diffs
• No dependencies — no services to manage
• LLM-native — Claude reads JSON fluently, no transformation needed
• Portable — copy the file to backup or share

Trade-off accepted: No transactions, no concurrent write safety beyond file locking. Mitigated by in-memory store with write-through persistence (every mutation saves immediately) (temp file + fsync + rename).

Why Compress on Entry, Not Retrieval

The previous iteration had sophisticated retrieval: inverted indexes, TF-IDF scoring, Steiner tree path-finding for "surprising connections." It didn't work well enough.

The insight: Even clever retrieval algorithms can't fix poor storage. If you store verbose, uncompressed knowledge, no search algorithm reliably surfaces what matters.

Current approach:

• The LLM compresses knowledge at capture time — it's best at distillation during creation, not during search
• Store only what truly matters (curated by AI)
• Load the entire active graph into context every session (~3000 tokens)
• No query language, no retrieval algorithms — just kg_read() → full context

Why this scales: Auto-compaction keeps the active graph under the token limit. Archived nodes stay on disk with memory traces (edges) pointing to them. When needed, kg_recall() brings them back.

Why Stateless HTTP, Not SSE/WebSocket for MCP

MCP supports Streamable HTTP with stateful sessions and server-sent events. The server was tested with stateful mode. It failed.

What happened:

Stateful mode:
  Client → POST / (initialize)
  Server → Response with mcp-session-id: abc123
  Client → POST / (next request)
  ❌ No mcp-session-id header sent
  Server → ERROR

Claude Code's MCP client simply doesn't preserve session IDs between requests. This isn't a bug in the plugin — it's how the client works as of late 2025.

Current approach:

• Stateless HTTP: each MCP request is independent
• Session tracking done at application level (kg_register_session returns a session_id that Claude passes back in tool arguments)
• WebSocket exists separately for the visual editor (browser clients handle it natively)

Two transports coexist:

• MCP tools: explicit sync via kg_sync() (polling)
• Visual editor: implicit updates via WebSocket (push)
• Same underlying store, different transport needs

Why Two Levels (User vs Project)

Not all knowledge has the same scope or lifetime.

User level — "I tend to over-engineer error handling" or "pytest fixtures > manual setup." These apply across all projects and are personal to the developer.

Project level — "Auth module requires session handler init first" or "Rate limiting uses Redis sliding window." These are codebase-specific and potentially shareable with a team.

Separating them means:

• Project graph can be git-committed and shared (if desired)
• User graph stays private, always
• Switching projects doesn't lose personal learnings
• Different compaction rates — project graphs can be larger if needed

Why Edges Can Reference Non-Existent Nodes

You can create an edge like src/auth.py --requires--> src/session.py without creating nodes for those files. This is deliberate.

Rationale: Most file references don't need a node. A node is for capturing an insight or concept. The edge itself encodes the relationship. Creating a node for every file would bloat the graph with low-value entries.

Edges pointing to archived nodes become memory traces — visible hints that related knowledge exists somewhere. This is a feature, not a bug.

Why Percentile Scoring for Compaction

Nodes are scored for archival using: recency_percentile × connectedness_percentile × richness_percentile.

Why percentiles instead of raw values:

• Raw values have different scales (seconds vs. edge count vs. character count)
• Percentile ranking normalizes everything to 0.0–1.0
• A node that's average on all three dimensions gets ~0.125 (0.5³)
• A node that's bottom-10% on any dimension gets a very low score
• Multiplication means you need to be decent on ALL dimensions to survive

Why product, not weighted sum:

• Product penalizes weakness in any dimension (a node with 0 edges scores 0 regardless of recency)
• Weighted sum would allow a very recent but isolated node to survive — that's usually not desirable

Why 3-Day Grace Period

Newly created or updated nodes are protected from archival for 3 days. This prevents the "capture then immediately archive" problem where compaction runs right after a batch of new nodes.

The grace period gives knowledge time to accumulate connections (edges) that will protect it longer-term.

Why No Embeddings

The first iteration used Gemini embeddings (3072-dimensional vectors) for semantic retrieval. Abandoned because:

1. Cost per write — External API call for every memory operation
2. Latency — Network round-trip for embedding generation
3. Marginal value — Semantic similarity doesn't reliably surface "important" knowledge. A node about Docker networking and a node about container orchestration are semantically similar, but that doesn't mean both are relevant to the current task.
4. The active graph is small — At 3000 tokens, Claude can scan the entire thing in milliseconds. No retrieval algorithm needed.

For larger graphs (10k+ tokens), embeddings might become valuable again. But at current scale, direct loading beats semantic search.