Implementation Plan

Implementation Plan

A provider-agnostic, cross-LLM shared long-term memory MCP server.

Principles: plain text is the only source of truth · zero extra generative-LLM calls (local deterministic embeddings don't count) · HTTP-native · managed DB + semantic retrieval from day one · three pluggable layers (Auth / Store / Retriever) · limit/cursor pagination with judgment left to the client.

1. Tech stack (locked)

Dimension	Choice	Notes
Language / framework	Python 3.11+ + FastMCP	ergonomic, rich embedding ecosystem
Transport	Streamable HTTP (skip stdio)	endpoint /mcp; stdio only as optional local debug
DB	Neon free Postgres + pgvector (day one)	use the pooled connection string
Primary retrieval	Semantic (pgvector nearest-neighbour)	live in Phase 1
Embeddings	Local model (fastembed, ONNX, no key)	a multilingual small model (e.g. multilingual-e5-small / bge-m3); dimension pinned into schema
Keyword / hybrid	Optional, added later (keyword + vector RRF)
Auth	per-user bearer key (only scheme, no OAuth)	thin authenticate() -> user_id; {api_key -> user_id} table
Deploy	Google Cloud Run	container ships the local embedding model, >=1GB RAM
Tests	pytest	including "can the LLM call the tools correctly"
Launch	PyPI + server.json + mcp-publisher	official Registry + Glama / mcpservers.org

2. Data model

memories table (Postgres + pgvector):

• id (uuid) · user_id (from auth) · namespace (shared/private) · content (plain text, source of truth)
• embedding vector(N) (N = model dimension, e.g. 384) · HNSW index
• tags (optional) · source (which LLM/session wrote it) · created_at / updated_at

Two core principles:

1. Chunk on write — large content is split into small chunks on ingest, one vector each; recall is always "N small things", handled by one limit.
2. The server owns embedding — whether Claude / GPT / Gemini connects, reads and writes use the same model, so the vector space is consistent and cross-LLM recall is accurate. The original stays as plain text: readable, exportable.

3. MCP tool surface

Tool	Signature	Annotation
remember	(content, namespace?, tags?, source?) -> id (embeds on write)	write
recall	(query, namespace?, limit=8, cursor?) -> {items,total,has_more,next_cursor} (vector NN)	readOnly
list_memories	(namespace?, limit=20, cursor?)	readOnly
get_memory	(id)	readOnly
update	(id, content) (re-embeds)	write
forget	(id)	destructive
save_session	(session_id, content, namespace?) auto chunk-on-write	write
load_session	(session_id, page=0, page_size=8000) char pagination	readOnly

• Resources: memory://{namespace} exposes a collection for auto-injection.
• Every tool: clear description, structured output, LLM-actionable errors.
• Judgment left to the client: default limit=8, similarity-ranked, {total,has_more,next_cursor} so the model decides whether it's enough / whether to page.

4. Three pluggable layers

Auth — authenticate(request) -> user_id

• BearerKeyAuth ({api_key -> user_id} table) — the only scheme.

Store — add / get / search / list / update / delete / count

• PostgresStore (Neon + pgvector) — default, from day one.
• (optional) local debug implementation.

Retriever — search(query, namespace, limit, offset) -> ranked rows

• VectorRetriever (pgvector + embedding) — MVP primary.
• KeywordRetriever / HybridRetriever (RRF) — optional, later.

The server owns mechanism (embedding, storage, ranking, bounded results, good defaults, metadata); the client/model owns judgment (what to query, whether it's enough, when to stop).

5. Roadmap (phases)

See Roadmap for the phase-by-phase breakdown with Definitions of Done, mapped to milestones and issues.

6. Risks & trade-offs (honest)

• The embedding model must be pinned. Once chosen, dimension and vector space are fixed; switching models means re-embedding everything. The server owning embedding keeps it consistent across LLMs.
• Chinese / cross-language: a multilingual embedding handles Chinese naturally, even Chinese-query-over-English content — easier than full-text search (a bonus of going vector-first).
• Container size / memory: the local model in the image (~100–400MB) plus some RAM; fastembed (ONNX) is lighter than sentence-transformers (no torch).
• Still "zero external / generative-LLM calls": embeddings run locally — no outbound request, no API key, deterministic. The selling point holds.
• Readability: vectors are only an index; the original is always plain text, exportable to markdown.
• Concurrent writes: Postgres handles multi-writer natively; MVP conflict policy is last-write-wins + namespace isolation.
• Bearer: use per-user keys (not a shared single token), or namespaces are meaningless.

7. Explicitly out of scope

• ❌ OAuth / complex multi-tenant auth (per-user bearer only)
• ❌ Inferring implicit preferences from behaviour (research-grade; even mem0 is ~30–45%)
• ❌ mem0-style automatic generative-LLM extraction / reconciliation (the tax we shed)
• ❌ Letta-style full stateful agent runtime (beyond the scope of an MCP server)

8. Portfolio framing

• README: one-line positioning + architecture diagram + per-tool schema + demo GIF.
• Why it's not a thin wrapper: plain-text source of truth, zero generative-LLM calls, local embeddings, three pluggable layers, provider-agnostic sharing.
• A comparison section: vs mem0 / KG memory / Letta.
• Résumé line: "Designed and shipped a provider-agnostic, pgvector-semantic, zero-external-LLM-call cross-LLM shared-memory MCP server (published to the official Registry)."