English | 中文 | 日本語 | 한국어 | Tiếng Việt | Français | Русский

sage-wiki is a graph memory and knowledge base that AI agents and humans build and query together. Drop in documents; an LLM compiler turns them into an interlinked wiki with a knowledge graph — agents query it through MCP, humans browse it as plain markdown. Enable the opt-in graph passes and it becomes an evidenced graph: typed entities, provenance-bearing relations, resolved aliases, and per-fact citations on answers. One Go binary scales it from a personal vault to a team hub to a company knowledge graph.

→ Get started: Install · Quickstart

Grown from Andrej Karpathy's idea of an LLM-compiled personal knowledge base, built with the Sage Framework. Some lessons learned along the way here.

• Graph memory with citations. Ask relational questions through wiki_graph_query — answers are grounded only in serialized graph edges; with the evidenced graph enabled, each citation carries its source document and confidence.
• Built for agents and humans. 19 MCP tools plus generated skill files teach agents when to search, capture, and compile; humans get Obsidian-native markdown, a TUI, and a web UI over the same data.
• Trust and provenance. Query outputs quarantine until verified; every evidenced relation records which document asserted it.
• Your sources in, a wiki out. The compile pipeline reads papers, notes, code, and email; summarizes; extracts concepts; and writes interconnected articles — the ingestion layer for everything above. Every new source enriches existing articles; the wiki compounds as it grows.
• Ask your wiki questions. Hybrid chunk-level search with LLM query expansion, re-ranking, and graph-aware context assembly returns cited answers.
• Scales to 100K+ documents. Tiered compilation indexes everything fast and spends LLM budget only where it matters.

Dots on the outer boundary represent summaries of all documents in the knowledge base, while dots in the inner circle represent concepts extracted from the knowledge base, with links showing how those concepts connect to one another.

• Personal — overlay an existing Obsidian vault (init --vault), run on local models for zero cost, and opt into the graph passes (ontology.triples + ontology.resolve) when you want the evidenced graph.
• Team — share one wiki via git or a self-hosted server, review entity-resolution proposals and output trust together, and federate multiple wikis with the hub. See Team Setup.
• Company — move storage to PostgreSQL/pgvector, turn on metrics, front the server with auth, and scale ingestion with tiered compilation.

Vector search retrieves passages that look like the query. A graph also records how things relate, so a question needing two or three hops is answered by traversal instead of hoping one chunk happens to contain the whole chain. sage-wiki builds that graph as a compile output — not a second database you have to keep in sync.

• Entities and typed relations. Each compile extracts entities (concepts, sources, artifacts) and links them with typed relations. The relation vocabulary is yours to define — see configurable relations.
• Evidenced edges. A relation can carry evidence (the span that supports it), confidence (0–1), and source_doc, so a conclusion traces to the sentence that justified the edge rather than to a whole document.
• Triples. An optional structured-output pass extracts subject → relation → object directly. Opt-in (ontology.triples): it adds one LLM call per document, and defaults never spend your key without asking.
• Entity resolution. "K8s" and "Kubernetes" become one node. Proposals are review-gated by default rather than silently merged.

The graph is a retrieval channel, not a side view. Every search fuses three channels — lexical (BM25), vector, and graph proximity: query terms seed entities, a bounded traversal ranks their neighborhood, and the three fuse at search.hybrid_weight_graph. An empty ontology costs nothing and leaves results byte-identical, so the graph earns its place incrementally.

Query it directly, or let an agent do it over MCP:

sage-wiki ontology query --entity kubernetes --depth 3 --direction both
sage-wiki provenance "service mesh"    # which sources produced this concept

Edges are bi-temporal: contradicting a fact invalidates the old edge instead of colliding, default answers are contradiction-free, and as_of queries answer "what did we believe in January?" Ambiguous contradictions still surface through output trust review. For corpus-wide questions ("main themes across everything?"), opt-in community detection (ontology.communities.enabled) generates cached community summaries and answers via wiki_graph_query mode: "global". Depth and mechanics: graph memory.

# CLI only (no web UI)
go install github.com/xoai/sage-wiki/cmd/sage-wiki@latest

# With web UI (requires Node.js for building frontend assets)
git clone https://github.com/xoai/sage-wiki.git && cd sage-wiki
cd web && npm install && npm run build && cd ..
go build -tags webui -o sage-wiki ./cmd/sage-wiki/

sage-wiki init my-wiki && cd my-wiki
# Add sources to raw/
cp ~/papers/*.pdf raw/
# Edit config.yaml to add api key, and pick LLMs
sage-wiki compile                                  # first compile
sage-wiki search "attention mechanism"             # hybrid search
sage-wiki query "How does flash attention work?"   # cited Q&A
sage-wiki tui                                      # terminal dashboard
sage-wiki serve --ui                               # browser (webui build)
sage-wiki compile --watch                          # watch folder

Every config.yaml key, annotated line by line: Configuration.

Project layout (what init creates — selected entries, illustrative not exhaustive):

my-wiki/
├── config.yaml           # providers, models, compiler, search, ontology
├── raw/                  # drop sources here (articles, papers, code, images)
├── wiki/                 # compiled output — Obsidian-compatible markdown
│   ├── summaries/        # per-source LLM summaries
│   ├── concepts/         # concept articles (the knowledge graph)
│   ├── images/           # vision-captioned image descriptions
│   ├── outputs/          # filed query answers (trust.include_outputs: "true")
│   ├── under_review/     # filed answers awaiting trust review (default)
│   └── archive/          # pruned articles
├── .sage/wiki.db         # one SQLite file: FTS index, vectors, ontology, queue
└── .manifest.json        # source↔article mapping + compile state

cd ~/Documents/MyVault
sage-wiki init --vault
# Edit config.yaml to set source/ignore folders, add api key, pick LLMs
sage-wiki compile --watch

Prefer containers? Prebuilt multi-arch Docker images and compose files are covered in the self-hosted server guide.

Just drop files into your source folder — sage-wiki detects the format automatically. Images require a vision-capable LLM (Gemini, Claude, GPT-4o). Need a format not listed? sage-wiki supports external parsers — scripts in any language reading stdin, writing text to stdout.

Out of the box the wiki builds a knowledge graph from keyword proximity — concepts linked where relation keywords co-occur with a [[wikilink]] in the same block. Enable the opt-in graph passes to turn that into an evidenced graph:

• Triple extraction (ontology.triples.enabled) — one extra LLM call per fully-compiled document extracts typed entities and relations, each carrying an evidence span, confidence, and source document.
• Entity resolution (ontology.resolve.enabled) — surface-form variants ("NASA" / "National Aeronautics and Space Administration") are linked to a canonical entity. High-confidence proposals apply automatically (threshold 0.85; set exactly 1.0 for review-only), and every link is exactly reversible with ontology resolve --unlink.
• Graph QA — the wiki_graph_query MCP tool answers multi-hop relational questions grounded only in a bounded, serialized set of edges; citations carry source_doc and confidence when the edge is evidenced (keyword-proximity edges carry neither). Regular Q&A context also names the connecting edge under each related article.

Depth, costs, review workflow, and undo semantics: Graph Memory.

The core surface; run sage-wiki <command> --help for flags.

Topic-specific command families live with their guides: pack * in CONTRIBUTING, auth * (login, import, status, logout, migrate) in Subscription Auth, and verify / outputs * in Output Trust.

sage-wiki tui

A full-featured terminal dashboard with 4 tabs:

• [F1] Browse — Navigate articles by section (concepts, summaries, outputs). Arrow keys to select, Enter to read with glamour-rendered markdown, Esc to go back.
• [F2] Search — Fuzzy search with split-pane preview. Type to filter, results ranked by hybrid score, Enter to open in $EDITOR.
• [F3] Q&A — Conversational streaming Q&A. Ask questions, get LLM-synthesized answers with source citations. Ctrl+S saves answer to outputs/.
• [F4] Compile — Live compile dashboard. Watches source directories for changes and auto-recompiles. Browse compiled files with preview.

Tab switching: F1-F4 from any tab, 1-4 on Browse/Compile, Esc returns to Browse. Quit with Ctrl+C.

sage-wiki serve --ui        # http://127.0.0.1:3333, requires -tags webui build

• Article browser with rendered markdown, syntax highlighting, and clickable [[wikilinks]]
• Hybrid search with ranked results and snippets
• Knowledge graph — interactive force-directed visualization of concepts and their connections
• Streaming Q&A — ask questions and get LLM-synthesized answers with source citations
• Table of contents with scroll-spy; dark/light mode with system preference detection; broken article links shown in gray

Built with Preact + Tailwind, embedded via go:embed (~1.2 MB, ~420 KB gzipped); omit -tags webui for a CLI/MCP-only binary. Auth tokens, allowed hosts, and deployment hardening: Self-Hosted Server.

Add to .mcp.json (Claude Code; other agents in the Agent Memory Layer guide):

{
  "mcpServers": {
    "sage-wiki": {
      "command": "sage-wiki",
      "args": ["serve", "--project", "/path/to/wiki"]
    }
  }
}

Network clients: sage-wiki serve --transport sse --port 3333. The server exposes 19 tools — search, read, graph query, capture, wiki_query (question answering with trust-reviewed filing), compile-on-demand and more; setup per agent and capture workflows live in the Agent Memory Layer guide.

HTTP API (/v1, experimental) — any language can call the same tools over REST: sage-wiki serve --ui --port 3333 mounts 20 routes under /v1 (Bearer auth via SAGE_WIKI_TOKEN, structured errors, idempotent writes). Long-running compile/lint run as async jobs: POST /v1/jobs/compile or POST /v1/jobs/lint returns 202 + a job_id to poll. Contract: api/openapi.yaml (OpenAPI 3.1, drift-checked against the MCP tool set). Guide: docs/guides/http-api.md. Pre-1.0 — pin a version.

Agent skill files — sage-wiki skill refresh --target <agent> writes a behavioral section into the agent's instruction file (CLAUDE.md, .cursorrules, …) teaching it when to search, what to capture, and how to query, derived from your config. Targets: claude-code, cursor, windsurf, agents-md (Antigravity), codex, gemini, generic.

Install sage-wiki's reference skill so a coding assistant knows the full tool surface — all 19 MCP tools, the /v1 REST equivalents, opt-in flags, tiers, async compile semantics, and error codes — without reading this README:

# Claude Code
npx skills add https://github.com/xoai/sage-wiki --skill sage-wiki

# Or manually: copy skills/sage-wiki/SKILL.md to .claude/skills/

The sage-wiki-integrate pipeline skill wires sage-wiki into a new repo interactively (detect language → install client or configure MCP → smoke-test store-and-retrieve):

npx skills add https://github.com/xoai/sage-wiki --skill sage-wiki-integrate

Both skills are generated from the live MCP registry (go run ./tools/skillgen/) and drift-checked in CI — they cannot go stale when tools change. Pre-1.0 — pin a version.

Knowledge capture — agents store insights back via wiki_capture / wiki_learn, closing the read-capture-evolve loop. Workflows and tips: Agent Memory Layer.

Typed clients over the /v1 REST API (pre-1.0 — pin a version):

Python — pip install sagewiki (≥3.9, httpx only):

from sagewiki import SageWiki

c = SageWiki()  # SAGE_WIKI_URL / SAGE_WIKI_TOKEN from env
for r in c.search("attention", limit=5).results:
    print(r.final_score, r.content[:80])
job = c.compile(topic="attention")
job.wait(timeout=600)  # explicit timeout required

TypeScript — npm install sagewiki (zero runtime dependencies, global fetch; Node ≥18, Deno, Bun, edge runtimes):

import { SageWikiClient } from "sagewiki";

const c = new SageWikiClient();
const results = await c.search("attention", { limit: 5 });
const job = await c.compile({ topic: "attention" });
await job.waitUntilDone({ timeoutMs: 600_000 });

Both clients cover the full /v1 surface: search, provenance, graph queries, the compiled wiki, captures/writes, and async compile/lint jobs with a code-driven error taxonomy. Docs: Python · TypeScript · HTTP API guide. Go programs can skip HTTP entirely — see Embedding in a Go program.

Copy-paste framework integrations, exercised in CI against a live server:

• examples/langgraph/ — memory-backed LangGraph nodes (Python client): retrieval with the uncompiled_sources → topic-compile pattern, plus capture.
• examples/vercel-ai-sdk/ — search, graphQuery, provenance as Vercel AI SDK tools (TypeScript client); edge-deployable.

To call the same tools from your own Go process — no subprocess, no stdio or port to manage — use pkg/sagewiki with mcp-go's in-process transport:

srv, err := sagewiki.NewServer("/path/to/wiki")  // project must already exist
if err != nil {
    return err
}
defer srv.Close()  // the caller owns the DB handle here

cli, err := client.NewInProcessClient(srv.MCPServer())
if err != nil {
    return err
}
defer cli.Close()

if err := cli.Start(ctx); err != nil {
    return err
}
if _, err := cli.Initialize(ctx, mcp.InitializeRequest{}); err != nil {
    return err
}

res, err := cli.CallTool(ctx, mcp.CallToolRequest{
    Params: mcp.CallToolParams{
        Name:      "wiki_search",
        Arguments: map[string]any{"query": "attention", "limit": 5},
    },
})

The project must already exist and the caller owns the database handle, so Close is required — unlike serve, nothing else closes it. Logs go to the host's stderr, and initialize reports the sage-wiki build version (dev from a plain go build); call sagewiki.SetVersion at startup to report your own version string instead.

The package is experimental while sage-wiki is pre-1.0: the Go signatures are meant to stay put, but tool names, argument schemas, and config.yaml layout can change in any release. Pin a version.

• Storage — SQLite by default (single file, zero config); PostgreSQL + pgvector for server deployments. Switching and pool sizing: Storage Backends.
• Observability — structured log snapshots and an opt-in /metrics endpoint: Metrics.
• Structured outputs — LLM extraction passes use each provider's native mechanism (Anthropic tool-use, OpenAI response_format, Gemini responseSchema) with a validating fence-strip fallback.
• Credentials — subscription tokens live in the OS keychain where available; run sage-wiki auth migrate once to move file-stored credentials over. Subscription Auth.
• Configuration — every key, annotated, with multi-provider recipes and the serve-mode compile worker: Configuration.
• Entity resolution — 0.85 auto-apply, exactly reversible with --unlink; see Graph memory above.
• Custom relation/entity types — extend built-ins or add your own (ontology.relation_types), with multilingual synonyms and type restrictions: Configurable Relations.
• Output trust — query outputs quarantine until grounded, confirmed by consensus, or manually promoted: Output Trust.
• Search tuning — chunking, expansion, re-ranking, graph expansion, and opt-in ANN: Search Quality.

sage-wiki tracks token usage and estimates cost for every compile. Prompt caching (default on) reuses system prompts across calls within a compile pass — Anthropic and Gemini cache explicitly, OpenAI caches automatically — saving 50-90% on input tokens. Batch API (Anthropic, OpenAI, and Gemini) halves cost for large compiles:

sage-wiki compile --batch       # submit batch, checkpoint, exit
sage-wiki compile               # poll status, retrieve when done

compile --estimate previews cost; compiler.mode: auto batches automatically past a threshold. Details: Configuration.

Tiered compilation routes each source by type and usage instead of LLM-compiling everything:

For large vaults: index everything at Tier 1 (a 100K-doc vault in ~5.5 hours), then compile on demand — auto-promotion, backpressure, and code parsers are covered in Large Vault Performance.

Packs bundle ontology types, prompts, and skill triggers for a domain. Eight bundled packs work offline:

sage-wiki init --pack academic-research applies one at init; pack install <name|url> adds more. Creating and publishing packs: CONTRIBUTING.

Handle any file format with a script in any language (stdin → text on stdout), declared in parsers/parser.yaml behind a double opt-in — they run as unsandboxed subprocesses with timeout enforcement and environment stripping. Authoring and hardening details: CONTRIBUTING; the trust-boundary discussion: Team Setup.

Three sharing patterns — git-synced, shared server, hub federation — plus team trust review and cost management: Team Setup.

Two suites answer different questions. Full detail: eval/benchmarks/REPORT.md and eval/REPORT.md.

Memory benchmarks — can it answer questions about a long conversation? Published datasets, LLM-judged, using the prompts and procedure from mem0ai/memory-benchmarks with sage-wiki as the backend (gpt-5 answerer/judge, scoped samples):

Not a like-for-like ranking: mem0 runs their managed platform on full question sets, these are scoped samples (±4–5pp), and the compile pipelines differ. The caveats are spelled out in the report.

Quality + performance eval — is the wiki well-formed and fast? Runs on any compiled wiki, no API keys, seconds. Median across 10 real wikis: overall 87.4%, fact extraction 100%, search recall@10 100%, cross-reference integrity 100%. In-process retrieval: FTS5 top-10 0.035 ms, hybrid RRF 4.9 ms, graph BFS 0.001 ms.

python3 eval/eval.py .                      # quality + perf on your wiki
python3 -m pytest eval/eval_test.py -q      # harness self-tests

• Storage: SQLite with FTS5 (BM25 search) + BLOB vectors (cosine similarity) + compile_items table for per-source tier/state tracking
• Ontology: Typed entity-relation graph with BFS traversal and cycle detection
• Search: Unified pipeline — document- and chunk-level FTS5 and vectors fused by weighted RRF with the ontology graph as a third channel, corpus-adaptive stopwording, title-proxy column weights, and a recency tie-breaker on documents with a known origin date. LLM query expansion and coverage-gated re-ranking are opt-in per call on the search surfaces and on by default for Q&A, which also gets 4-signal graph context expansion. Search responses signal uncompiled sources for compile-on-demand.
• Compiler: Tiered pipeline (Tier 0: index, Tier 1: embed, Tier 2: code parse, Tier 3: full LLM compile) with adaptive backpressure, concurrent Pass 2 extraction, prompt caching, batch API (Anthropic + OpenAI + Gemini), cost tracking, compile-on-demand via MCP, quality scoring, and cascade awareness. Embedding includes retry with exponential backoff, optional rate limiting, and mean-pooling for long inputs. 10 built-in code parsers (Go via go/ast, 8 languages via regex, structured data key extraction).
• MCP: 19 tools (7 read, 9 write, 3 compound) via stdio or SSE, including wiki_graph_query for provenance-cited multi-hop graph QA, wiki_compile_topic for on-demand compilation and wiki_capture for knowledge extraction
• TUI: bubbletea + glamour 4-tab terminal dashboard (browse, search, Q&A, compile) with tier distribution display
• Web UI: Preact + Tailwind CSS embedded via go:embed with build tag (-tags webui)
• Scribe: Extensible interface for ingesting knowledge from conversations. Session scribe processes Claude Code JSONL transcripts.
• Packs: Contribution pack system with 8 bundled packs, Git-based registry, install/apply/remove/update lifecycle, transactional apply with snapshot rollback, fill-only merge, and config allowlist security.
• External Parsers: Runtime-pluggable file format parsers via stdin/stdout subprocess protocol. Sandboxed execution with timeout, env stripping, and network isolation (Linux).

Zero CGO. Pure Go. Cross-platform.

MIT