Sourcebook is a git-backed markdown wiki that AI coding agents read and update over MCP.
I built it because project knowledge kept getting lost between agent sessions. How a service actually works, why we picked one approach over another, what our internal terms mean, what broke last month and why: that lived in design docs, incident writeups, old chat logs, and people's heads. Every session started cold and I re-explained the same things.
Sourcebook gives that knowledge one structured place. Agents search it before starting work, update pages when they learn something, and add pages when nothing covers the topic. Documents are validated against schemas, every change is a git commit with an author, and the tooling flags pages that have gone stale or started contradicting each other.
It is aimed at a small team on a long-running project that uses coding agents daily. The wiki is a curated, bounded set of pages that someone keeps current, not an index over every file you have.
Status: an experiment, published to see whether the approach is useful to anyone else. If you would run this, or something is missing that stops you, open a Discussion.
You paste last week's incident postmortem into an ingestion session. Sourcebook proposes a plan: add the new failure mode to the affected service's system card, add a step to the incident-response process doc, define one new glossary term. Each change cites the postmortem. Nothing else is touched, because the prompts treat "already covered" as a correct answer. You apply it, and each change lands as a git commit attributed to you.
Three weeks later a teammate asks their agent how that service's retry logic works. The agent reads the system card, which arrives with a notice that its last_verified date has aged. The work the agent just did confirms the doc is still accurate, so it calls verify_document once and the clock resets, committed under the teammate's name. Nobody scheduled a documentation review.
Six document types, each with required frontmatter and required sections:
| Type | Holds |
|---|---|
| System card | How one system or service works: purpose, stack, patterns, gotchas |
| Process | A procedure someone follows, step by step |
| Decision record | An ADR: what was decided, what was rejected, the consequences |
| API reference | The parts of a library or framework API your team actually uses |
| Knowledge page | A synthesized answer worth keeping, usually promoted from a conversation |
| Source summary | What an ingested source said, and which pages it changed |
Alongside those: a glossary (terminology as data rather than prose), conventions (hard rules agents must not break), and context bundles (a named set of pages an agent loads in one call, trimmed to a token budget).
- Agents use the MCP server to search, read, write, verify, ingest, promote, and lint.
- Humans use the web UI to browse, edit, and review, with dashboards for staleness, lint findings, and recent activity.
- The CLI (
sourcebook) handles setup, validation, lint, bundles, and running the servers.
All three call the same modules, so search ranking and bundle assembly cannot drift between them (ADR-002).
This repo ships with its own wiki, so you can browse a populated instance before building anything:
git clone <this-repo> sourcebook && cd sourcebook
pip install -e .
sourcebook serve # this repo's own wiki at http://localhost:5000Not on PyPI yet, so installing from a clone is the supported path. The sourcebook CLI then works from any directory.
To start your own, run this inside your project's git repository, so doc changes get history and attribution:
cd ~/code/your-project
sourcebook init # scaffolds docs/ai/ with schemas, seed examples, glossary
sourcebook serve # web UI on :5000
sourcebook serve-mcp # MCP server on :8000 (second terminal)Then point an agent at it:
claude mcp add --scope user sourcebook --transport http http://localhost:8000/mcpDocker runs both servers under one supervisor:
docker compose up --build -dThe seed content demonstrates each document type using a small fictional product. Replace it with your own. CLAUDE.md in this repo shows the kind of agent instructions that get the wiki consulted and maintained as part of normal work.
An LLM-maintained wiki rots if nothing pushes back: duplicate pages, silent rewrites that drop hard-won detail, claims nobody re-checks. Most of Sourcebook is the machinery that resists that. It does not eliminate the problem, but it makes it visible and cheap to correct.
- Schemas are enforced as errors. An unknown
type:is rejected rather than warned about, so a frontmatter typo cannot produce a document that silently skips every other check (ADR-001). - Writing identical content does nothing. No log entry, no commit. Sources are SHA-256 tracked, so re-ingesting unchanged material is free and re-running a batch is idempotent.
- Creating a page requires ruling out the existing ones. The write path asks for evidence that the nearest pages were considered first.
- Stale pages recruit their readers. Every doc carries
last_verified. When a stale one is served over MCP it comes with a note asking the reading agent to confirm it with a singleverify_documentcall, or to fix it. Types whose lifecycle is not date-based, such as decision records, are exempt instead of being flagged forever (ADR-004). - Lint runs structurally and semantically. Orphan pages, dependency mismatches, and tag drift come from static checks. Contradictions and stale claims across documents come from an LLM pass.
- Git is the database. Every change is a commit attributed to the human who triggered it, so history, diff, blame, rollback, and review are ordinary git operations. There is nothing else to run alongside it, with the trade-offs that implies.
Run one shared instance rather than a copy per person. That gives you one live state, no push races between personal clones, and one staleness dashboard that means the same thing for everyone. Deploy the Docker setup on a server inside your network and have everyone point their agent at it.
Sourcebook has no authentication, so do not expose it to the internet. Port bindings, the SOURCEBOOK_ALLOWED_HOSTS allowlist, and reverse-proxy details are in the deployment guide.
Sourcebook ships without authentication. It is designed for a private, trusted network, where write friction would kill the contribution rate that keeps a wiki alive. Every mutating request is audit-logged (IP, user agent, timestamp; see ADR-003), and git history attributes every change, but nothing stops a request.
Do not expose this to the public internet. Deploy it inside a network boundary you trust, or put your own auth proxy in front of it. If there is real interest, an auth story is the first thing on the roadmap. Browser-driven attacks are covered separately in SECURITY.md.
Things Sourcebook does not do, so you do not have to discover them the hard way:
- Search is keyword-based. A weighted scorer over names, tags, headings, and body text. It works well when documents have good names and tags, which the schemas push you toward, but it matches words rather than meaning: a vague description of a page will not find it. Fine at hundreds of documents, unproven beyond that.
- It expects a curated corpus. Sourcebook holds pages someone is willing to keep current. It does not index an arbitrary pile of files, and it has no answer for a corpus nobody maintains.
- No authentication, by design. See the trust model above. If you cannot provide a trusted network boundary, this is not deployable for you yet.
- The LLM-backed features need an LLM. Semantic lint and the chat tab require an OpenAI-compatible endpoint and an API key you provide. Everything else runs without one: schemas, validation, structural lint, staleness, search, and the whole MCP surface.
- Fan-out ingestion is not atomic. Actions apply sequentially, so if one fails mid-batch, earlier actions are already committed. Git history is the recovery path, and the source's completion marker is only set when everything succeeded, but you can end up in a partially applied state.
sourcebook serveis Flask's development server. Fine on your own machine. Use the Docker setup, or your own WSGI server behind a reverse proxy, for anything shared.- Scale is unvalidated. The design targets a small team and a corpus of tens to hundreds of documents. Nothing has been tested on thousands of documents or high write concurrency.
- It is an experiment. Prompts, tool contracts, and schemas may change between versions without a deprecation path.
docs/ai/ is a live Sourcebook wiki documenting Sourcebook: a system card for the tool, its design decisions as decision records, a glossary, conventions, and a context bundle that packs it all for an agent. Run sourcebook --root docs/ai bundle design to see exactly what an agent would load. ADR-004 is the one I would read first, since it is my answer to how an LLM-maintained corpus avoids rotting.
| Variable | Purpose | Default |
|---|---|---|
SOURCEBOOK_ROOT |
Path to the docs/ai/ tree |
auto-discovered |
SOURCEBOOK_MCP_URL |
Public MCP URL shown in setup instructions | port-swap fallback |
SOURCEBOOK_ALLOWED_HOSTS |
Comma-separated hostnames this instance answers for; blocks DNS rebinding. * disables the check |
loopback only |
SOURCEBOOK_GIT_AUTHOR_NAME / _EMAIL |
Git identity for AI-authored commits | sourcebook-bot / sourcebook-bot@sourcebook.local |
SOURCEBOOK_DEPLOY_KEY |
SSH key path for git push | unset |
SOURCEBOOK_LLM_API_KEY |
Bearer token for LLM-backed features (semantic lint) | unset |
SOURCEBOOK_LLM_BASE_URL |
OpenAI-compatible endpoint for the chat tab | http://localhost:8000/v1 |
SOURCEBOOK_LLM_MODEL |
Model name served by that endpoint | local-model |
SOURCEBOOK_CHAT_URL |
Optional upstream for the secondary chat tab | unset |
Python 3.10+ for the MCP server, which is an MCP SDK constraint. The CLI and web UI run on 3.9+.
pip install -r requirements.txt -r requirements-dev.txt
python -m pytest tests/ -m "not docker"
ruff check .437 tests at the time of writing. CI runs ruff and the same suite on Python 3.10 and 3.12. See CONTRIBUTING.md for code layout and PR expectations.
The pattern this implements was articulated in Andrej Karpathy's LLM wiki gist: rather than re-deriving understanding from raw sources on every query, an LLM compiles them into a structured wiki once and then maintains it. Sourcebook is an independent take on that idea, focused on the maintenance half.