Skip to content

Contributing & Codebase Map

Jump to bottom
[Muji] Tan Le edited this page Aug 31, 2026 · 3 revisions

Contributing & Codebase Map

Rulebook: CONTRIBUTING.md (setup, test loops, commit conventions, release flow). This page is the tour guide — where things live and where to start reading when you want to change something.

TL;DR — getting started

pip install -e ".[dev]"   # pytest + watchdog + build + ruff + grpcio-tools
pytest -m core            # <3s smoke (one test per core function)
pytest                    # full suite (CI runs this)

Pre-commit hook: scripts/install-dev-hooks.sh sets core.hooksPath to run ruff on staged .py files.

Before tagging a release: walk docs/release-checklist.md — each item maps to a real failure mode hit in this project's history.

The 5-layer mental model

cairn is a layer cake. Every layer has a CLI mirror and an MCP mirror over the same store.

        ┌─────────────────────────────────────────┐
agents →│  MCP server (28 tools, stdio + SSE)     │← humans (cairn CLI)
        └─────────────────────────────────────────┘
                          │
   ┌──────────┬───────────┼───────────┬──────────┐
   ▼          ▼           ▼           ▼          ▼
 graph    compass       memory     knowledge    wiki
(9 tools)(+ KB, 5)    (8 tools)   (5 tools)   (generator)
   │
   ▼
SQLite (.kg) + .knowledge/ markdown bundle, under ~/.cairn/<workspace-key>/

The one design rule that explains most of the codebase:

cairn never calls an LLM itself. Where synthesis helps (compass/wiki generation), it queues a task; a deterministic critic fact-checks every result against the graph before it lands.

Where to start reading by intent

I want to… Start in Key files
Add / fix an MCP tool src/cairn/mcp_server/ tools_{graph,compass,memory,knowledge}.py, server.py (_EXPECTED_TOOL_COUNT = 27 boot assertion)
Add a new language parser src/cairn/parsers/ _registry.py, base.py, an existing parser (e.g. kotlin.py), then graph/scanner.py:EXTENSION_MAP + graph/builder.py:get_parser
Change how edges resolve src/cairn/graph/resolver.py 5 tiers: type-aware → same-file → import-aware → same-repo → global
Change build / update behavior src/cairn/graph/ builder.py (full build), incremental.py (cairn update), dataflow.py + traversal.py (derived indexes)
Work on memory / scoring src/cairn/memory/ scoring.py (7 signals), promotion.py (raw→drafts→tribal→archived, recall), store.py
Tweak compass / wiki generation src/cairn/compass/, src/cairn/wiki/ generator.py, critic.py (fact-check gate)
Wire a new AI client src/cairn/agent_install/clients/ one module per client; register in _common.py:CLIENTS; fold paths.cairn_home_env() into the config shape (env-propagation contract) and add an env comparison to merge._already_installed for the new shape
Add an install hook src/cairn/hooks/ + agent_install/clients/<name>.py claude_hooks.py (PostToolUse → cairn update, Stop → memory capture)

Directory map (src/cairn/)

Dir What it does Don't miss
graph/ Core. Scanner, builder, resolver, traversal, embeddings, derived indexes. ~9K LOC. builder.py (build pipeline + SCIP merge), incremental.py (cairn update), resolver.py (resolution model), schema.py (migrations)
parsers/ tree-sitter parsers per language + SCIP importer. 9 languages. _registry.py (plugin scan), scip_importer.py (the source='merged' merge), kotlin.py (the operator fun invoke rewrite)
mcp_server/ The agent-facing API. 28 tools across 5 tools_*.py modules. server.py:47 (tool-count assertion), structured.py (output models)
cli/ The human-facing mirror. ~90 commands across groups. main.py (registry), agents.py / uninstall.py (install), compass.py / wiki.py (critic-gated writes)
memory/ Tribal memory: record, score, recall, decay. scoring.py (7-signal weights), promotion.py (lexical + semantic recall), store.py (TIER_DIRS)
compass/ Per-module 25–35 line navigation guides. generator.py, critic.py (blocks writes on failure), router.py (ask_compass)
wiki/ Wiki generation: deterministic stats page (generator.py) + the agent-decoupled pipeline — planner (catalog.py), manifest (manifest.py), queue/promotion pipeline (pipeline.py), refine-catalog validator (refine.py), Sources-footer parser (sources.py). pipeline.py:run_wiki_generate (the MCP/CLI entry)
knowledge/ The OKF business-doc store. store.py, workflow.py (doc_type="workflow")
okf/ Open Knowledge Format: concept + front-matter + conformance + provenance. concept.py, conformance.py
llm/ Task queue — the only LLM boundary. tasks.py (kinds, lifecycle), client.py (FileQueueBackend)
agent_install/ Install into AI clients (9 today). clients/ (one module each), _common.py (_INSTRUCTIONS_BODY template, cairn_home_env() wiring, install-time verification), detect.py, merge.py
agent_integration/ The shipped skill + commands + Cursor rules. skill/SKILL.md, skill/references/*.md (must stay accurate)
hooks/ post-commit / PostToolUse / session-end hooks. claude_hooks.py (the live entrypoint), git_hooks.py
retrieval/ Retriever / Fusion / Reranker protocol layer (not FTS itself). protocols.py, vector_scan.py
viz/ Mermaid / DOT / JSON renderers. renderers.py
bench/ Perf + scaling suites. perf_suite.py, scaling_suite.py
utils/ Tiny: git remote/commit helpers. git.py

Workflows worth knowing

The build → update split

  • cairn build (graph/builder.py): full parse, builds derived indexes.
  • cairn update (graph/incremental.py): diffs against git HEAD + existing graph, reindexes changed files, rebuilds derived indexes when anything changed.
  • On a fresh clone with a clean tree, cairn update sees no changes — start with cairn build.

Critic-gated writes (the safety contract)

  • compass generate / compass flow: critic failure → sys.exit(1), no write. A hallucinated symbol can never land in a compass doc.
  • wiki generate (deterministic path): critic runs, errors are printed, but the concept still writes (intentional — never silently lose the body).
  • wiki generate --llm: nothing writes without the critic — page tasks only promote to Wiki-Article concepts on a passing, footer-verified completion.
  • The LLM task queue: every submitted result runs the same deterministic critic in complete_task.

The resolution model (read this once)

Edges are labelled exact / ambiguous / unresolved. Graph tools default to precise (follow only exact); --fuzzy adds name-only candidates. See docs/examples/resolution-walkthrough.md.

Agent-driven contributions

cairn is itself built with AI agents, and ships an agent_integration/ skill for agents working in this (or any) repo. If you're an agent reading this:

  • The workspace instructions in AGENTS.md are the contract — follow them (explore-first workflow, call ask_compass before editing, impact_analysis before breaking changes, record_memory after learning something).
  • The skill references under src/cairn/agent_integration/skill/references/ must stay accurate — if you change a tool signature, update them.
  • Never read .knowledge/memory/raw/ or .knowledge/memory/drafts/.

Clone this wiki locally