How Columbus Works

How Columbus Works

Columbus separates where things are from what they currently say, and only ever stores the first.

your working tree ──(tree-sitter)──▶ index: metadata + git anchors + vectors  (the chart)
        │                                          │
        │   agent: columbus search "read config"   │   (query embedded on-device)
        ▼                                          ▼
   re-parse live ◀──────────────  vector kNN, then deterministic re-rank
        │
        ▼
   ranked results + EXACT line ranges + "why relevant"  ──▶  LLM-ready

The two halves

1. The chart (the index). Built by parsing the working tree with embedded tree-sitter. It records metadata — symbol names, kinds, containers, roles (impl/test/doc), import edges — git anchors that tie entries to a known tree state, and an embedding vector per symbol/file (computed on-device) for semantic retrieval. It is a cache, not a content store: the vectors are derived, and it never holds code lines or file bodies.

2. The working tree (the truth). When you query, Columbus re-parses the relevant files live to reconstruct the exact snippet and current line range. The chart says which file and symbol; the working tree says what it says right now.

Because the "what" is never cached, an answer cannot be stale — see Never Stale: Live Reconstruction.

The pipeline, end to end

1. Discover files (respecting .columbus.json excludes and git).
2. Parse with tree-sitter → symbols, members, roles, import edges.
3. Embed each symbol/file on-device (bge-small-en-v1.5 via ONNX).
4. Store metadata + git anchors in SQLite (FTS5) and vectors in sqlite-vec (vec0).
5. Query → embed the query, retrieve nearest neighbors, re-rank with deterministic heuristics (no LLM).
6. Reconstruct snippets + exact ranges live from the working tree.
7. Project the same typed result into text, --json, or --llm.

Storage

• SQLite database (with FTS5) lives in your OS data dir, keyed by project_id — never inside the repo.
• .columbus.json (config) lives in the repo dir but is git-excluded.
• The DB stores: file/symbol metadata, the dependency graph, embedding vectors (sqlite-vec/vec0), durable memory, and epics/stories/tasks. It does not store code.

See Configuration for exact paths.

What ranking uses (no LLM)

Retrieval is semantic (vector similarity), but the final ranking, the "why relevant" text, and risk hints are deterministic — vector score blended with heuristics: exact/prefix/signature name matches, role weighting (impl over test/doc), content density. The same query (and model) always produces the same order. See Determinism & Local-Only.

What Columbus does not do

It does not orchestrate your agent, gate transitions, run guardrails, or call an LLM. It is a context server; policy lives in the agent. See Scope Boundary.

Related

Never Stale: Live Reconstruction · Determinism & Local-Only · Supported Languages