Architecture

Architecture

A daemon (scholion-daemon, axum, port 7420) owns all state. The scholion / av CLI and the MCP tools are thin HTTP clients of its /v1 API.

 HOST                                  DOCKER (or lite mode on the host)
 ┌───────────────────────────────┐     ┌───────────────────────────────────────────────────┐
 │ agents (Claude Code, Codex …) │     │ scholion-daemon :7420  (axum, owns all state)     │
 │   edit files in               │ HTTP│  ├─ routes, lifecycle operations                  │
 │   ~/ws/<repo>/<task>/  ◀──────┼bind─┼─▶├─ scholion-vcs   (jj-lib)  ─▶ jj store          │
 │                               │mount│  ├─ scholion-store (sqlx)    ─▶ postgres 16       │
 │ scholion / av CLI ────────────┼JSON▶│  ├─ scholion-index (tree-sitter) ─▶ postgres      │
 │   finds its task via .av-task │     │  ├─ scholion-pack  (context packer)               │
 └───────────────────────────────┘     │  └─ outbox worker ─▶ embedder (TEI / Ollama)      │
                                       │                   ─▶ qdrant                       │
                                       └───────────────────────────────────────────────────┘

Crates

Crate	Role
scholion-core	Shared types, errors, the Vcs / IntentStore / Semantic traits every unit talks through, the lifecycle state machine and the pure routing rules.
scholion-vcs	The only crate that touches jj-lib. Imports a git repo, creates a jj workspace per task, snapshots, diffs and reads files by commit id.
scholion-store	Postgres IntentStore: tasks, records, anchors, conventions, ledger, receipts, backlog, outbox. Embedded migrations.
scholion-index	Tree-sitter extraction (Rust, TypeScript, Python): symbols, reference graph, skeletons, code chunks; incremental by content hash. A task reads main's index plus its own changes.
scholion-semantic	Embedder (TEI, Ollama, a scripted fake) and VectorIndex (Qdrant, an in-memory fake).
scholion-pack	The context packer: seeds → candidates → scoring → budget → tiers → ledger and receipt. Golden-file tests.
scholion-daemon	Configuration from environment variables, HTTP routes under /v1, lifecycle operations, routing and backlog, the MCP server at /mcp, the dashboard at /ui, the outbox worker.
scholion-cli	The scholion binary and its av alias: subcommands that mirror the API, lite mode, agent wiring, hooks, av watch.
scholion-agentrun	Runs an agent CLI as a child process with a deadline, logs and usage totals. Used by the benchmark harness.
scholion-bench	The benchmark harness (see Benchmarks).
scholion-testutil	TestDb: a throwaway Postgres database per test.

Where state lives

Store	Holds	Notes
jj repo	File contents, changes, operation log	Each repo is a git import. A change id is the jj commit id.
Task workspaces (AV_WS_ROOT)	Each task's working copy	Agents edit here on the host; clients are given host paths.
Postgres	Tasks, records, anchors, conventions, ledger, receipts, code index, backlog, outbox	Source of truth.
Qdrant	Vectors for records, conventions and code chunks	Derived; filled from the outbox.

Main paths

• Start a task: create the jj workspace at trunk, write the intent record and return the first pack. The task reads main's code index plus its own changes.
• Checkpoint: snapshot the workspace (a new jj commit), diff it against the task's base, re-index only the changed files, store the new head and the touched paths.
• Record: compute the anchored code's hashes at the current head, then write the record, its anchors and hashes, any cited receipt and an outbox row in one transaction.
• Context: gather candidates (Postgres, the index, jj file reads, and semantic search if configured), score, fit the budget, render, update the session ledger, store the receipt.
• Outbox worker: every second, claims up to 32 outbox rows (FOR UPDATE SKIP LOCKED, one worker per database), embeds them in batches and writes them to Qdrant. A failed row is retried with backoff and never blocks the others.

Concurrency

• Per task: writes (checkpoint, record, submit, validate, sync, handoff, claim, merge) take an in-process lock for that task, so a slower request never stores an older head after a newer one.
• Fencing: a claim is one conditional update that increments the task's generation; writes carrying an older generation are refused. Agents are told apart by --agent / AV_AGENT.
• Per repo: jj mutations hold a per-repo mutex. Reads address immutable commits by id and take no lock, so packs never wait behind another agent's snapshot.
• In Postgres: status changes are conditional updates (of two racing submit calls exactly one wins); outbox rows are claimed with SKIP LOCKED.

Failure behaviour

• Qdrant or the embedder down: packs fall back to structure and full-text (degraded: semantic); the outbox keeps the work and the worker catches up later.
• An index write and its outbox row commit together, so Qdrant can lag but never miss an entity.
• Backup remote down: the merge still succeeds; the push is retried in the background with backoff.
• Workspace creation fails: the task is marked abandoned rather than left open.

HTTP API

Everything is under /v1 (tasks, records, context, repos, files, diff, log, agents, backlog, events). /healthz and /ui are public; every other route requires the bearer token when AV_TOKEN is set. The CLI and MCP tools are the documented clients; the routes themselves are in crates/scholion-daemon/src.