Grimoire 📖

An illuminated logbook for your MLflow experiments. Point it at a tracking server and it keeps a local mirror of your runs, works out what you are comparing and what is going wrong, and has Claude write narrative chapters: conclusions, pitfalls, and suggested next runs, rendered as a spellbook you can annotate, update, export, and share with colleagues.

Claude is used through the claude CLI (your existing subscription), never through an API key, and only when you explicitly ask for a chapter.

Quickstart

Requirements: Python ≥ 3.11 (managed by uv), the Claude Code CLI logged in, and an MLflow server.

uv sync
cp grimoire.toml.example grimoire.toml   # edit: tracking_uri
uv run grimoire sync                     # mirror runs into the data dir
uv run grimoire divine                   # facts only, no LLM: statuses, comparisons
uv run grimoire scribe                   # Claude writes the chapters
uv run grimoire serve                    # open http://127.0.0.1:7777

From the web UI you can do everything end-to-end: Begin a New Entry → pick experiments and/or individual runs (the selection basket spans experiments) → Inscribe. Each tome page has Check for updates (a free diff against MLflow, no LLM) and a user-confirmed Write updated chapter, plus run renaming, an editable alias legend, per-run comments the Scribe treats as ground truth, and exports (markdown, CSV, LaTeX, print-to-PDF).

Renaming a run rewrites it everywhere at once: the tables, the comparison groups, the eval tables, the chapter prose, the glance cards, the changelog, and the exports. Chapters are immutable once written, so the rename is applied to their text on the way out rather than by rewriting history.

Where the data lives

Everything Grimoire produces sits in one directory, the data dir:

<data_dir>/
├── grimoire.db     # the mirror + all chapters, notes, settings (SQLite)
└── chronicle/      # chapters also written out as plain markdown

By default the data dir is the folder containing grimoire.toml. Override it (highest wins):

1. --data-dir flag on any command: grimoire --data-dir /path serve
2. GRIMOIRE_DATA_DIR environment variable
3. data_dir = "/path" in grimoire.toml

To move an existing grimoire: stop the server, mv grimoire.db* chronicle/ <new-dir>/ (the glob catches SQLite's -wal/-shm sidecars if present), set data_dir in grimoire.toml, restart. All state travels with those two entries; nothing else is written anywhere.

Pacing MLflow syncs

A first sync fetches every metric's full history for every run (there's nothing to diff against yet), which can load a shared MLflow tracking server even for a handful of runs if they log verbosely. Pace it in grimoire.toml:

[chronicler]
metric_fetch_workers = 2        # concurrent get_metric_history calls
metric_fetch_delay_ms = 200     # pause before each call, in milliseconds

Lower workers / higher delay is gentler on the server but slower to sync; these are the defaults already. grimoire sync and the web UI's Check for updates / chapter regeneration print exactly which run/metric is being fetched as they go.

Sharing with colleagues

Put the data dir on a shared disk and serve a read-only copy:

grimoire serve --data-dir /shared/grimoire-data --mode edit                # you
grimoire serve --data-dir /shared/grimoire-data --mode viewer \
               --host 0.0.0.0 --port 7800                                  # them

Viewer mode allows reading, exporting, and pinning signed margin notes; everything that mutates tomes or spends money returns 403. Edit mode refuses non-loopback addresses unless you pass --allow-remote-edit, because anyone reaching an edit server can summon the Scribe on your subscription.

MCP: let your Claude consult the grimoire

grimoire mcp serves the logbook to local Claude clients (Claude Code, the desktop app) over stdio only. It is read-only by construction (PRAGMA query_only), with no network listener and no MLflow or claude access. Tools: list_tomes, read_tome, find_runs.

Register it, pointing at a specific checkout and a specific data dir:

claude mcp add grimoire -- uv run --directory /path/to/grimoire-repo \
    grimoire --config /path/to/grimoire.toml --data-dir /path/to/data mcp

--directory picks the code (and the default config, if --config is omitted: grimoire.toml in that folder); --data-dir picks which grimoire it reads (skip it if data_dir is already set in that grimoire.toml: the flag only needs to be there to override it). Both flags are optional when the repo's own config and data dir are what you want. Then ask Claude things like "what did the encoder-ablation tome conclude?" or *"find runs matching resnet.gated".

No local checkout? Run straight from GitHub with uvx instead of --directory:

claude mcp add grimoire -- uvx --from "git+https://github.com/<you>/grimoire@v0.1.0" \
    grimoire --config /path/to/grimoire.toml mcp

Pin @<tag-or-commit> rather than a branch: claude mcp add bakes this command into your MCP config, so it runs automatically every time a session starts. Tracking a mutable branch means a future push to that branch changes what silently executes on your machine, including grimoire's own read-only safeguards.

For developers

Four layers, deliberately decoupled; the LLM only ever sees pre-digested facts, never raw curves. The full guide lives at docs/index.html (rendered by GitHub Pages).

module	role
chronicler.py	incremental MLflow → SQLite mirror; per-metric change detection, never re-fetches unchanged data
diviner.py	pure numpy/SQL analysis: effective statuses (MLflow's RUNNING lies), comparison axes/cohorts/groups; results cached per data-fingerprint in run_digests
markup.py	markdown to HTML, re-serialized from a tag allowlist (everything printed with |safe goes through here)
scribe.py	claude -p --json-schema subprocess; schema-enforced chapters, previous chapter fed back as memory, skip-if-unchanged via payload hash
tomes.py	one "bound view" over experiments and/or hand-picked runs: the sync → divine → scribe pipeline shared by CLI and server
server.py	FastAPI + Jinja2 UI; views.py pure presentation helpers; export.py table/document exports
mcp_server.py	the read-only stdio MCP bridge
db.py / config.py / settings.py	schema with guarded additive migrations; grimoire.toml loading; runtime key/value store

Conventions worth keeping:

• The Scribe eats structured facts. New signals belong in the Diviner (as findings or payload fields), not as prose in the prompt.
• Money is explicit. Every claude invocation traces back to a user action (button, CLI command); nothing calls the LLM on its own.
• Migrations are additive: append nullable columns to _MIGRATIONS in db.py; never rewrite tables.
• Every run belongs to at least one comparison group. Strict groups are maximal cliques of "differs on at most 3 axes"; a run that no clique reaches gets a looser pass, then a nearest-neighbour pairing. Git commit and branch count as axes, so they land in "what they share" or "how they differ".
• Anything the researcher writes is keyed by something stable. Renames key on the run label, not its alias; group captions key on a hash of the group's members, because group numbering shifts as runs come and go.
• Accent discipline in the UI: gold = ornament, oxblood = failure/danger, cornflower (--accent) = important information.

Security notes

• grimoire.toml (internal hostnames) and the data dir (your research notes) are gitignored; commit grimoire.toml.example instead.
• The web UI has no authentication. Share viewer mode only; it blocks all mutating routes (including the MLflow proxy endpoints, which would otherwise be an SSRF vector) except margin notes.
• The MCP server is stdio-only and read-only; it cannot write, reach the network, or spend money. Research data never leaves the machine it runs on.
• Rendered markdown (chapters, commentary) goes through markup.py, which re-serializes the HTML from an allowlist and drops link schemes that can execute. Chapters quote MLflow run names verbatim, so whoever can name a run is an untrusted input to every reader's page.