Give coding agents your project's own ground truth. An agent editing a structured config can read every file in the repo and still be inferring what happens at runtime. This packages the four capabilities that close that loop — semantic lint, deterministic replay, seeded batch simulation with threshold gating, and guarded read-only queries — as MCP tools built from four small functions the adopting project writes.
pip install "groundtruth-mcp[mcp]" # or: pip install -e ".[mcp]" from a checkout
groundtruth --config examples/checkout-flow/groundtruth.toml lint broken_checkoutWhy it is worth a look
Raise one number in the bundled example — shipping.abandon_chance from 0.05 to 0.28 — and lint still exits 0 while the gate fails at 66.8% success against an 80% floor. Structurally perfect, twenty-one points of conversion gone. No schema, type system or code review catches that; a seeded batch with a declared band catches it in four seconds, on the pull request.
What is in it
Toolkit— four hooks (loader,validator,runner,simulator). Register only arunnerand you get bothreplayandsimulate.- Twelve declarative check types over a small path selector, so every issue carries the exact config path to edit (
states[3].transitions[1].to). Malformed rules fail at load time, not check time. - Deterministic simulation — nearest-rank percentiles, threshold bands as
<aggregate>:<name>expressions, a batch fingerprint, and an opt-in determinism check that fails the gate outright when a seed does not reproduce. - Read-only data access enforced by the datastore (
mode=ro+PRAGMA query_onlyon SQLite,READ ONLYtransactions on PostgreSQL), with post-fetch column redaction and<untrusted>fencing. - One
groundtruth.tomlread by both the MCP server andgroundtruth simulate --gate, so the thresholds an agent optimises against cannot drift from the ones CI enforces.
Verified
69 tests on Python 3.11–3.13, ruff and mypy clean, coverage floor 75%, and a real stdio round trip against the MCP SDK — all enforced in CI, alongside the simulation gate itself and an assertion that lint broken_checkout still exits 1.
Full notes: CHANGELOG.md · Adoption guide: docs/ADOPTION.md · 中文文档