Skip to content

Releases: ZhenGtai123/groundtruth-mcp

v0.1.0 — first public release

Choose a tag to compare

@ZhenGtai123 ZhenGtai123 released this 17 Aug 14:19

Give coding agents your project's own ground truth. An agent editing a structured config can read every file in the repo and still be inferring what happens at runtime. This packages the four capabilities that close that loop — semantic lint, deterministic replay, seeded batch simulation with threshold gating, and guarded read-only queries — as MCP tools built from four small functions the adopting project writes.

pip install "groundtruth-mcp[mcp]"     # or: pip install -e ".[mcp]" from a checkout
groundtruth --config examples/checkout-flow/groundtruth.toml lint broken_checkout

Why it is worth a look

Raise one number in the bundled example — shipping.abandon_chance from 0.05 to 0.28 — and lint still exits 0 while the gate fails at 66.8% success against an 80% floor. Structurally perfect, twenty-one points of conversion gone. No schema, type system or code review catches that; a seeded batch with a declared band catches it in four seconds, on the pull request.

What is in it

  • Toolkit — four hooks (loader, validator, runner, simulator). Register only a runner and you get both replay and simulate.
  • Twelve declarative check types over a small path selector, so every issue carries the exact config path to edit (states[3].transitions[1].to). Malformed rules fail at load time, not check time.
  • Deterministic simulation — nearest-rank percentiles, threshold bands as <aggregate>:<name> expressions, a batch fingerprint, and an opt-in determinism check that fails the gate outright when a seed does not reproduce.
  • Read-only data access enforced by the datastore (mode=ro + PRAGMA query_only on SQLite, READ ONLY transactions on PostgreSQL), with post-fetch column redaction and <untrusted> fencing.
  • One groundtruth.toml read by both the MCP server and groundtruth simulate --gate, so the thresholds an agent optimises against cannot drift from the ones CI enforces.

Verified

69 tests on Python 3.11–3.13, ruff and mypy clean, coverage floor 75%, and a real stdio round trip against the MCP SDK — all enforced in CI, alongside the simulation gate itself and an assertion that lint broken_checkout still exits 1.

Full notes: CHANGELOG.md · Adoption guide: docs/ADOPTION.md · 中文文档