Skip to content

Bench v1.1.0: Governance on a Claude Code Subscription

Choose a tag to compare

View all tags
@dburks-svg dburks-svg released this 12 Jul 21:44
· 228 commits to main since this release
v1.1.0
cb2bd9d

Bench now governs itself out of the box with no API key: the new claude_code provider routes the Challenger, Defender, and Oracle through the local claude CLI in headless mode, riding your Claude Code subscription. The ledger also gained an independent meta anchor, and vetoes finally speak their documented format.

What's new in v1.1.0

claude_code provider (no API key required)

  • New BENCH_PROVIDER=claude_code dispatches each pipeline stage through the local claude CLI in headless JSON mode
  • Children spawn with BENCH_SUBPROCESS=1 so Bench's own hook fails open instead of recursing
  • Hardened per two security audits: path sanitization, prompt-injection resistance for the judge, per-stage timeouts (BENCH_CLAUDE_TIMEOUT)
  • Now the default provider for Bench's own self-governance

Ledger meta anchor

  • verify cross-checks ledger-meta.json (latest_hash, entry_count) and fails with META_MISMATCH, so a wholesale chain rewrite that is internally consistent no longer passes
  • Writer and verifier share a single META_FILENAME constant so they cannot drift

Hook output you can act on

  • Vetoes now emit the documented BENCH VETO [C-XXX]: ... format with remediation guidance
  • Oracle advisories are surfaced to the developer instead of stopping at the ledger

Test suite and CI

  • 300+ stdlib unittest tests covering the pipeline, ledger, CLI, viewer, stats, and hook, with a GitHub Actions workflow
  • Shared synthetic-chain fixture module as the single source of truth for entry shape

Hardening and fixes

  • call_model response parsing wrapped in typed error guards on every provider path (it truly never raises)
  • Silent exception fallbacks in diff handling now log to stderr (constitutional constraint C-001)
  • Diff paths resolve against the file-derived repo root instead of the current working directory
  • Citation counting and oracle validation fixes in the pipeline
  • Viewer temp file no longer deleted before the browser loads it
  • openai is a true soft dependency (only needed for BENCH_PROVIDER=openrouter); anthropic has a pinned lower bound
  • Removed the orphaned util/ package and machine-local settings from tracking

The ledger is the proof

Every one of these changes passed through Bench's own pipeline. The shipped ledger holds 149 hash-chained verdicts spanning April 22 to July 12, and python -m cli verify confirms the chain VALID with the meta anchor verified.

Install

git clone https://github.com/Nuralyn/Bench.git

See README.md for setup and configuration.

License

MIT

Assets 2
Loading