Skip to content

EvalFence v0.1.0

Choose a tag to compare

View all tags
@tiramitree tiramitree released this 28 Jul 09:53
v0.1.0
50e221b

EvalFence v0.1.0 is the first source-only release of the fail-closed auditor for declared agent-benchmark metric and provenance contracts.

Included:

  • Rust CLI for single-case and bounded JSONL batch audits.
  • Stable findings for missing prediction/gold declarations, gold-derived or unapproved prediction sources, unsafe relative paths, interval/count errors, and supplied precision/recall/F1 mismatches.
  • Hash-bound ContextBench case-study adapter at upstream commit 1436c28a8eb95496da4ea69ad458b9f8a8eb7d61.
  • Generic public privacy gate, threat model, contract, fixtures, and contribution templates.

Verified at commit 50e221b:

  • main CI run 30348232629: quality, Windows, macOS, and ContextBench jobs all passed.
  • tag CI run 30348374895: the same four jobs all passed.
  • Rust formatting, Clippy with warnings denied, all-target tests, and release build passed in public CI.
  • The pinned 500-row case study generated 421 nonempty simulated fallback cases and reproduced the registered observations and violation counts.

Boundaries:

  • EvalFence checks adapter-declared contracts; it does not prove arbitrary upstream data flow.
  • The case study does not establish that any submission omitted model_patch or that aggregate/leaderboard metrics were affected.
  • This is not a model-quality or harness-quality benchmark, sandbox, production deployment, external review, adoption, or endorsement signal.
  • The repository was built with substantial AI assistance under tiramitree.

No precompiled binaries are attached. The release contains GitHub-generated source archives only; binary distribution remains deferred until a separate third-party Rust dependency license/NOTICE audit.

Assets 2
Loading