Skip to content

EvalFence v0.2.0

Choose a tag to compare

View all tags
@tiramitree tiramitree released this 28 Jul 11:46
v0.2.0
ebb91ec

EvalFence v0.2.0 adds an independent fail-closed keyed-manifest integrity contract while preserving the v0.1 interval, metric, and batch contracts.

Included:

  • evalfence.keyed-manifest.v1 and the audit-manifest CLI command.
  • Stable findings for empty or duplicate IDs, conflicting valid payload digests, allowlist and required-coverage violations, reported-count mismatches, invalid digest syntax, and order-dependent last-write-wins collapse.
  • Reports use per-case group ordinals in generated findings and witnesses instead of serializing manifest record or policy IDs. Caller-controlled case_id is still copied to reports and requires its own privacy review.
  • A hash- and AST-bound synthetic control for SWE-bench/SWE-bench@f7bbbb2ccdf479001d6467c9e34af59e44a840f9, including exact prediction-key constants and loader/consumer syntax guards.

Verified at commit ebb91ec:

  • main CI run 30355886136: quality, Windows, macOS, and ContextBench jobs all passed.
  • main SWE-bench manifest run 30355885891 passed.
  • tag CI run 30356013022 and tag SWE-bench manifest run 30356011144 passed.
  • Rust formatting, Clippy with warnings denied, all-target tests, release build, both source-bound controls, and generated-evidence privacy gates passed in public CI.
  • Reversing two synthetic same-ID records with different payload digests changed the simulated last-write-wins survivor while retaining the registered duplicate, conflict, and order-dependence findings.

Boundaries:

  • The SWE-bench control does not import or execute SWE-bench and uses no Docker, dataset, model, GPU, API, leaderboard submission, or paid service.
  • It does not establish that any real prediction file contains duplicates, that a published score changed, or that SWE-bench is defective or nonconforming.
  • EvalFence does not authenticate adapter-selected payload bytes or prove arbitrary upstream data flow.
  • This release is not a model-quality or harness-quality benchmark, sandbox, production deployment, external review, adoption, or endorsement signal.
  • The repository was built with substantial AI assistance under tiramitree.

No precompiled binaries are attached. The release contains GitHub-generated source archives only.

Assets 2
Loading