EvalFence v0.3.0

EvalFence v0.3.0 adds an independent fail-closed custom-agent boundary contract while preserving the v0.1 interval/metric and v0.2 keyed-manifest contracts.

Included:

• evalfence.agent-boundary.v1 and the audit-agent-boundary CLI command.
• Declared checks for input allowlists, task/scoring-oracle classifications, mutable source aliases, read/write capability groups, callable and mediated counts, and baseline-snapshot order.
• Stable findings for oracle exposure, unapproved inputs, unmediated write callables, pre-baseline constructor code, mutable aliases, invalid names/digests/policies/counts, contradictory order, aggregate count overflow, and inconsistent absent-input declarations.
• A source- and runtime-bound adapter for microsoft/STATE-Bench@4efcbf2d4fe60df04878859b692d9391f3d5b33a with exact Git blob registration, AST guards, a clean-worktree gate, Python UTF-8 normalization, three offline domain observations, and one deterministic no-API causal control.

Registered STATE-Bench observations at the pinned revision:

• Three public test tasks expose 14 state-requirement items, nine task-requirement items, and 20 callable write handlers at the declared custom-agent constructor boundary.
• The observed event order in each domain is constructor, baseline snapshot, first agent turn, final snapshot.
• One bounded shopping control reaches deterministic state score 1 with two normal harness-executed tool calls when using the supplied state requirements; removing only state_requirements produces score 0, no calls, no errors, and an empty state diff.
• EvalFence emits three EF301, three EF303, one EF304, and two EF305 findings for the registered declaration.

Verified at commit de3c49e6a6960fbafea59381a69439558a239e0e:

• main CI run 30401556015 passed quality, Windows, macOS, and ContextBench jobs.
• main SWE-bench manifest run 30401556000 and STATE-Bench boundary run 30401556102 passed.
• tag CI run 30401692397, tag SWE-bench manifest run 30401692383, and tag STATE-Bench boundary run 30401692378 passed.
• Rust formatting, Clippy with warnings denied, all-target tests, release build, all three source-bound controls, exact evidence reproduction, and generated-evidence privacy gates passed in public CI.

Boundaries:

• Input and oracle classifications are adapter declarations; the Rust core does not infer them or prove that an adapter observed the real runtime.
• The STATE-Bench control is not a protocol-compliant or official benchmark score, model evaluation, security assessment, defect report, cheating claim, or leaderboard reproduction.
• Removing one context field does not isolate arbitrary same-process Python. Filesystem access, task-ID-based lookup, and other same-process hooks remain outside this control.
• The control does not establish that any submitted agent used the declared fields or callables, or that any public result was affected.
• This release is not a production deployment, external review, adoption, or endorsement signal.
• STATE-Bench remains copyright its authors under the MIT License; EvalFence does not vendor its source, tasks, environments, prompts, or trajectories.
• The repository was built with substantial AI assistance under tiramitree.

No precompiled binaries are attached. The release contains GitHub-generated source archives only.

EvalFence v0.2.0

EvalFence v0.2.0 adds an independent fail-closed keyed-manifest integrity contract while preserving the v0.1 interval, metric, and batch contracts.

Included:

• evalfence.keyed-manifest.v1 and the audit-manifest CLI command.
• Stable findings for empty or duplicate IDs, conflicting valid payload digests, allowlist and required-coverage violations, reported-count mismatches, invalid digest syntax, and order-dependent last-write-wins collapse.
• Reports use per-case group ordinals in generated findings and witnesses instead of serializing manifest record or policy IDs. Caller-controlled case_id is still copied to reports and requires its own privacy review.
• A hash- and AST-bound synthetic control for SWE-bench/SWE-bench@f7bbbb2ccdf479001d6467c9e34af59e44a840f9, including exact prediction-key constants and loader/consumer syntax guards.

Verified at commit ebb91ec:

• main CI run 30355886136: quality, Windows, macOS, and ContextBench jobs all passed.
• main SWE-bench manifest run 30355885891 passed.
• tag CI run 30356013022 and tag SWE-bench manifest run 30356011144 passed.
• Rust formatting, Clippy with warnings denied, all-target tests, release build, both source-bound controls, and generated-evidence privacy gates passed in public CI.
• Reversing two synthetic same-ID records with different payload digests changed the simulated last-write-wins survivor while retaining the registered duplicate, conflict, and order-dependence findings.

Boundaries:

• The SWE-bench control does not import or execute SWE-bench and uses no Docker, dataset, model, GPU, API, leaderboard submission, or paid service.
• It does not establish that any real prediction file contains duplicates, that a published score changed, or that SWE-bench is defective or nonconforming.
• EvalFence does not authenticate adapter-selected payload bytes or prove arbitrary upstream data flow.
• This release is not a model-quality or harness-quality benchmark, sandbox, production deployment, external review, adoption, or endorsement signal.
• The repository was built with substantial AI assistance under tiramitree.

No precompiled binaries are attached. The release contains GitHub-generated source archives only.

EvalFence v0.1.0

EvalFence v0.1.0 is the first source-only release of the fail-closed auditor for declared agent-benchmark metric and provenance contracts.

Included:

• Rust CLI for single-case and bounded JSONL batch audits.
• Stable findings for missing prediction/gold declarations, gold-derived or unapproved prediction sources, unsafe relative paths, interval/count errors, and supplied precision/recall/F1 mismatches.
• Hash-bound ContextBench case-study adapter at upstream commit 1436c28a8eb95496da4ea69ad458b9f8a8eb7d61.
• Generic public privacy gate, threat model, contract, fixtures, and contribution templates.

Verified at commit 50e221b:

• main CI run 30348232629: quality, Windows, macOS, and ContextBench jobs all passed.
• tag CI run 30348374895: the same four jobs all passed.
• Rust formatting, Clippy with warnings denied, all-target tests, and release build passed in public CI.
• The pinned 500-row case study generated 421 nonempty simulated fallback cases and reproduced the registered observations and violation counts.

Boundaries:

• EvalFence checks adapter-declared contracts; it does not prove arbitrary upstream data flow.
• The case study does not establish that any submission omitted model_patch or that aggregate/leaderboard metrics were affected.
• This is not a model-quality or harness-quality benchmark, sandbox, production deployment, external review, adoption, or endorsement signal.
• The repository was built with substantial AI assistance under tiramitree.

No precompiled binaries are attached. The release contains GitHub-generated source archives only; binary distribution remains deferred until a separate third-party Rust dependency license/NOTICE audit.