Skip to content

Evaris v0.4.0

Choose a tag to compare

@akkaraponph akkaraponph released this 17 Nov 06:47
· 14 commits to main since this release

Acceptance mode, Git import, frozen reviewer files, per-criterion reports and
Invaris campaign verification, closing the gaps testers found when evaluating a
real Node project.

  • "mode": "acceptance" evaluates a single arm with any ID; reports carry
    mode and no comparison, and compare refuses the store.
  • allowedChanges entries are glob patterns (*, ?, [...], whole-segment
    **), and the new protectedPaths always fails authorization.
  • limits.maxEntries raises the 2000-entry snapshot bound (up to 200000), and
    command-exit criteria accept absolute dependencies such as node_modules,
    linked beside the scratch copy.
  • Import accepts finalSource of kind git: Evaris exports the commit with
    git archive, adds the prepared generated files and records the
    deliveredRevision (evaris.run/v1alpha2).
  • reviewerFiles freezes reviewer scripts and hidden tests; evaluate refuses
    to grade when one changed.
  • Reports move to evaris.report/v1alpha2 with every criterion's status,
    reason and message, and evaris show --evaluation ID [--criterion ID]
    prints one evaluation with bounded command output.
  • Invaris support widens from ee92f3c to v0.3.0, v0.4.0 and 0.5.x. Evaris
    probes invaris --version, records it, and refuses an untested release.
    Findings accept an optional line.
  • Verification can gate a case on an Invaris campaign ("action": "campaign"
    with frozen files). Every experiment in the report must have run a frozen
    harness with its frozen digest; a harness read from the agent's tree is
    EVIDENCE_CORRUPT.
  • Compatibility: 0.3.0 refuses the new optional fields, v1alpha2 runs and
    reports. Invaris 0.5.0 reports that carry finding line need Evaris 0.4.0.

Bundled controls are synthetic, not real model measurements. Commands and Invaris guards are not sandboxed; trusted command grading requires --allow-commands. Command dependencies are outside the integrity check and their contents are not hashed. Campaign reports are checked for identity, decision, campaign digest and harness digests, not decoded in full. The earlier three-trial pilot produced NOT_COMPARABLE comparisons and does not establish agent or instruction effectiveness. Windows is not supported.