Repository navigation
Evaris v0.2.0
Evaris 0.2.0 shows the evaluation workflow on real external agent sessions. The CLI, commands and stored schemas are unchanged from 0.1.0.
- Runbook for evaluating a real external agent: docs/pilot.md (validate, prepare, run externally, import observed metadata, evaluate, report, compare), with lessons for the Claude Code CLI host: permission mode, scratchpad location, sources for observed metadata and instruction-loading detection. Documented commands use one binary path.
- A preregistered process pilot, kept private with its evidence: 12 real Claude Code CLI sessions in 3 trials (2 original Go cases × baseline/specialist), with every
observedfield taken from the host transcript. One case separated the arms descriptively (baseline failed one stated rule in 2 of 3 trials). Every comparison was NOT_COMPARABLE because specialists read the instruction file through a shell command that the preregistered loading rule did not count. This is not evidence of agent or instruction effectiveness. - An independent AI operator session reproduced all 12 evaluations and 3 comparisons from the retained evidence. No second person reproduced the grading, and the summary was reviewed by an independent AI session.
Install
Download the binary for Linux or macOS, amd64 or arm64. Verify its entry in SHA256SUMS, make it executable, and place it on PATH.
Alternatively, with Go 1.27.4 or newer:
go install github.com/akkaraponph/evaris/cmd/evaris@v0.2.0
evaris --versionVerification
Release commit: e12566ba22fa2d63fe3ecb68ed61bf3dbd38b065. Hosted Go checks cover build, vet, race tests, offline acceptance and pinned real-Invaris integration on Linux and macOS. Binaries were built with Go 1.27.4 using the reproducible build in docs/release.md; macOS and an offline Linux container produced byte-identical binaries. Linux/arm64 and macOS/arm64 binaries passed version and experiment-validation smoke checks.
Known limitations
- The pilot has 3 trials, one host and model, two small tasks and a stand-in instruction file; it is descriptive only and no comparison was possible. Host metadata is operator-attested from transcripts. No semantic review.
- Command graders and Invaris guards require
--allow-commands, execute trusted commands and are not sandboxed. - Windows and Invaris revisions other than the pinned revision are unsupported.
- Stored
v1alpha1records have no cross-minor compatibility promise; evidence stores are not migrated.
See pilot runbook, support and reproducible builds and changelog.