Skip to content

Evaris v0.3.0

Choose a tag to compare

@akkaraponph akkaraponph released this 16 Nov 07:21
· 17 commits to main since this release

Workspace initialization, npm distribution, clearer command and import failures,
and command criteria that tolerate build output. Records keep their v1alpha1
schema versions; the one new field is optional.

  • evaris init creates a local workspace, and common commands use local defaults.
  • The evaris-cli npm package distributes native Linux and macOS binaries.
  • command-exit criteria accept an optional snapshotExclude list of relative
    paths, such as .next, that the scratch manifests skip. A build that leaves
    more than 2000 entries no longer turns a passing command into an ERROR.
  • A command criterion that ends in ERROR now carries the specific reason
    (LIMIT_EXCEEDED, PATH_UNSAFE) instead of IO_ERROR, and a message in
    evaluation.json that names the failing step, the command's exit code, the
    kind of limit and a fix. Messages come from a fixed catalogue with no paths,
    command text or oracle values; any other grader text is still cleared.
    report.md does not show them yet.
  • INVALID_CONFIG for agentClaim lists the accepted values, and
    REVISION_MISMATCH on import says which identity field differs and what it
    must be. docs/workflow.md documents both fields.
  • Compatibility: an oracle that uses snapshotExclude, and a preparation frozen
    from it, are refused by 0.2.0 and earlier as an unknown field. Oracles without
    it, and every other record, are unchanged. Invaris support is still only the
    pinned revision ee92f3c.

Bundled controls are synthetic, not real model measurements. Commands and Invaris guards are not sandboxed; trusted command grading requires --allow-commands. Invaris is optional and pinned to source ee92f3c00c1b59b2f576bad05569fb6eafb83097. The earlier three-trial pilot produced NOT_COMPARABLE comparisons and does not establish agent or instruction effectiveness; grading was reproduced by an independent AI operator, not a second person. Windows is not supported.