What's Changed
This release introduces a portable eval package infrastructure with formal separation between repository automation and human-directed execution. The skill infrastructure now supports harness-agnostic evaluation through runner-owned and orchestrator-owned dispatch mechanisms, enabling evaluation packages to be prepared deterministically and executed by explicitly authorized external orchestrators using Codex, GitHub Copilot, OpenCode, or other runners.
Key improvements include a comprehensive eval runner conformance and regression test suite, updates to repository governance with AI/LLM evaluation automation safeguards and Karpathy behavioral guidelines, enhanced skill descriptions across all repo-managed skills, and validation tooling with suite-based matrix execution. The changes formalize the strict separation of concerns between preparation (deterministic, model-free) and execution (explicit, human-directed), with all repository automation remaining model-free while only external orchestrators may invoke model-backed runs.
Important
Repository automation, CI workflows, and all automatic preparation steps remain deterministic and model-free. Only a human-directed external Eval Orchestrator is authorized to invoke model-backed evaluation runs through the selected runner.
Sources:
- V0.9.1/evals by @gimlichael and @aicia-bot in #39
Full Changelog: v0.9.0...v0.9.1
