Highlights
- Adds source-backed model and benchmark capability discovery before execution.
- Adds informed mode, output, resource, and claim selection through run-contract v2.
- Enforces end-to-end capability activation across checkpoint, wrapper, adapter, writer, validator, and visualizer.
- Adds truthful T+1 replay, predicted-vs-observed separation, and human-readable behavior analysis.
- Reframes G8 as an approved goal-driven study rather than an automatic batch expansion.
- Hardens mutable API boundaries, result-blind pilot selection, repair precedence, manifests, resumability, failure taxonomy, and delivery safety.
Compatibility
- v0.2.0 creates
embodied-run-contract/v2contracts by default. - Existing v1 contracts receive explicit migration guidance.
- A v1 contract never gains implicit capability, media, study, training, or permission authority.
- Full training remains outside the default evaluation workflow and requires a separate handoff and approval.
Validation
- 35 unit and regression tests passed.
- GitHub Actions passed on Windows and Linux with Python 3.10 and 3.12.
- Repository and Agent Skill validators passed.
- Five isolated forward-test scenarios passed: action-only, required future prediction, resource-constrained study design, training-boundary handoff, and append-only repair lifecycle.
- CI evidence: https://github.com/Yinzhanqing/embodied-eval-automation/actions/runs/30804419172
Installation
Install the tagged skill directory with Codex skill-installer:
https://github.com/Yinzhanqing/embodied-eval-automation/tree/v0.2.0/skills/embodied-eval-automation
The tagged URL is recommended for reproducible use. The main branch may contain unreleased changes.
Scope
This release contains workflow instructions, templates, and standard-library validation tools. It does not include model weights, benchmark code, datasets, generated episodes, videos, or private infrastructure details.
Full Changelog: v0.1.0...v0.2.0