Skip to content

v0.2.0 - Informed and capability-faithful embodied evaluation

Latest

Choose a tag to compare

@Yinzhanqing Yinzhanqing released this 03 Aug 10:21

Highlights

  • Adds source-backed model and benchmark capability discovery before execution.
  • Adds informed mode, output, resource, and claim selection through run-contract v2.
  • Enforces end-to-end capability activation across checkpoint, wrapper, adapter, writer, validator, and visualizer.
  • Adds truthful T+1 replay, predicted-vs-observed separation, and human-readable behavior analysis.
  • Reframes G8 as an approved goal-driven study rather than an automatic batch expansion.
  • Hardens mutable API boundaries, result-blind pilot selection, repair precedence, manifests, resumability, failure taxonomy, and delivery safety.

Compatibility

  • v0.2.0 creates embodied-run-contract/v2 contracts by default.
  • Existing v1 contracts receive explicit migration guidance.
  • A v1 contract never gains implicit capability, media, study, training, or permission authority.
  • Full training remains outside the default evaluation workflow and requires a separate handoff and approval.

Validation

  • 35 unit and regression tests passed.
  • GitHub Actions passed on Windows and Linux with Python 3.10 and 3.12.
  • Repository and Agent Skill validators passed.
  • Five isolated forward-test scenarios passed: action-only, required future prediction, resource-constrained study design, training-boundary handoff, and append-only repair lifecycle.
  • CI evidence: https://github.com/Yinzhanqing/embodied-eval-automation/actions/runs/30804419172

Installation

Install the tagged skill directory with Codex skill-installer:

https://github.com/Yinzhanqing/embodied-eval-automation/tree/v0.2.0/skills/embodied-eval-automation

The tagged URL is recommended for reproducible use. The main branch may contain unreleased changes.

Scope

This release contains workflow instructions, templates, and standard-library validation tools. It does not include model weights, benchmark code, datasets, generated episodes, videos, or private infrastructure details.

Full Changelog: v0.1.0...v0.2.0