Skip to content

v3.0.8

  • v3.0.8
  • 2b060f2

  • Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    GPG key ID: B5690EEEBB952194
    Verified
    Learn about vigilant mode

  • Choose a tag to compare

    View all tags
v3.0.8: feat(validator): report each episode's end reason and duration (#406)
  • v3.0.8
  • 2b060f2

  • Choose a tag to compare

    View all tags

  • Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    GPG key ID: B5690EEEBB952194
    Verified
    Learn about vigilant mode
@that-guy-wade that-guy-wade tagged this 08 Nov 21:47 
A world-pack episode the validator force-stops (finish_all) is graded "did not
finish" with no record of why. The validator already knows: the run key's spend
limit (key_exhausted), the sandbox's per-task TIMED_OUT/FAILED/SUCCESS, or no
per-task result at all. Each episode result now carries end_reason (finished,
budget_exhausted, timeout, agent_error, agent_stopped, sandbox_stopped,
environment_fault) and wall_seconds (agent's first call to the episode's close;
0 if never started). Diagnostic only: scoring is unchanged.

Both travel as extra keys on the episode-result entry; a Backend without the
fields ignores them.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Assets 2
Loading