Skip to content

v0.12.7

Choose a tag to compare

@coder-eval-release-bot coder-eval-release-bot released this 25 Sep 20:52
· 36 commits to main since this release

Bug Fixes

  • criteria: Search past the 2000-char bound only in Bash commands
    (#196,
    3aa2db3)

  • criteria: Search the whole command in command_executed and bump harness SDKs for Opus 5.5
    (#196,
    3aa2db3)

  • criteria: Search the whole command in command_executed, not just the first 2000 chars
    (#196,
    3aa2db3)

  • docs: Drop the budget-breach trigger, cut read_only to its contract
    (#197,
    28c846e)

  • evalboard: List date-named runs flagged adhoc in the Ad-hoc section
    (#200,
    5bf4a72)

  • harbor: Declare SUPPORTS_ATIF as a ClassVar to match harbor 0.23
    (#196,
    3aa2db3)

Build System

  • deps: Bump harness SDKs and CLIs to latest
    (#196,
    3aa2db3)

  • deps: Exempt claude-agent-sdk from the 48h package-age gate and take the latest SDK
    (#196,
    3aa2db3)

  • deps: Hold the SDK bump to releases past CI's 48h package-age gate
    (#196,
    3aa2db3)

Continuous Integration

  • harbor: Exempt claude-agent-sdk from the package-age gate in Harbor E2E
    (#196,
    3aa2db3)

Documentation

  • agents: Update Pi and Antigravity references for the harness bump
    (#196,
    3aa2db3)

  • criteria: Name the budget-breach trigger on the post-failure path
    (#197,
    28c846e)

Features

  • criteria: Let a run_command criterion opt in to post-failure grading
    (#197,
    28c846e)

  • evalboard: Count agent time apart from eval overhead, and filter runs by variant
    (#198,
    b04829c)

  • evalboard: Headline agent time per passed task, not full task wall clock
    (#198,
    b04829c)

  • evalboard: Show agent time on the run page and filter it by variant
    (#198,
    b04829c)

  • evalboard: Show summed agent time in the overview window tile
    (#198,
    b04829c)

  • evalboard: Simplify the task timing strip into one bar and two groups
    (#198,
    b04829c)

  • evalboard: Split the task timeline into agent time and eval overhead
    (#198,
    b04829c)