Skip to content

v0.12.0

Choose a tag to compare

View all tags
@coder-eval-release-bot coder-eval-release-bot released this 09 Sep 20:23
· 20 commits to main since this release
v0.12.0
234dd54

Bug Fixes

  • Code review fixes for pi-harness (#159,
    57c9e33)

  • container: Arm the heartbeat watchdog only inside the container
    (#154,
    0fe8cb0)

  • criteria: Refuse an out-of-sandbox criterion path instead of scoring it 0.0
    (#154,
    0fe8cb0)

  • eval: Address the medium and low findings from the branch review
    (#154,
    0fe8cb0)

  • eval: Close the verdict-correctness gaps in detached grading
    (#154,
    0fe8cb0)

  • evalboard: Show per-row cost for open-weight harnesses (Pi)
    (#159,
    57c9e33)

  • execute: Close the verdict-changing and trust-boundary defects in detached grading
    (#154,
    0fe8cb0)

  • execute: Close the verdict-divergence, trust-gate and fabricated-rate defects
    (#154,
    0fe8cb0)

  • execute: Move post_run to the grading phase and stop the record lying about its driver
    (#154,
    0fe8cb0)

  • pi: Address bai-uipath review — apportionment caveat + token/telemetry minors
    (#159,
    57c9e33)

  • pi: Do not forward allowed_tools/disallowed_tools to Pi
    (#159,
    57c9e33)

  • pi: Multi-model review — gate error-crash on intentional cuts + dead dup + doc/guard nits
    (#159,
    57c9e33)

  • pi: Review blockers — session-id sanitize, error crash, telemetry warn, tool map, CE047
    (#159,
    57c9e33)

  • security: Close the three CodeQL findings on the detached-grading diff
    (#154,
    0fe8cb0)

  • tasks: Make the two remaining absolute criterion paths reachable, and add CE055
    (#154,
    0fe8cb0)

  • tests: Kill the CodeQL taint source and the Windows mode assertion
    (#154,
    0fe8cb0)

Documentation

  • agents: 4/4 — add Pi harness docs + enumeration-surface parity
    (#159,
    57c9e33)

  • docker: Fix env_passthrough model name + list Pi in baked-toolchain docs
    (#159,
    57c9e33)

  • pi: 3/4 — document docker support for Pi
    (#159,
    57c9e33)

  • pi: Correct plugins->--skill support + OPENROUTER passthrough claims
    (#159,
    57c9e33)

  • pi: Fix tool-enforcement claims + cost docstring after review
    (#159,
    57c9e33)

Features

  • agents: 2/4 — add PiAgent harness (pi --mode json) + registration
    (#159,
    57c9e33)

  • cli: coder-eval execute + detached grading via evaluate <run_dir>
    (#154,
    0fe8cb0)

  • cli: Add coder-eval execute — run tasks without grading them
    (#154,
    0fe8cb0)

  • cli: Grade an executed run afterwards — evaluate <run_dir> + Sandbox.adopt
    (#154,
    0fe8cb0)

  • cli: Make --resume distinguish "executed" from "graded"
    (#154,
    0fe8cb0)

  • docker: 1/4 — bake the pinned Pi CLI into docker/Dockerfile
    (#159,
    57c9e33)

  • evalboard: Show the Pi logo + "Pi" in the harness view
    (#159,
    57c9e33)

  • models: 1/4 — add AgentKind.PI + PiAgentConfig model
    (#159,
    57c9e33)

  • pi: Add Pi harness (--type pi) with docker + skill injection
    (#159,
    57c9e33)

  • pi: Load agent.plugins skills via --skill + harden turn/token handling
    (#159,
    57c9e33)

  • sandbox: 2/4 — forward OPENROUTER_API_KEY into docker containers
    (#159,
    57c9e33)

  • tasks: 3/4 — add local pi_smoke_test task
    (#159,
    57c9e33)

Refactoring

  • agents: Hoist shared plugins->skills resolver into agents/_skills.py
    (#159,
    57c9e33)

Testing

  • Fix two CI-only failures in the detached-grading tests
    (#154,
    0fe8cb0)

  • docker: 4/4 — guard that the Dockerfile bakes a pinned Pi CLI
    (#159,
    57c9e33)

  • pi: Assign the awaited cancel result to silence CodeQL 'no effect'
    (#159,
    57c9e33)

  • pi: Install shutil.which patch so the env-info test doesn't need the real CLI
    (#159,
    57c9e33)

  • pi: Port OpenCode teardown + cost-fallback matrices (blockers 3, 4)
    (#159,
    57c9e33)

  • pi: Scrub personal scratchpad path from happy-stream fixture
    (#159,
    57c9e33)

Assets 2
Loading