Skip to content

v0.12.17

Latest
Latest

Choose a tag to compare

View all tags
@coder-eval-release-bot coder-eval-release-bot released this 06 Nov 21:24
· 1 commit to main since this release
v0.12.17
8073193

Bug Fixes

  • Address grading recovery before-merge feedback
    (#248,
    bc45225)

  • Align Delegate smoke prompt with automation use
    (#248,
    bc45225)

  • Make grading failure and budget returns explicit
    (#248,
    bc45225)

  • Preserve execution facts during grading recovery
    (#248,
    bc45225)

  • Preserve logical turn identity across retries
    (#250,
    291ac2a)

  • Recover grading errors without erasing execution failures
    (#248,
    bc45225)

  • Recover grading failures without replaying agents
    (#248,
    bc45225)

  • Reject scripted prompt overrides and derive dialog length
    (#246,
    51f4ab3)

  • Require patched oauthlib dependency (#248,
    bc45225)

  • Separate grading attempts from execution facts
    (#248,
    bc45225)

  • Skip budget grading when grading is disabled
    (#248,
    bc45225)

  • deps: Bump filelock to 4.2.4 for pip-audit
    (#239,
    484c3d5)

  • deps: Bump filelock, litellm, msgpack, oauthlib, virtualenv for new CVEs
    (#240,
    e8904d9)

  • deps: Bump pyasn1 to 0.6.5 for four High advisories
    (#248,
    bc45225)

  • deps: Bump pyasn1 to 0.6.5 for four High advisories
    (#247,
    48791f6)

  • deps: Bump pyasn1 to 0.6.5 for four High advisories
    (#246,
    51f4ab3)

  • deps: Bump urllib3 to 2.8.0 and virtualenv to 21.7.13 for pip-audit
    (#239,
    484c3d5)

  • deps: Bump virtualenv to 21.14.13 for osv-scanner
    (#239,
    484c3d5)

  • docker: Accept only bridge for grading_network and refuse a stray grading allowlist
    (#239,
    484c3d5)

  • docker: Allow only the model hosts each llm_only task uses
    (#239,
    484c3d5)

  • docker: Code review fixes for network: llm_only
    (#239,
    484c3d5)

  • docker: Keep the row status on refused grading hosts and harden the grading handshake
    (#239,
    484c3d5)

  • docker: Let a re-grade replace the recorded grading denials
    (#239,
    484c3d5)

  • docker: Make the egress probe's model-host check work on every harness
    (#239,
    484c3d5)

  • docker: Reclaim leaked egress networks and report hosts refused during grading
    (#239,
    484c3d5)

  • docker: Suppress the grading watcher's own cancel without an empty except
    (#239,
    484c3d5)

  • evalboard: Fold delegate-sdk into the delegate harness
    (#235,
    a253f34)

  • evalboard: Guard harness lookups against Object.prototype keys
    (#235,
    a253f34)

  • evalboard: Keep criterion defaults for rows without the fields
    (#235,
    a253f34)

  • reports: Gate judge cards on the result class and fix a vacuous token test
    (#235,
    a253f34)

  • samples: Address review on 3d-scan-calc verifier
    (#237,
    021534c)

  • samples: Grade 3d-scan-calc against fixed expected values
    (#237,
    021534c)

  • simulation: Export DialogUser as a plain alias for CodeQL
    (#246,
    51f4ab3)

  • simulation: Retain exhaustion facts and expand scripted dataset turns
    (#246,
    51f4ab3)

Build System

Chores

  • docker: Remove redundant parts of the llm_only change
    (#239,
    484c3d5)

Documentation

  • Fix text that still describes removed fallbacks
    (#235,
    a253f34)

  • Remove text that still describes removed fallbacks
    (#235,
    a253f34)

  • docker: 5/5 — document network: llm_only and add a negative egress task
    (#239,
    484c3d5)

  • docker: Record the OpenCode and Pi llm_only runs
    (#239,
    484c3d5)

  • tasks: List docker_egress_llm_only among the smoke members
    (#239,
    484c3d5)

Features

  • Configure whole-turn agent retry limits (#247,
    48791f6)

  • docker: 1/5 — add network: llm_only and egress_allowlist to DockerDriverConfig
    (#239,
    484c3d5)

  • docker: 2/5 — add the stdlib-only egress proxy for network: llm_only
    (#239,
    484c3d5)

  • docker: 3/5 — derive the llm_only egress allowlist on the host
    (#239,
    484c3d5)

  • docker: 4/5 — run llm_only containers behind a per-task egress sidecar
    (#239,
    484c3d5)

  • docker: Add network: llm_only with a separate grading allowlist
    (#239,
    484c3d5)

  • docker: Give grading its own egress allowlist under network: llm_only
    (#239,
    484c3d5)

  • simulation: Add scripted user turns (simulation.turns)
    (#246,
    51f4ab3)

Refactoring

  • Drop run-format fallbacks no recent run needs
    (#235,
    a253f34)

  • Fold delegate-sdk into delegate, drop run-format fallbacks older than 30 days
    (#235,
    a253f34)

Testing

  • Avoid retry backoff in grading provenance fixtures
    (#248,
    bc45225)

  • Preserve unmeasured scores and Windows event loops
    (#248,
    bc45225)

  • docker: Consolidate the egress tests (#239,
    484c3d5)

  • docker: Keep side effects out of asserts in the egress runner tests
    (#239,
    484c3d5)

  • docker: Make the bad-region setup-error test independent of local credentials
    (#239,
    484c3d5)

  • docker: Resolve the CodeQL alerts in the egress tests
    (#239,
    484c3d5)

  • tasks: Add a thorough egress probe task for network: llm_only
    (#239,
    484c3d5)

  • tasks: Probe the LiteLLM model host in the egress task
    (#239,
    484c3d5)

Assets 2
Loading