·
2 commits
to main
since this release
This release adds evaluation regression gating, session comparison for replay, and significant hardening of served-agent safety controls across A2A, MCP HTTP, and chat surfaces.
What's New
- Adds
--baselineand--regression-toleranceflags todocker agent eval, enabling CI-style regression detection by comparing a run against a previously saved baseline and failing on quality regression - Adds
docker agent sessions diffcommand to compare the behavior of two recorded sessions, reporting the first point of divergence between runs - Adds unattended tool execution restrictions for A2A, requiring explicit security configuration for network listeners
- Adds centralized unattended safety resolution for served agents, with ordered concrete safety policies and hardened HTTP serving for MCP and chat surfaces
- Adds session-origin isolation for A2A session resumes
Bug Fixes
- Fixes the baseline gate to be grounded on the file that eval writes, and fails closed on error
- Fixes replay comparison to handle sub-agent turns, compare arguments semantically, and sanitize output
- Fixes A2A session resumes to be isolated by origin
Technical Changes
- Refactors
sessions diffCLI command to resolve session references - Refactors safety flag validation to be shared across CLI surfaces
- Refactors HTTP auth and origin matchers into a shared
httpsecpackage - Refactors A2A server assembly, extracting it from
Run - Replaces
CAGENT_MODELS_GATEWAYwithDOCKER_AGENT_MODELS_GATEWAYin docs and samples - Documents the evaluation regression gate and its two flags, the
docker agent sessions diffcommand, and served-agent safety controls
What's Changed
- docs: update CHANGELOG.md for v1.125.0 by @docker-read-write[bot] in #3995
- feat(replay): compare the behaviour of two recorded sessions by @dwin-gharibi in #3948
- feat(eval): compare a run against a saved baseline and fail on regression by @dwin-gharibi in #3946
- docs: replace CAGENT_MODELS_GATEWAY with DOCKER_AGENT_MODELS_GATEWAY in docs and samples by @aheritier in #3997
- feat: harden served-agent safety controls by @aheritier in #4000
Full Changelog: v1.125.0...v1.126.0