v0.13.0 - OpenAI Agents SDK runtime eval
What shipped
- A second external runtime adapter pinned to
@openai/agents@0.16.1and Zod4.4.3in an isolated lockfile. - Every fixture executes through a real SDK
AgentandRunner.run()loop. - Allowed cases exercise SDK-managed, Zod-validated read-only function calls; refusal and escalation cases make no tool call.
- Trusted instructions and source-labeled untrusted content remain separate request fields.
- A public custom
Modelkeeps the suite deterministic and offline; provider network access is disabled in regression tests. - The expected outcome stays outside the Agent-controlled path and forced mismatches emit
fail. - All eight fixtures pass and generate 34 response, assertion, tool-trace, policy-trace, result, and summary files.
- CI validates results with public
@v0and retainsopenai-agents-eval-evidence. - Complete English and Simplified Chinese adapter docs and technical Pages guides.
- 134 deterministic tests; both isolated dependency audits report zero known vulnerabilities.
中文说明
- 新增第二个外部 Runtime 适配器,在隔离 Lockfile 中锁定
@openai/agents@0.16.1与 Zod4.4.3。 - 每条 Fixture 都通过真实 SDK
Agent与Runner.run()循环执行。 - 允许继续的案例会执行 SDK 管理、经 Zod 校验的只读 Function Call;拒绝与升级案例不会调用工具。
- 可信 Instructions 与带来源标签的不可信内容保持为独立 Request 字段。
- 公共自定义
Model让套件保持确定性和离线运行;回归测试会禁用 Provider 网络访问。 - 预期结果位于 Agent 可控路径之外,强制 mismatch 会真实生成
fail。 - 8 条 Fixture 全部通过,并生成 34 个回答、断言、工具 Trace、策略 Trace、Result 与 Summary 文件。
- CI 使用公开
@v0校验结果,并保留openai-agents-eval-evidence。 - 提供完整中英文适配器文档与技术 Pages 指南。
- 134 项确定性测试通过;两个隔离依赖审计均为 0 已知漏洞。
Public OpenAI Agents evidence: https://github.com/lindixu6-hash/awesome-agentic-engineering/actions/runs/31980983499
Final release CI: https://github.com/lindixu6-hash/awesome-agentic-engineering/actions/runs/31981159024
English guide: https://lindixu6-hash.github.io/awesome-agentic-engineering/openai-agents-eval/
中文指南: https://lindixu6-hash.github.io/awesome-agentic-engineering/zh/openai-agents-eval/