Skip to content

v0.13.0 - OpenAI Agents SDK runtime eval

Choose a tag to compare

@lindixu6-hash lindixu6-hash released this 17 Aug 00:10

What shipped

  • A second external runtime adapter pinned to @openai/agents@0.16.1 and Zod 4.4.3 in an isolated lockfile.
  • Every fixture executes through a real SDK Agent and Runner.run() loop.
  • Allowed cases exercise SDK-managed, Zod-validated read-only function calls; refusal and escalation cases make no tool call.
  • Trusted instructions and source-labeled untrusted content remain separate request fields.
  • A public custom Model keeps the suite deterministic and offline; provider network access is disabled in regression tests.
  • The expected outcome stays outside the Agent-controlled path and forced mismatches emit fail.
  • All eight fixtures pass and generate 34 response, assertion, tool-trace, policy-trace, result, and summary files.
  • CI validates results with public @v0 and retains openai-agents-eval-evidence.
  • Complete English and Simplified Chinese adapter docs and technical Pages guides.
  • 134 deterministic tests; both isolated dependency audits report zero known vulnerabilities.

中文说明

  • 新增第二个外部 Runtime 适配器,在隔离 Lockfile 中锁定 @openai/agents@0.16.1 与 Zod 4.4.3
  • 每条 Fixture 都通过真实 SDK AgentRunner.run() 循环执行。
  • 允许继续的案例会执行 SDK 管理、经 Zod 校验的只读 Function Call;拒绝与升级案例不会调用工具。
  • 可信 Instructions 与带来源标签的不可信内容保持为独立 Request 字段。
  • 公共自定义 Model 让套件保持确定性和离线运行;回归测试会禁用 Provider 网络访问。
  • 预期结果位于 Agent 可控路径之外,强制 mismatch 会真实生成 fail
  • 8 条 Fixture 全部通过,并生成 34 个回答、断言、工具 Trace、策略 Trace、Result 与 Summary 文件。
  • CI 使用公开 @v0 校验结果,并保留 openai-agents-eval-evidence
  • 提供完整中英文适配器文档与技术 Pages 指南。
  • 134 项确定性测试通过;两个隔离依赖审计均为 0 已知漏洞。

Public OpenAI Agents evidence: https://github.com/lindixu6-hash/awesome-agentic-engineering/actions/runs/31980983499

Final release CI: https://github.com/lindixu6-hash/awesome-agentic-engineering/actions/runs/31981159024

English guide: https://lindixu6-hash.github.io/awesome-agentic-engineering/openai-agents-eval/

中文指南: https://lindixu6-hash.github.io/awesome-agentic-engineering/zh/openai-agents-eval/