Skip to content

v1.9.1

Choose a tag to compare

View all tags
@Yunnglin Yunnglin released this 21 Jul 03:26
· 102 commits to main since this release
v1.9.1
9d1b353

中文版

基准测试数据集

  • 智能体评测: 新增 Claw-Eval (#1487)、ResearchRubrics (#1478)、Toolathlon 智能体基准测试
  • 多模态评测: 新增 TVBench 视频理解基准测试 (#1471)
  • 通用评测: 新增 WideSearch、PerspectiveGap (#1461) 基准测试

功能增强

  • 性能测试: 新增 workload_trace 数据集插件，支持生产流量回放 (#1494)
  • 性能测试: 统一 --dataset-args 参数，支持固定长度输入 (#1483, #1495)
  • 性能测试: 新增 /v1/rerank endpoint 支持 (#1498)
  • Web 服务: Dashboard 新增性能测试归档(archive)功能 (#1484)
  • 模型支持: 支持 Anthropic prompt caching (#1444)
  • 安全增强: 使用 SecretStr 对 eval / perf 密钥进行脱敏 (#1490)
  • 性能优化: 重构 benchmark 数据集加载逻辑 (#1482)；优化 config 与 CLI 冷启动导入 (#1491)
  • 前端优化: 加固前端工作流与报告生成 (#1492)

问题修复

  • 修复 evaluator 样本总数日志，并对 per-subset --limit 给出警告 (#1497)
  • 修复 Windows 下代码执行评分问题 (#1488)
  • 修复 line_by_line 数据集 dict body 字段处理 (#1485)
  • 修复多行 SSE 格式(id/event/data)解析问题 (#1474)
  • 修复 trivia_qa prompt 及 repeats 格式化问题 (#1476)
  • 修复 agent 沙箱命令超时未终止问题
  • 修复 swe-bench 沙箱环境变量传递问题 (#1470)
  • 修复 bfcl OpenAI base URL 归一化问题 (#1468)
  • 修复 scicode 依赖，锁定 scipy < 1.14 (#1467)
  • 修复 openai 流式响应中断重试问题 (#1464)
  • 修复 openai 流式 TTFT 未包含 delta.reasoning 的问题 (#1463)
  • 修复 sandbox manager stop 失败后的资源清理问题 (#1465)

English Version

Benchmark Datasets

  • Agent Evaluation: Added Claw-Eval (#1487), ResearchRubrics (#1478), and Toolathlon agent benchmarks
  • Multimodal Evaluation: Added TVBench video understanding benchmark (#1471)
  • General Evaluation: Added WideSearch and PerspectiveGap (#1461) benchmarks

Feature Enhancements

  • Performance Testing: Added workload_trace dataset plugin for production traffic replay (#1494)
  • Performance Testing: Unified --dataset-args with fixed-length input support (#1483, #1495)
  • Performance Testing: Added /v1/rerank endpoint support (#1498)
  • Web Service: Added performance benchmark archive to the dashboard (#1484)
  • Model Support: Added Anthropic prompt caching support (#1444)
  • Security: Masked eval / perf secrets with SecretStr (#1490)
  • Performance: Refactored benchmark dataset loading (#1482); refined config and CLI cold-start imports (#1491)
  • Frontend: Hardened frontend workflows and reporting (#1492)

Bug Fixes

  • Fixed resolved sample total logging and added warning for per-subset --limit (#1497)
  • Fixed code execution scoring on Windows (#1488)
  • Fixed dict body field handling in line_by_line dataset (#1485)
  • Fixed multi-line SSE parsing with id, event, and data fields (#1474)
  • Fixed trivia_qa prompt and repeats formatting (#1476)
  • Fixed agent sandbox commands not terminating on timeout
  • Fixed sandbox environment variable passing for swe-bench (#1470)
  • Fixed OpenAI base URL normalization for bfcl (#1468)
  • Fixed scicode dependency by pinning scipy below 1.14 (#1467)
  • Fixed retry on interrupted OpenAI streaming responses (#1464)
  • Fixed missing delta.reasoning in OpenAI streaming TTFT (#1463)
  • Fixed resource cleanup after sandbox manager stop failure (#1465)
Assets 2
Loading