Skip to content

Document ThinkingBox-Bench v1 release - #12

Merged
Liang-Chun Tsai (ltsai-dev) merged 8 commits into
mainfrom
ltsai-microsoft-document-thinkingbox-bench-release
Aug 17, 2026
Merged

Document ThinkingBox-Bench v1 release#12
Liang-Chun Tsai (ltsai-dev) merged 8 commits into
mainfrom
ltsai-microsoft-document-thinkingbox-bench-release

Conversation

@ltsai-dev

@ltsai-dev Liang-Chun Tsai (ltsai-dev) commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add ThinkingBox-Bench v1.0 as the canonical 507-task benchmark release and make it prominent in the root README
  • consolidate releases/ to the two supported evaluations: ThinkingBox-Bench and Airline Tau-Bench
  • move the canonical 507-entry test list under releases/thinkingbox_bench_v1/
  • remove the superseded External Retail and Sandbox RL Zendesk release wrappers
  • document deterministic final-state evaluation and the 30 rubric-enhanced tasks
  • normalize Airline Tau-Bench commands to the repository's side-by-side checkout layout

ThinkingBox-Bench composition

Domain Tasks
Retail and e-commerce 98
Travel and hospitality 104
Auto insurance 100
Neobank support 104
Consulting IT/HR support 101
Total 507

Liang-Chun Tsai and others added 8 commits August 17, 2026 11:02
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f71fc0b5-25dc-43ae-8eea-04d0e45f8de9
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f71fc0b5-25dc-43ae-8eea-04d0e45f8de9
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f71fc0b5-25dc-43ae-8eea-04d0e45f8de9
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f71fc0b5-25dc-43ae-8eea-04d0e45f8de9
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f71fc0b5-25dc-43ae-8eea-04d0e45f8de9
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f71fc0b5-25dc-43ae-8eea-04d0e45f8de9
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f71fc0b5-25dc-43ae-8eea-04d0e45f8de9
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f71fc0b5-25dc-43ae-8eea-04d0e45f8de9
@ltsai-dev
Liang-Chun Tsai (ltsai-dev) merged commit 9fd1631 into main Aug 17, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant