Repository navigation
分享:900 题上下文评测,长会话截断策略可量化 #3884
Hayabusa-YL
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
做了一套长上下文直读评测,插件生态里长会话截断的策略可以拿它量化对照:LongBench v1 四个英文子集 900 题,单次调用、不切片,900 题 24 分 46 秒、token 成本约 4.5 元,单题不到一分钱。
代码和结果:https://github.com/Hayabusa-YL/longbench-harness
有反馈在这个 thread 里回。
All reactions