Skip to content

SemBench 0.2.0

Choose a tag to compare

View all tags
@zbennett10 zbennett10 released this 03 Sep 20:08
· 18 commits to main since this release
v0.2.0
df80198
  • run-load: donor->recipient streams with a fixed number in flight; requests per second, output tokens per second, TTFT percentiles for donors and recipients.
  • run-live-gateway --post-donor-delay-ms for engines that index donors asynchronously.
  • ROI cost model recalibrated to 8K to 24K measurements on an A10G.
  • Long-context manifest generator and the exact arm recipe in examples/overhead-benefit/.
Assets 2
Loading