Skip to content

v0.1.0

Choose a tag to compare

@npow npow released this 22 Feb 00:22
· 51 commits to master since this release

Initial release of context-bench.

  • OpenAIProxy system for benchmarking any OpenAI-compatible endpoint
  • Built-in metrics: MeanScore, PassRate, CompressionRatio, CostOfPass, ParetoRank
  • Dataset loaders: HotpotQA, GSM8K, BFCL, APIGen, SWE-agent, local JSONL
  • Markdown and JSON reporters
  • Works out of the box with Headroom and Compresr