Skip to content

v0.1.0

Choose a tag to compare

View all tags
@adeoluo adeoluo released this 02 Nov 02:24
· 20 commits to main since this release
v0.1.0
81fac6c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

First release of a from-scratch evaluation framework for language models and agents.

Highlights

  • Pluggable tasks and graders, caching, persistence, and regression tracking
  • Paired bootstrap, McNemar, and Cohen's d, cross-checked against scipy.stats (agreement to machine precision; opt-in validation group)
  • Contamination checks and a scale benchmark

Install: pip install the attached wheel.

Assets 4
Loading