v0.1.1

[0.1.1] 2026-11-09

Bug fixes

• Current OpenAI and Claude models, SDK majors and request parameters for the provider adapters (#11)

Documentation

• Standard README structure and language framing (#2)
• Vertical layered architecture diagram
• Horizontal layered architecture diagram
• Compact horizontal architecture diagram
• Recorded terminal demo of a real run, with its vhs tape (#3)
• Remove authorship statements from docs and scripts
• Recorded demos of more commands, with their vhs tapes (#4)
• Recorded demos of the HTTP service and more commands (#5)
• Describe the supply chain accurately (#6)
• Complete citation metadata and bump it on release (#8)
• Diagrams on white in the shared theme, make diagram for every diagram, and a self-contained Definition of Done (#10)

Build and CI

• Run CodeQL on main and on a weekly schedule only
• Build and attach the sdist and wheel to every release (#12)

Maintenance

• Add release automation, AGENTS.md, llms.txt and contributor docs
• Classify squash merges in the changelog by the commits they contain
• Latest toolchain and CI versions (#7)
• Release script keeps hard-coded version strings in step (#9)

v0.1.0

First release of a from-scratch evaluation framework for language models and agents.

Highlights

• Pluggable tasks and graders, caching, persistence, and regression tracking
• Paired bootstrap, McNemar, and Cohen's d, cross-checked against scipy.stats (agreement to machine precision; opt-in validation group)
• Contamination checks and a scale benchmark

Install: pip install the attached wheel.