Releases: undercutsh/firstpass
Releases · undercutsh/firstpass
Release list
v0.2.0 — Undercut: measured routing, public benchmarks, live site
Rebranded to Undercut — repo moved to undercutsh/firstpass, install is npx skills add undercutsh/firstpass.
What's in this release
- probe policy (shipped) — format-strict work starts cheap (model-agnostic), caps at standard. Replaces a vendor-specific rule that round-5 cross-vendor testing proved brittle.
- Public-benchmark validation — GSM8K + HumanEval, official test cases, raw results published in
testing/results/. - Landing page — getundercut.sh (Vercel + Cloudflare DNS), canonical to the full domain.
- Flagging-reliability test — stock dispatchers route 100% of units correctly despite ~90% flag accuracy; no custom flagging model needed.
- CI, CONTRIBUTING, issue/PR templates, trust backlog, verified models.md.
Measured headline (public benchmarks, official test cases)
- Up to −71% cost on GSM8K (OpenAI), equal-or-better pass
- Up to −95% cost on HumanEval (Gemini), identical pass
- Tiered never worse on any synthetic-suite cell; better on 6 of 12
See testing/README.md for full methodology and caveats.