v0.0.4: tak compare
This release adds tak compare, which turns recorded measurements into a pass/fail gate for a pull request, and fixes a v0.0.3 regression where a broken tak.toml could abort commands that never read benchmarks.
Added
-
tak comparediffs git-notes measurements between a base revision (defaultorigin/main) and a head (defaultHEAD), prints a markdown table, and exits non-zero when instruction counts rise more thangate_pct(default 1%). By @jdx in #22.$ tak compare v1.30.0 --rev v1.33.0Only instruction counts are gated. Wall clock is reported beside them for context and never gated, because on identical hardware it moves 4-20% run to run — a threshold tight enough to catch a real regression fires constantly. The comparison is keyed by benchmark, tool, and runner class: measurements from different runner types do not line up (counts shift between machine types by more than a real regression does), so they show up as added or removed rather than compared. Duplicate records reduce to their minimum, matching how tak reduces within a run, and a benchmark that stops running is called out rather than silently dropped from the table. Flags: global
--gate-pct(alsoTAK_GATE_PCT),--no-credit, and--no-gateoncompare.
Fixed
-
A broken
tak.tomlno longer aborts commands that ignore it.resolve_settingsran before dispatch for every subcommand and validated every benchmark, so an invalid[bench.x]would aborttak run -- ./mycli,tak push, andtak doctoreven though none of them read benchmarks. Resolution is now lazy — onlyrun,backfill, andsettingsconsult the file — and reads only the settings tables, so an invalid benchmark no longer blocks an explicit command. A TOML syntax error still fails the measuring commands, since the file may carry[env]rules deciding what is scrubbed from a subject's environment and silently applying a weaker filter is worse than refusing. This fix was approved for v0.0.3 but a commit never landed on main. By @jdx in #20. -
tak historyandtak comparenow resolve annotated tags correctly.git rev-parse v1.2.3on an annotated tag returns the tag object, but notes hang off commits, so naming a tagged revision read as "nothing recorded" — exactly the revisions people are most likely to name.notes::rev_parsenow peels to the underlying commit. By @jdx in #22.
Full Changelog: v0.0.3...v0.0.4