Releases: jdx/tak
Release list
v0.0.5: Trend and outlier fixes for compare reports
A small release with two fixes to how tak compare renders its report. Both were review findings from #22 that were pushed to that branch after it had already merged, so they shipped in v0.0.4.
Fixed
-
The trend sparkline now agrees with the table beside it.
gather_trendplotted every record's value whilecomparefolds duplicates to the minimum, so a commit carrying several records for one series — a CI re-run, a retry — contributed several points and a noisy re-run put a spike in the sparkline that the table did not show. Trend assembly now takes the minimum per commit, the same rulecompareuses. It also places the compared revision in chronological order when that SHA already appears in the walked history — for exampletak compare v1.33.0 --rev v1.30.0, which compares against an ancestor — instead of appending its value at the end and drawing a line that reads left-to-right but is not in time order. By @jdx in #23. -
The added/removed lists now keep tool identity. Two series differing only by tool rendered identically, so a report could say the same benchmark both started and stopped gating while meaning two different programs. Both lists now name a series by bench, tool (omitted for
self), and runner, matching how the table already names it. By @jdx in #23.
Full Changelog: v0.0.4...v0.0.5
v0.0.4: tak compare
This release adds tak compare, which turns recorded measurements into a pass/fail gate for a pull request, and fixes a v0.0.3 regression where a broken tak.toml could abort commands that never read benchmarks.
Added
-
tak comparediffs git-notes measurements between a base revision (defaultorigin/main) and a head (defaultHEAD), prints a markdown table, and exits non-zero when instruction counts rise more thangate_pct(default 1%). By @jdx in #22.$ tak compare v1.30.0 --rev v1.33.0Only instruction counts are gated. Wall clock is reported beside them for context and never gated, because on identical hardware it moves 4-20% run to run — a threshold tight enough to catch a real regression fires constantly. The comparison is keyed by benchmark, tool, and runner class: measurements from different runner types do not line up (counts shift between machine types by more than a real regression does), so they show up as added or removed rather than compared. Duplicate records reduce to their minimum, matching how tak reduces within a run, and a benchmark that stops running is called out rather than silently dropped from the table. Flags: global
--gate-pct(alsoTAK_GATE_PCT),--no-credit, and--no-gateoncompare.
Fixed
-
A broken
tak.tomlno longer aborts commands that ignore it.resolve_settingsran before dispatch for every subcommand and validated every benchmark, so an invalid[bench.x]would aborttak run -- ./mycli,tak push, andtak doctoreven though none of them read benchmarks. Resolution is now lazy — onlyrun,backfill, andsettingsconsult the file — and reads only the settings tables, so an invalid benchmark no longer blocks an explicit command. A TOML syntax error still fails the measuring commands, since the file may carry[env]rules deciding what is scrubbed from a subject's environment and silently applying a weaker filter is worse than refusing. This fix was approved for v0.0.3 but a commit never landed on main. By @jdx in #20. -
tak historyandtak comparenow resolve annotated tags correctly.git rev-parse v1.2.3on an annotated tag returns the tag object, but notes hang off commits, so naming a tagged revision read as "nothing recorded" — exactly the revisions people are most likely to name.notes::rev_parsenow peels to the underlying commit. By @jdx in #22.
Full Changelog: v0.0.3...v0.0.4
v0.0.3: Keep measurements clean, and stop losing them
This release scrubs forge tokens from measured commands, fixes a bug where a push could erase the remote's entire measurement history, and adds a settings registry to configure it. It also repairs the Docker build.
Added
- A settings registry driven by
settings.toml, withbuild.rsgenerating theSettingsstruct, defaults, and introspection metadata. A newtak settingssubcommand prints resolved values, their sources, and the computed scrub list;tak settings --docsadds descriptions and examples. The first two settings areenv_deny(variables removed from measured commands) andenv_allow(variables kept even thoughenv_denylists them, subtracting from the deny list). Resolution precedence is CLI > environment >tak.toml> default, exposed as--env-deny/--env-allowflags,TAK_ENV_DENY/TAK_ENV_ALLOW, and an[env]section intak.toml. By @jdx in #18.
Fixed
-
Forge tokens (
GITHUB_TOKEN,GH_TOKENby default) are now removed from the environment of every command tak measures, on both the wall-clock and cachegrind paths. A CLI that finds a token often does more work with it — authenticating, fetching, checking rate limits — which lands in the series as an unexplained step change. It also keeps a repository-write token out of scope for the downloaded binaries thattak backfillexecutes. Projects measuring a CLI that genuinely cannot start unauthenticated can opt a variable back in viatak.toml. By @jdx in #17.# tak.toml [env] allow = ["GITHUB_TOKEN"]
-
A push no longer erases the remote's measurements. One forced refspec served both fetch and push, so the retry-and-merge loop beneath it never ran and any writer with a stale or empty local notes ref replaced the whole remote history — which is exactly what a fresh CI checkout is, since
actions/checkoutdoes not fetch notes. Push now uses a non-forced refspec so a non-fast-forward is rejected and the merge path runs; fetch lands on a scratch ref and is merged into the local one withcat_sort_uniq. Reading (tak history) between recording and pushing no longer discards the unpushed local record. By @jdx in #19. -
The Docker build works again. The build stage was missing
crates/from its context, so cargo failed at manifest resolution — a regression from whenasset-pickerwas extracted into the workspace. By @jdx in #16.
Full Changelog: v0.0.2...v0.0.3