Skip to content

Releases: jdx/tak

v0.0.5: Trend and outlier fixes for compare reports

Choose a tag to compare

@jdx jdx released this 27 Jul 16:50
Immutable release. Only release title and notes can be modified.
672dd21

A small release with two fixes to how tak compare renders its report. Both were review findings from #22 that were pushed to that branch after it had already merged, so they shipped in v0.0.4.

Fixed

  • The trend sparkline now agrees with the table beside it. gather_trend plotted every record's value while compare folds duplicates to the minimum, so a commit carrying several records for one series — a CI re-run, a retry — contributed several points and a noisy re-run put a spike in the sparkline that the table did not show. Trend assembly now takes the minimum per commit, the same rule compare uses. It also places the compared revision in chronological order when that SHA already appears in the walked history — for example tak compare v1.33.0 --rev v1.30.0, which compares against an ancestor — instead of appending its value at the end and drawing a line that reads left-to-right but is not in time order. By @jdx in #23.

  • The added/removed lists now keep tool identity. Two series differing only by tool rendered identically, so a report could say the same benchmark both started and stopped gating while meaning two different programs. Both lists now name a series by bench, tool (omitted for self), and runner, matching how the table already names it. By @jdx in #23.

Full Changelog: v0.0.4...v0.0.5

v0.0.4: tak compare

Choose a tag to compare

@jdx jdx released this 27 Jul 01:04
Immutable release. Only release title and notes can be modified.
15da0fc

This release adds tak compare, which turns recorded measurements into a pass/fail gate for a pull request, and fixes a v0.0.3 regression where a broken tak.toml could abort commands that never read benchmarks.

Added

  • tak compare diffs git-notes measurements between a base revision (default origin/main) and a head (default HEAD), prints a markdown table, and exits non-zero when instruction counts rise more than gate_pct (default 1%). By @jdx in #22.

    $ tak compare v1.30.0 --rev v1.33.0
    

    Only instruction counts are gated. Wall clock is reported beside them for context and never gated, because on identical hardware it moves 4-20% run to run — a threshold tight enough to catch a real regression fires constantly. The comparison is keyed by benchmark, tool, and runner class: measurements from different runner types do not line up (counts shift between machine types by more than a real regression does), so they show up as added or removed rather than compared. Duplicate records reduce to their minimum, matching how tak reduces within a run, and a benchmark that stops running is called out rather than silently dropped from the table. Flags: global --gate-pct (also TAK_GATE_PCT), --no-credit, and --no-gate on compare.

Fixed

  • A broken tak.toml no longer aborts commands that ignore it. resolve_settings ran before dispatch for every subcommand and validated every benchmark, so an invalid [bench.x] would abort tak run -- ./mycli, tak push, and tak doctor even though none of them read benchmarks. Resolution is now lazy — only run, backfill, and settings consult the file — and reads only the settings tables, so an invalid benchmark no longer blocks an explicit command. A TOML syntax error still fails the measuring commands, since the file may carry [env] rules deciding what is scrubbed from a subject's environment and silently applying a weaker filter is worse than refusing. This fix was approved for v0.0.3 but a commit never landed on main. By @jdx in #20.

  • tak history and tak compare now resolve annotated tags correctly. git rev-parse v1.2.3 on an annotated tag returns the tag object, but notes hang off commits, so naming a tagged revision read as "nothing recorded" — exactly the revisions people are most likely to name. notes::rev_parse now peels to the underlying commit. By @jdx in #22.

Full Changelog: v0.0.3...v0.0.4

v0.0.3: Keep measurements clean, and stop losing them

Choose a tag to compare

@jdx jdx released this 26 Jul 00:33
Immutable release. Only release title and notes can be modified.
cc4950e

This release scrubs forge tokens from measured commands, fixes a bug where a push could erase the remote's entire measurement history, and adds a settings registry to configure it. It also repairs the Docker build.

Added

  • A settings registry driven by settings.toml, with build.rs generating the Settings struct, defaults, and introspection metadata. A new tak settings subcommand prints resolved values, their sources, and the computed scrub list; tak settings --docs adds descriptions and examples. The first two settings are env_deny (variables removed from measured commands) and env_allow (variables kept even though env_deny lists them, subtracting from the deny list). Resolution precedence is CLI > environment > tak.toml > default, exposed as --env-deny / --env-allow flags, TAK_ENV_DENY / TAK_ENV_ALLOW, and an [env] section in tak.toml. By @jdx in #18.

Fixed

  • Forge tokens (GITHUB_TOKEN, GH_TOKEN by default) are now removed from the environment of every command tak measures, on both the wall-clock and cachegrind paths. A CLI that finds a token often does more work with it — authenticating, fetching, checking rate limits — which lands in the series as an unexplained step change. It also keeps a repository-write token out of scope for the downloaded binaries that tak backfill executes. Projects measuring a CLI that genuinely cannot start unauthenticated can opt a variable back in via tak.toml. By @jdx in #17.

    # tak.toml
    [env]
    allow = ["GITHUB_TOKEN"]
  • A push no longer erases the remote's measurements. One forced refspec served both fetch and push, so the retry-and-merge loop beneath it never ran and any writer with a stale or empty local notes ref replaced the whole remote history — which is exactly what a fresh CI checkout is, since actions/checkout does not fetch notes. Push now uses a non-forced refspec so a non-fast-forward is rejected and the merge path runs; fetch lands on a scratch ref and is merged into the local one with cat_sort_uniq. Reading (tak history) between recording and pushing no longer discards the unpushed local record. By @jdx in #19.

  • The Docker build works again. The build stage was missing crates/ from its context, so cargo failed at manifest resolution — a regression from when asset-picker was extracted into the workspace. By @jdx in #16.

Full Changelog: v0.0.2...v0.0.3

tak-cli-v0.0.2

Choose a tag to compare

@jdx jdx released this 25 Jul 21:48
Immutable release. Only release title and notes can be modified.
aa7f818

🚀 Features

  • declare benchmarks in tak.toml by @jdx in #8
  • add logo and favicon set by @jdx in #9