Skip to content

Releases: ESchuderer/BenchKeeper

benchkeeper 6.25

Choose a tag to compare

@github-actions github-actions released this 28 Oct 00:52
f35b729
  • App, Sessions board redesigned (#406): a project rail first (newest activity, session count, sparkline, last tier; a chip row on a phone), tiles for sessions, agent time, cost, tokens, tests-pass rate and merged pull requests with a sparkline and the change against the previous period of the same length, the per-day chart stacked by project with a metric switch (cost, agent time, tokens, sessions) and legend chips that hide a series, comparison bars by tier, pattern and setup with the sample size on every bar (under 20 sessions: hatched, marked, left out of the highest and lowest line), cost per merged pull request and model mix by week, histograms of agent time and turns, a 12-week activity calendar with active days and longest streak, and branches and pull request states of the chosen project. Own design tokens (radius, shadow, glow, a six-color series palette checked at 3:1 in light and dark, monospace tabular figures, reduced motion). Same rows as before: no new field, no message text, no network request.
  • Breaking, before 7.0: the per-day chart is stacked by project, not by setup (the setup comparison moved to its own bars); the rail has no all-directories toggle (#404 adds it to the Project select).
  • Plugin version 6.25.0.
  • Upgrade: npx benchkeeper@6.25.0 apply.

benchkeeper 6.24

Choose a tag to compare

@github-actions github-actions released this 27 Oct 23:52
93102c3
  • App, skills: community skills of the library and the global skills install from the default branch of their source instead of a commit. The project's state file records the commit the install got, benchkeeper skills list --check (and the Catalog's "Check for updates") asks each source for its default branch and marks moved skills outdated, skills update re-adds them. The skills CLI runs as the newest release at least 7 days old (as the host tools do; the npm signature check, the dependency cooldown and ignore_scripts stay), never below the app's minimum 1.7.2; BENCHKEEPER_SKILLS_CLI=x.y.z runs exactly that version. The doctor no longer warns about the CLI pin. library.json no longer holds a ref per community skill. benchkeeper insights no longer reports skills behind the library (it needs the network). A repository with many skills counts as moved when any file in it changed.
  • Plugin version 6.24.0.
  • Upgrade: npx benchkeeper@6.24.0 apply.

benchkeeper 6.23

Choose a tag to compare

@github-actions github-actions released this 27 Oct 02:01
ab4b14f
  • App, benchkeeper insights --json (#321, part of #273): findings with a stable id, a severity, the evidence and a fix from three signals. Installed Claude Code skills (global folders and enabled plugins) that none of the sessions of the last 90 days used, with the tokens their descriptions cost per turn: it needs 20 sessions in that window and reports nothing below that. The doctor's findings, as hygiene. Pins behind the newest release, as freshness: the doctor's pins and outdated items and the skills of registered projects behind the library. --project dir adds that project's doctor checks. Insights is a view of its own in the UI, which comes in a later child; the text form of the command does too. The keys are in the table of machine-readable documents in app/README.md.
  • Plugin version 6.23.0.
  • Upgrade: npx benchkeeper@6.23.0 apply.

benchkeeper 6.22

Choose a tag to compare

@github-actions github-actions released this 27 Oct 00:02
9a28b89
  • App, Extensions view and benchkeeper extensions list [--agent claude|codex] [--project dir] [--json] (#217, shares the data model of the Rules view): the MCP servers, hooks and subagents Claude Code and Codex load, plus the commands and output styles of Claude Code (Codex has none listed), read from files only (no agent starts) and shown per agent, globally or for a registered project. Each row has its scope, owner (BenchKeeper, plugin, project, managed, yours), the file it comes from and a "runs code" mark for hooks with a command, stdio MCP servers and commands that run a shell line; a global server or hook of yours links to Doctor's removal offer. MCP servers show name, transport and whether they are on, off or wait for approval. Environment values and headers never appear, a URL shows scheme and host only, a command line is masked, and a file that does not parse is named without its text. Read-only: toggles wait for #184 and #188.
  • Plugin version 6.22.0.
  • Upgrade: npx benchkeeper@6.22.0 apply.

benchkeeper 6.21

Choose a tag to compare

@github-actions github-actions released this 26 Oct 23:02
1b1912e
  • App, usage per account and limit notices (#338, part of #300): benchkeeper accounts track <name> [--off] adds a status line script and a StopFailure hook (rate_limit, authentication_failed, billing_error) to a Claude Code account's folder, as a plan with a diff and undo (a dev container account is written through devcontainer exec); an existing status line of another kind is refused. The script keeps only the used percentage and reset time of the 5-hour and 7-day window and the type and time of a failed turn, never a credential. accounts status (--json key usage), doctor and the Accounts view show what is left per window (100 minus used_percentage; a window past its reset time is unknown, not full), a notice below twice the minimum and the latest failure.
  • accounts limits [name] sets the minimum headroom (defaults 8 percent for the 5-hour and 3 percent for the 7-day window, per account too), the notice level, --pool and the launch settings. run below the minimum shows the account, window, what is left and the reset time and asks before it starts (--yes or --continue-below on starts anyway). Opt-in auto-pick, off by default (--auto-pick on): a launch without --account under a default account below its minimum starts under the pool member with the most headroom, or stops with the earliest reset time, or with --when-none wait waits for it; never inside a running session. The provider's consumer terms prohibit bypassing protective measures and are silent on switching accounts at a limit, so the setting says so and the owner carries that call. Codex has no documented usage source and shows none.
  • Breaking, before 7.0 (no compatibility path): accounts list --json has pool and limits per account and settings; accounts status --json has usage per account; accounts.json may hold settings and, per account, limits and pool.
  • Plugin version 6.21.0.
  • Upgrade: npx benchkeeper@6.21.0 apply.

benchkeeper 6.20

Choose a tag to compare

@github-actions github-actions released this 26 Oct 20:01
9fb88ef
  • App, two security controls in the Claude Code settings catalog (#313, ranks 5 and 6 of the control candidates in docs/security.md, part of #272): env.CLAUDE_CODE_SUBPROCESS_ENV_SCRUB (strips credentials from the environments of Bash commands, hooks and stdio MCP servers; GitHub tokens and proxy settings stay) and permissions.deny, each with its documentation link and a one-line reason. doctor warns while the scrub is not on (1, taking the value of the highest settings layer, else the process environment) and while no Read deny rule that reaches a home folder (the home anchor, //**/ or an absolute home path) covers each of the .ssh, .aws and .config/gh folders; it skips both warnings while a settings file cannot be read. benchkeeper settings set env.CLAUDE_CODE_SUBPROCESS_ENV_SCRUB 1 turns the scrub on. Setup writes neither: a subprocess that needs a removed variable would break, and the deny rules merge into a list the user may already have. A deny rule does not cover a script that opens the files itself. Existing catalog entries are unchanged.
  • Plugin version 6.20.0.
  • Upgrade: npx benchkeeper@6.20.0 apply.

benchkeeper 6.19

Choose a tag to compare

@github-actions github-actions released this 26 Oct 19:37
7d99562
  • App, benchkeeper maintain knowledge show|clear [dir] (#314, part of #274): show prints the entries the maintaining-repos skill stored for a project in .benchkeeper/knowledge/ (--json prints dir, path, files and ignored in the usual envelope); clear removes that folder after a question (--yes skips it) and nothing else of the project. A file that does not parse, is a link or is larger than 1 MiB is ignored with a reason, a linked .benchkeeper is never followed or removed, and token shapes in stored text are masked in the output.
  • maintaining-repos reads the project's local knowledge base before a pass and writes verdicts back after it (knowledge.md next to SKILL.md: files and entry shapes, reuse only while the sha256 of every file the entry names is unchanged, expiry of reference-based and rejected verdicts, size cap, modes 0700 and 0600, stored text is data). Breaking, before 7.0 (no compatibility path): the baseline file .benchkeeper/maintenance.json is replaced by .benchkeeper/knowledge/verdicts.json; an old file is not read.
  • Plugin version 6.19.0.
  • Upgrade: npx benchkeeper@6.19.0 apply.

benchkeeper 6.18

Choose a tag to compare

@github-actions github-actions released this 26 Oct 16:02
46d0cf0
  • orchestrating-projects author filter for worker text (#311, rank 1 of the control candidates in docs/security.md): the worker spec template has a Trusted authors: line. A worker follows issue, pull request, review and comment text only from the policy's trustedAuthors, the repository owner and users with write access (authorAssociation OWNER, MEMBER or COLLABORATOR), and reads any other text, bots included, as data, never instructions. The coordinator picks only issues written by a trusted author. Breaking, before 7.0 (no compatibility path): a review bot's comments are no longer followed unless its login is in trustedAuthors of docs/agents/orchestration.json (new optional policy field, example in policy.example.json).
  • Skill lint: .github/scripts/skill-lint.mjs fails a skill whose files mention a worker and hand it issue or pull request text without the Trusted authors: line and the words "data, never instructions" (a failing and a passing fixture in skill-lint.test.mjs). The filter is wording in the worker session, so it lowers the hit rate and is no boundary.
  • Plugin version 6.18.0.
  • Upgrade: npx benchkeeper@6.18.0 apply.

benchkeeper 6.17

Choose a tag to compare

@github-actions github-actions released this 26 Oct 06:01
f06410e
  • App, update-all for chosen projects (#280): benchkeeper update-all [dir..] takes registered project folders and updates only those, with --dry-run, --baseline, --max and the cap on pushed repositories counting only the list. Without folders it behaves as before. A folder that is not a registered project stops the command before anything is listed or changed, with exit 2 and the message <dir>: not a registered project. The browser's Update all already took a project list.
  • Breaking, before 7.0 (no compatibility path): the doctor message for a skill file that links a missing target now starts with the skill and the project (skill <name> in <project dir>: <file> links <target>, which does not exist); a script that matches the old text <file> links ... at the start of the message must match the new one. The link in the accessibility library skill to its sibling web-quality-audit is still reported: #276 did not remove that skill, and the dead link is in the upstream text.
  • Plugin version 6.17.0.
  • Upgrade: npx benchkeeper@6.17.0 apply.

benchkeeper 6.15

Choose a tag to compare

@github-actions github-actions released this 26 Oct 04:02
97414ed
  • maintaining-repos covers general improvement and cleanup (#274): the pass now starts on a clean tree and adds three parts, each in a file next to SKILL.md that is read only when its part runs: cleanup with a tool table per stack and pinned versions (tools.md: knip, jscpd, dependency-cruiser, with the Python and Rust candidates marked unverified), architecture checks (architecture.md: cycles, forwarding modules, one source of truth, documented against actual structure, and what to do without a GLOSSARY.md or ADRs), and a test audit by mutation testing (test-audit.md: StrykerJS through its command runner, a bound of one module per run, concurrency 4, a memory cap, a heap cap and a per-mutant timeout, survivors sorted into equivalent, unreachable, message and gap, and a test is removed only when the module's mutation result is unchanged without it). The report is one ranked list with a local baseline file, .benchkeeper/maintenance.json, that the next pass compares against. The rule "remove a test only together with the code it covers" now also allows removal with a recorded reason. The description stays a user-started skill (disable-model-invocation), so it costs no context in Claude Code. New eval set skill-maintaining-repos (should-not-load cases; not run). Evidence: the research comment on #274 and evaluations/practices/code-health-pass.md.
  • Plugin version 6.15.0.
  • Upgrade: npx benchkeeper@6.15.0 apply.