Repository navigation
v0.2.0
Auto-released from the merge of #19 (trigger: push).
Faster audits:
-
Requests now run up to six at a time, and the work that doesn't depend on order is overlapped:
- the remaining run pages, once the first page gives the count;
- every sampled run's jobs;
- the workflow files;
- the repository's metadata and listings.
Measured on three real repositories, an audit went from 33 s to 9.5 s and from 21 s to 7 s, with
the same API calls. -
The 25-call reserve holds with requests in flight: each one counts as spent until it is answered,
and an answer that arrives late with an older, higher count doesn't raise it. -
A run that moves between pages while they are read is counted once.
Security review (hostile input from the audited repository):
- No more slow matching. A long unterminated
${{in a workflow value took minutes to hours to
match, and a long job name took seconds. Both checks are now linear (prefix and suffix tests,
and in-order literal search). - Report text stays text. Every workflow, job, step and file name in the markdown report and in
comparedisplays as written: no new heading, no HTML or hidden comment, no link or tracking
image, no terminal control codes. A bare URL or email address, which GitHub would link by itself,
is shown as code. Example blocks can't close their own fence. - The token goes to
GITHUB_API_URL(https only). On GitHub Enterprise Server or GHE.com it no
longer goes to api.github.com. - Requests are bounded. File paths are percent-encoded,
.and..aren't accepted as an owner
or repository name, each request has a 30 s deadline, and an answer over 20 MB is an error (an
error answer is read only up to 64 KB). - The injection scan reads more fields: a
workflow_run's display title and its pull requests'
head refs, committer names and emails, and head repository descriptions. It also follows
untrusted text throughenv:, so${{ env.TITLE }}counts whenTITLEholds one of these.
The report says what the scans don't check. - The HTML report loads no web fonts, so opening it contacts no one.
- The README notes that the GitHub CLI's token usually has write scope, and that a fine-grained
read-only token is enough.
Found by auditing real repositories:
- A workflow is labelled with its own name from the workflows API. The list is read page by page
until every workflow the runs name is found; that is usually one call. A workflow with
run-name:names each run after its pull request, so the most common run name could be a pull
request's title. If the workflows answer can't be read, the run names still label it. - The repository root is listed as
contents, notcontents/. The trailing slash is not the
documented form, and a proxy in front of the API refused it. - Behind a proxy, the CLI and the action now use it. Node's fetch ignores
HTTPS_PROXYunless
NODE_USE_ENV_PROXY=1is set at start-up, so every call went direct and failed (a 401, or no
connection). When a proxy is set, they run themselves again with the switch on (Node 22.21 and
later). On an older Node they warn that the proxy is ignored.
Security scans of every workflow file (up to 30, not only the sampled workflows'):
- actions not pinned to a commit: a step or reusable-workflow
uses:on a tag or branch, which
can be moved to other code, with third-party and GitHub's own actions told apart. The fix pins
the full SHA with the tag in a comment. - untrusted input in a script: a
run:script or github-scriptscriptthat interpolates a
title, body, comment, commit message or author, or a head branch name, per GitHub's hardening
guide. The expression is parsed, so only one whose value carries that text counts: a
contains(...)test, a comparison, a field that only picks a branch (title && 'yes' || 'no') or a
quoted field name does not, and property names match in any case. The example is the
pattern on the step it names (the expression inenv:, the variable read in the step's language),
and the fix names what a variable does not fix: a second parser, an option that runs a program,
an env file, a dash-led argument. - job token wider than the job:
permissions: write-all(confirmed), or no permissions block,
leaving the repository's default token (inferred), including a reusable-workflow call that hands
the caller's token on. The fix is a read-only default, widened per job. - All three are risks, ranked first. The report says how many workflow files the scans read, of how
many listed, and names any that could not be read; when the directory cannot be listed it says the
total is unknown.
Measurement precision:
- A weekly total is now a count times a mean, never a median: run times are skewed by a few long
runs, and the median understated the week. The table's per-run figure stays the median, labelled
typical; the savings that multiply a per-run cost (duplicates, outdated runs, reruns) use the mean. - Every outcome is costed at its own measured cost: up to two failed, cancelled and timed-out runs
per sampled workflow are timed beside the successes (cost_by_outcomein the JSON), since a
failure often stops early and a timeout runs to its limit. An outcome with no sampled run is
costed like a success and named in the report. Runs that ran no job (skipped, waiting for
approval, failed to start, stale) cost nothing.
Recommendations:
- New rule jobs without a working timeout: a job with no
timeout-minutes(GitHub's default is
360 minutes), or one more than 10 times its longest sampled run, gets a limit from its own runs:
3 times the longest, rounded up to 5 minutes, at least 10. Matrix legs are matched to their job.
Reported as a guard against hung jobs (guardunit), not a saving.
Self-hosted runners (the askalf/pitlane method, measured):
- New rule fork pull requests run on self-hosted runners: each job's
ifandruns-onare
evaluated as GitHub would for a pull request from a fork (a small evaluator for the expression
language,src/expr.mjs), so a guard counts by what it does: a job that picks a self-hosted
runner for a fork is a risk, one whoseifis false for it or whoseruns-onsends it to a
GitHub-hosted runner is not. A value the audit cannot resolve (vars,matrix) makes it
possible, not confirmed. The example wraps the job's own runner selection unchanged. - New rule self-hosted runners reused across jobs: a runner id that took two or more sampled
jobs is a persistent runner, whose workspace and state carry between jobs. An ephemeral row that
keeps its name but registers anew each job is not reuse. - Both carry a
riskunit: they rank before every saving, lead the report under "Start with", and
the summary counts them as risks to fix before any saving.
API budget:
- When GitHub sends its
x-ratelimit-limitheader, the report states the budget the audit drew on: the calls
made and what the token had left of its hourly limit ("41 API calls, 959 of 1000 left this
hour"), in the CLI's summary line, the action's log, the report's "How it was measured" and the
JSON (rate_limit). On a budget of 1,000 or less it names the larger one: a GitHub App's
installation token, 5,000 or more an hour. - A stop near the rate limit says the hourly limit it hit and how to get a larger budget.
- The README's "Rate limits" section sets up a GitHub App token for the action, with the pinned
actions/create-github-app-tokenstep and the two read permissions an audit needs.
Expert guidance:
- Every finding now carries its trade-off in the report, as docs/rules.md always described, and,
where the fix is a change to the workflow file, the YAML for it: the setup action's cache input
for the package manager the workflow installs with (npm, pnpm, yarn, pip, poetry, pipenv, uv,
go, bundler) or an actions/cache step keyed on the lockfile (cargo, bun, npm with no lockfile),
the concurrency block, the push trigger, or the checkout inputs. - slow checkout reads the checkout step:
fetch-depth: 0, afetch-depthabove 1,lfs,
submodulesorfetch-tagsare named as the cause (confirmed); a default checkout, which
already fetches one commit, is no longer told to setfetch-depth: 1, and gets sparse-checkout
instead (inferred); with no checkout step to read, or a depth set by an expression, the cause is
reported as unknown. - The same-commit-twice rule reads the repository's default branch (one more API call): the push
trigger example names it, and when it cannot be read says to fill it in, instead of assuming
main, which would switch off push CI on a repository whose default branch is develop. comparereports a finding that is gone in its own unit (job-minutes, billed minutes or
developer wait), not always as job-minutes.- The outdated-runs fix no longer suggests an inline
{ group: ${{ ... }} }concurrency mapping,
which is not valid YAML; the example is a block.
Measurement:
- Runs are grouped by workflow id, not by run name. A pull request that adds a workflow gives its
runs the pull request's title, which split one workflow into two rows (this repository's Fuzz
appeared twice, once as "[hold] Fuzz the workflow, …"). Each workflow is labelled with the name
most of its runs carry; two workflows sharing a name get their path appended. - A finding that saves under 1 minute a week is left out. Queue wait and matrix size carry a
waitunit and stay, reported as developer wait; before, any saving that rounded below 1 was
mislabelled as developer wait. - Billed minutes: GitHub bills each hosted job rounded up to a whole minute. Each workflow
getsbilled_minutes_week(self-hosted jobs count 0), the table a Billed min/week column, and
comparea billed row and total. - New rule short hosted jobs bill a whole minute: hosted workflows whose billed minutes are at
least twice their job-minutes, by 30 minutes a week or more. Its saving is in billed minutes,
and the report says so. It does not fire on a public repository, whose standard hosted runners
are free; the report says so beside the billed figure (visibilityin the JSON, read from the
repository's metadata). - Developer wait is shown only for workflows somebody waits on (mostly push, pull request or merge
group runs); a scheduled or chained workflow shows none (waited_onin the JSON). - The queue-wait evidence says how many sampled jobs its p90 is over.
Ease of use and understanding:
-
The report opens with Start with (the first finding and its saving), then what the findings
add up to. A finding that saves developer wait is no longer summed as "0 job-minutes". -
Each finding is its own section: where, saves, confidence, why, fix, trade-off, YAML.
-
A legend explains the table: job-minutes against billed minutes, developer wait, queue, and each
share (work, deps, overhead, head, tail and the rest). -
Step names drop their action ref (
Run actions/checkout@3d3c42e…readsRun actions/checkout),
and steps under a second are left out of the longest steps. -
--only NAME(CLI, repeatable or comma-separated) and the action'sonlyinput time just the
named workflows, for measuring one workflow before and after a change. Names match what GitHub
shows, so two workflows that share a name are both selected; a name with no runs is reported. -
The HTML page carries the same: billed minutes, trade-offs, examples, the legend, and a
developer-wait tile taken from a workflow somebody waits on. -
Fuzzing: three Jazzer.js targets run weekly and on every pull request that touches them
(.github/workflows/fuzz.yml), over workflow files, API run history and saved report files, with
a persistent corpus. Jazzer is installed from.clusterfuzzlite/, so the package keeps no
dependencies..clusterfuzzlite/also builds the targets for a weekly ClusterFuzzLite coverage
report. -
ci-wizard compareno longer crashes on two kinds of hand-edited report: a measurement that is not
a number shows as-, and a report whose workflows or findings are not objects, or whose
printed fields (the repository, a workflow name, a finding's text) are objects, is rejected with
the usual "is not a ci-wizard --json report" message. Found by the new fuzzing. -
A workflow file whose job lists
stepsas something other than a list, or whose top level is a
list, no longer stops the audit: it yields no steps or facts, as an unparseable file does. -
Releases are automatic, as in askalf/dario: the version bump on
mainis the release.
publish.ymltags the merge commit, cuts the GitHub release with that version's CHANGELOG
section, attaches the attested tarball (.sigstore.jsonand.intoto.jsonlbeside it), and
publishes to npm (still gated onNPM_PUBLISH). It fills only what is missing, and an hourly
run finishes a release that failed part-way. -
npm testfails a version with no CHANGELOG section; CI fails an outside contributor's pull
request that changes the version (scripts/check-release-authority.mjs).
Built, tested and packed by publish.yml on this run; the tarball attached here carries a keyless Sigstore build attestation (gh attestation verify <tarball> --owner askalf).