Skip to content

kit 6.9.0

Choose a tag to compare

@github-actions github-actions released this 23 Aug 06:00
· 20 commits to main since this release
5855126

Added

  • Fail on NEW dependency debt, not on the debt you already have (#524).

    A repo with thirty known advisories cannot adopt a gate that fails on all thirty — it gets
    disabled the same afternoon. kit check --category security now carries an advisory baseline
    check that freezes today's debt in a data file and fails the moment something new appears.

    • The repo's own package manager does the auditing — npm, pnpm, yarn or bun, whichever the
      committed lockfile names. No new tool to install, and no cloud service handed the manifest.
    • The known list is data, not code (.kit/advisories.json): GHSA id → package, severity,
      title, sorted, with no timestamp, so a dependency bump is a small readable diff in review rather
      than a code change.
    • The file may only shrink. A baseline entry that no longer applies is a finding of its own.
      Without that rule the list silently accumulates dead ids and the gate stops meaning anything;
      with it, fixing a vulnerability and pruning its line belong in the same commit.
    • Remaining debt is summarised per severity on every run — "no new advisories (npm); known
      debt: 1 critical, 3 high, 4 moderate"
      — so the size is visible without opening the file.

    kit security advisories reports the current state; --accept writes the baseline, adding and
    pruning in one step. Opt-in by construction: with no baseline committed the check skips, naming
    the command that adopts it. It also skips under an air-gap posture rather than pretending the
    registry answered, and an audit that could not run fails as didNotRun rather than reporting a
    clean result.

    The parser is shape-tolerant because the four managers bury the same three facts at different
    depths. That mattered more than expected: bun puts the package name in the object key and in
    no field at all, so a field-only parser found zero advisories in a repo that has twenty-eight —
    and would have reported "no new advisories" over a critical deserialization type-confusion in
    seroval. Reading keys blindly is the opposite failure, since npm nests everything under a
    vulnerabilities key, so structural keys are excluded and a key only counts when it looks like a
    package name. Verified against real output from both, plus pnpm's advisories map.

  • What reaches the browser is now checked (#523).

    kit covered credentials that were committed (trufflehog over history, the staged-file scan) and
    .env hygiene. Neither sees the leak that costs the most: a VITE_* or NEXT_PUBLIC_* variable
    holding a real secret is inlined into the bundle at build time and shipped to every visitor —
    without ever being committed. .gitignore does not protect against it and history scanning cannot
    see it.

    Two checks, both deterministic:

    client-exposed env names — a client-exposed prefix plus a secret-shaped name is a leak by
    construction, because the framework will inline it. Nine prefixes (VITE_, NEXT_PUBLIC_,
    PUBLIC_, REACT_APP_, EXPO_PUBLIC_, NUXT_PUBLIC_, GATSBY_, VUE_APP_, STORYBOOK_)
    against SECRET|TOKEN|PASSWORD|CREDENTIAL|PRIVATE|_KEY|API_KEY. Names that are secret-shaped and
    public by design are excused — ..._ANON_KEY, ..._PUBLISHABLE_KEY, ..._SITE_KEY, a Sentry
    DSN, a client id, a measurement id, a VAPID public key. That list is what makes the check keepable:
    KEY alone matches NEXT_PUBLIC_SUPABASE_ANON_KEY and NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY, the
    two most common client env vars there are, both meant to be published.

    Anything else fails, with the escape hatch spelled out in the suggestion:

    [scan.client_exposed_allow]
    VITE_DEMO_SECRET_KEY = "demo tenant, rotated nightly"

    The reason is required — an entry without one is reported, because "somebody allowed this once" is
    not something anyone can audit later. Only names are read from .env* files; the values are
    dropped at the parse, since the prefix decides exposure and nothing downstream needs the content.

    built bundle secrets — the built output is scanned for real credential shapes, which catches
    the case the name check cannot: a key hardcoded in source that never went through env at all. The
    scanner for this already existed (scanBuildArtifacts) and was reachable only from
    kit security scan-build, so no automatic verdict had ever looked at build output — where the
    leak actually lands.

    Scoped so it stays usable, both limits measured rather than guessed: it requires a client framework
    (kit's own dist/ is a Node CLI, and scanning it produced 73 "credential shapes" — every one a
    test fixture or one of kit's own detection patterns), and it looks in workspace packages, not just
    the root (a real repo declares workspaces at the root and ships from apps/web, so a root-only
    check called it "nothing builds for a browser"). Compiled tests, mocks and fixtures are excluded
    and the count of what was set aside is printed. An unbuilt project reports "present but not built
    — run the build, then re-check"
    rather than passing.

    Verified against a real Vite workspace (apps/web/dist — pass) and against a planted leak: the
    name check fails on VITE_STRIPE_SECRET_KEY while leaving VITE_API_URL and
    NEXT_PUBLIC_SUPABASE_ANON_KEY alone, and the bundle check finds an sk_live_… inlined into
    dist/assets/index-abc.js at critical severity, telling the operator to rotate first — removing
    it from the build does not un-publish what already shipped.

  • kit usage — what kit knows, and proof that the floor still works (#519).

    Six tabs, switchable in the terminal with the number keys: Floor (audited operations and
    refusals per operation), Coverage (the enumerated checks of the last saved run — pass, warn,
    fail, and what could not run, with each reason), Memory (the store you own: its path, its
    size on disk, messages and sessions, tokens generated vs context re-read, and the per-project
    breakdown), Triage (what was checked before it was installed), Machine (every repo this
    machine has sealed an audit log in) and Proof.

    It is deliberately not a token-savings or money report. There is no counterfactual for "what
    would this have cost without kit", and a saved-you-N number without one is invented — the same
    rule the rest of kit is built on. What is measurable is the reliability axis, and the twelve
    could-not-run rows are as much the product as the twenty-one that passed: an agent asked "is
    this safe?" improvises a subset, answers with no denominator, and leaves no record of what it
    skipped.

    kit usage --prove is the part that is not a dashboard. Counting recorded activity proves a
    gate existed, not that it still works, so the Proof tab runs the floor against inputs it must
    refuse — on your machine, offline, in a throwaway directory that is removed afterwards:

    • an install whose target cannot be resolved ($PM install x) must be blocked (exit 2);
    • a command that installs nothing must still be allowed — otherwise the first control proves
      nothing, since a gate that refuses everything is an outage;
    • a staged credential must fail git commit in a temp repo with kit's own hook installed;
    • a clean commit must still go through;
    • and with --deep, two consecutive runs must produce an identical verdict set.

    A control that cannot be set up reports inconclusive with its reason. It never reports a pass
    it did not observe.

    Two further tabs answer the questions a client asks. Standards scores all eight mapped
    standards (ASVS L2, LLM Top 10, SSDF, Agentic Top 10, MCP Top 10, AIUC-1, GCP WAF Security, NIST
    800-53) against the last saved run, and separates the two numbers that are usually conflated:

    standard           verified  mapped   gap  manual
      asvs                    4       7     2       3
      nist-800-53             4      11     0       3
    

    A control kit maps to a check is a claim; a control whose check actually ran and passed is
    evidence. mapped − verified is claimed coverage with nothing behind it yet, and reporting only
    the first number is how a coverage map becomes marketing.

    Keys answers "are my keys exposed" without ever handling a key: how many [secrets.keys] are
    declared and by which backend, which the last run could not resolve, what the history scan found
    split into verified-live / unverified / fixtures / accepted, and whether .env is gitignored.
    Names only — the module never reads a value. A repo with no saved run reports resolution as
    unknown, not as zero resolved, because those look identical and mean opposite things.

    --json for the whole set, --tab <name> for one; a non-TTY prints every tab rather than
    silently handing a piped reader one eighth of the report. Eight tabs no longer fit on one row, so
    the tab row wraps instead of truncating — a truncated row would hide whole dimensions of the
    report behind a key nobody knows to press.

  • kit config sections and a generated docs/CONFIGURATION.md (#527).

    .kit.toml has 23 sections. Six of the most-used — [tools], [services], [secrets],
    [skills], [governance], [hooks] — carried no description anywhere in the type; [mcp] and
    [supply_chain] appeared in no document at all; the rest were explained in passing across
    thirty-odd files. kit config knobs listed 30 knobs covering four sections. So "what can I
    configure, and where do I change it?" had no answer surface — not in the docs, not from the CLI.

    src/config-surface.ts now declares every section with three things and nothing else: what it
    is, what declaring it buys (the reason to bother, not a restatement of the name), and the
    smallest example that does something real — plus the command that sets it up and the document that
    goes deeper, where those exist.

    $ kit config sections
    .kit.toml sections  (23 available · 7 declared here)
    
      ○ [context]  The account and project each CLI must be pointed at, per tool.
          buys: A tool answering as the wrong org becomes a red row instead of a filtered
                result set that looks complete — and a pre-push hook can block the
                wrong-project push outright.
          [context.gcloud]
          account = "me@example.com"
          project = "acme-prod"
          kit context check
    

    docs/CONFIGURATION.md is generated from the same table, and the test fails in both
    directions: a section in kitConfig with no entry (the operator cannot discover what nobody
    described) and an entry for a section kit no longer has (the reader is sent to configure something
    kit will ignore). Unlike the flag surface, over-documenting is an error too. Regeneration is
    byte-identical, with a test that names the first differing line — the lesson from #525.

    kit config recommend remains the "what should I change" surface; this is the reference behind it.

  • Audit entries now say where they happened, and whether they were a test (#526).

    kit's own log had 7 199 events and not one carried a location. Every entry named a key and an
    operation, and no place — so the 4 053 policy-check events, almost all of them the test suite
    exercising the gate, could not be separated from an operator's. kit usage had to print "operator
    and test activity cannot be told apart in these totals"
    instead of a number. The evidence existed
    and was unusable.

    Two fields, and the choice of fields is the point:

    • repoowner/repo from the origin remote, or the working tree's directory name when
      there is no remote. Deliberately not the absolute cwd: this log is exportable
      (kit audit export, the Remote push), and an absolute path carries the operator's username and,
      in a consultancy tree, client names. owner/repo is what the repository already publishes in
      its own remote, so it adds no exposure while still telling two trees apart. A test asserts no
      home path can appear in a serialised entry.
    • test — stamped at write time from NODE_TEST_CONTEXT (which node:test sets and spawned
      CLIs inherit) or KIT_TEST=1. After the fact there is nothing left to distinguish a
      test-generated event by, so it has to be recorded when it happens.

    Resolution is cached per directory: the append path handles thousands of events, and a git spawn
    per event would not be acceptable. A directory whose identity cannot be determined caches that
    too, so a non-git tree does not re-spawn git on every event. An explicitly supplied repo is
    never overwritten — the broker files evidence on another tree's behalf and knows its own answer.

    kit audit shows both, and prints for entries written before the fields existed rather than
    guessing.

Changed

  • A scanner that finds nothing now says what it looked for (#525).

    Some tables have to be hardcoded — nothing in a repository can tell you what a Stripe key looks
    like. What must never be hidden is where such a table ends. "No credential patterns found" reads
    as "there are no credentials"; the true statement is "none of the shapes kit knows". So the
    denominator is part of the claim:

    ✓ scan-build: no matches for 25 known credential shapes — a shape outside that set is not detected
    ✓ scan-artifact: file.js — no matches for 25 known credential shapes
    

    The count is SECRET_SHAPE_COUNT, derived from the pattern list rather than written down: a
    hardcoded count is a claim that rots the first time someone adds a pattern. While adding it, a
    grep for label: said 29 and the derived count said 25 — the grep had counted the interface field
    and a doc line, which is the argument for deriving it in one datum.

    secrets scan's clean line now names its detector too — "no committed secrets (trufflehog
    detector set, full history)"
    — because that path's bound is trufflehog's set over history, a much
    larger one than kit's own pattern list used by scan-staged and scan-build, and a reader cannot
    size the claim without knowing which applied. The fallback path already stated its bound
    explicitly and is unchanged.

    The interactive view uses raw stdin directly rather than readline, which swallowed the
    keypress, and restores the terminal from process.once("exit") plus SIGINT/SIGTERM handlers
    using writeSync — a buffered write is lost when the process exits immediately after it, and a
    view that leaves the terminal in the alternate screen with raw mode on makes the operator's shell
    look dead. Verified through a pty: q, Esc, Ctrl-C, SIGINT and SIGTERM all give the screen
    back.

Fixed

  • Regenerating flag-surface.ts produced a 512-line diff over identical content (#525).

    scripts/derive-command-flags.mjs --emit wrote one verb per line; Prettier reformats that to one
    flag per line. So running the documented regeneration command reflowed the whole file without
    changing a single fact — and I read that diff as the generator having narrowed the accepted flags
    of 70+ verbs, reported it as a defect, and repeated it in two PR bodies and a shared-memory entry
    before comparing either side's content. Measured properly: 0 of 72 verbs differ.

    emit() now produces Prettier's shape (width-based, same printWidth of 100), so regeneration is
    byte-identical, and a test compares the generated string against the committed file and names the
    first differing line. A generated file nobody dares regenerate is a hardcoded table wearing a
    generator's hat.

  • The plugin registry advertised packages that do not exist, and ratings nobody measured (#528).

    kit plugin list printed this:

    stripe/payments 1.0.0 ★★★★◆ 4.8
      Install: npm install @kit/plugins/stripe
    

    Checked against npm and GitHub, every claim on that line was false:

    Claimed Actual
    @kit/plugins/stripe no such package — the real one is sandstream-kit-plugin-stripe, so the install command could not work
    version 1.0.0 the published packages are 0.1.0 / 0.2.0
    github.com/sandstream/kit-stripe HTTP 404
    ★★★★◆ 4.8, 1250 downloads invented — no source exists for either
    updated: <now> new Date().toISOString() at import, so the registry always claimed to be current

    And it listed five of eleven shipped plugins, so kit plugin search cloudflare answered "No
    plugins found matching: cloudflare"
    about a package published on npm at that moment. Six plugins
    — cloudflare, github, sentrux, sentry, snyk, wiz — were undiscoverable through kit's own discovery
    surface.

    The registry is now generated from each plugin package's own manifest: real npm name, real
    version, real description, one real repository URL. rating, downloads and published are
    omitted, and their fields made optional — the display code already treats them as optional, so
    absent means unshown rather than estimated. updated is gone for the same reason.

    Six tests hold it there: every shipped package must be listed, every entry must match the manifest
    it names (package, version, and a runnable install command), no entry may carry a rating or
    download count, every repository link must be the one real repository, each plugin must be findable
    by the name a person would type, and regeneration must be byte-identical.

  • A rebase is not a bypass (#522).

    Every git rebase produced one "commit(s) bypassed pre-commit hook — (sentinel-missing)" entry
    per replayed commit. Four false positives came out of a single day of rebasing branches, on
    commits whose pre-commit hook HAD run — on the original. A false positive in a security banner is
    worse than no banner: it teaches the operator to skip the line, and then the real --no-verify
    scrolls past unread.

    Git replays commits without running pre-commit, but it does run post-commit for each replayed
    commit, so the sentinel is absent and the commit looked bypassed. Measured while fixing it: during
    a replay the post-commit hook sees rebase-merge/ and CHERRY_PICK_HEAD present, an ordinary
    commit sees neither, and GIT_REFLOG_ACTION is not exported to post-commit at all — so the state
    is unambiguous exactly where the decision is made. The event is now recorded as replayed and
    prints nothing. It is still recorded: the log is an audit trail, and "this commit was replayed" is
    a fact worth keeping — it just is not a finding.

    Entries written before this fix are re-classified too, so the four already in a log stop being
    counted without anyone editing an append-only file. The witness is the reflog, which records the
    operation that created each sha. Only messages that create a commit count — rebase (pick),
    (squash), (fixup), (reword), cherry-pick, am — and deliberately not rebase (finish),
    which names the branch tip afterwards and would therefore excuse a genuine --no-verify commit
    that happened to be that tip. That would turn a false positive into a false negative, which is the
    one direction this must never take. A reflog that has expired answers "unknown", and an unknown
    entry stays counted.

    The banner now says N entries came from a rebase/cherry-pick replay — not counted. rather than
    silently reporting a smaller number than the log is long.

  • A skipped scanner was counted as a passing one in the verdict line (#521).

    Run from a workspace root that holds several repos side by side — web/, illithid/ — every
    manifest-dependent scanner skipped truthfully ("no package.json found") and the summary printed:

    All 25 checks passed ✓
    

    Fifteen of those twenty-five had never run. The same command one directory down, where the code
    actually lives, reported 30 known dependency vulnerabilities (high), 22 unpinned dependencies
    and 18 secret-shaped strings in git history. The green line was covering all of it.

    printSummary counted status === "pass" || status === "skip" as OK and then declared "All N
    checks passed" whenever that count reached the total. This is the same defect class as #517 — a
    check that could not run rendering as success — one level up, in the line most people read
    instead of the rows. Now there are three states rather than two:

    All 26 checks passed ✓                              (everything ran, everything passed)
    10 passed  ·  15 could not run                      (everything that ran passed)
    10/26 passed  ·  15 could not run  ·  1 real issue  (something to act on)
    

    A run with no applicable checks at all now says no checks applied here instead of "All 0 checks
    passed ✓". Skips still do not gate — they are counted and named, not failed.

  • A directory with no manifest is not a directory with nothing to scan (#521).

    New scan scope check, enumerated in every security run, because each individual skip was true
    and the missing row was the one stating the consequence — that the verdict described an empty
    directory while the code sat one level down:

    • no manifest here, projects below → warn (high), naming them: "no manifest here — this
      verdict covers none of 3 project(s) below: illithid, web, web-corrupt-backup"
      ;
    • a root manifest declaring workspaces → pass: "12 nested package(s) covered via
      workspaces"
      — a root-level scan genuinely does cover those;
    • a root manifest without workspaces, siblings below → warn (medium): they are separate
      projects and were not scanned;
    • an ordinary single project → pass, quietly.

    The three passing cases carry as much weight as the warning one: a check that warns on every
    monorepo gets switched off within a week, and then the case it was written for goes unnoticed too.

    And in that wrong-directory case the individual skips are no longer left as honest
    not-applicables. "No manifest at this path" and "I looked in the wrong place" are the same
    sentence from a scanner's point of view and opposite facts from the operator's, so each
    manifest-absence skip becomes a warning that says where the code actually is:

    ! npm audit     warn  no package.json found — but 3 project(s) below do (illithid, web,
                          web-corrupt-backup); this scan looked in the wrong place   [medium]
    

    Only in that case. socket scan (cloud-only, excluded by design) and opt-in SAST stay skips, and
    an ordinary project or a workspace root that genuinely covers its children is untouched. Note
    that requiring kit to run where .kit.toml lives would not have helped here: the workspace root
    in the report has its own .kit.toml.

  • kit ci rendered a skipped check as a failure on GitHub and as a pass on GitLab (#517).
    Two surfaces, two opposite lies, one missing case.

    The step-summary icon was pass ? ✅ : warn ? ⚠️ : ❌ — no branch for skip — so a run in a
    directory that is not a project printed twenty-plus ❌ rows above a footer reading
    "2 passed, 1 failed, 1 warnings": the table and the tally contradicted each other in the same
    output, because the summary counted skips and the footer never printed them. Three of those red
    rows were worse than misleading — socket scan is excluded by design (cloud-only),
    bumblebee was switched off by the operator via KIT_BUMBLEBEE, and SAST/guarddog are
    opt-in — so kit's own documented design read as broken.

    The GitLab JUnit writer emitted a bare <testcase> for a skip, and JUnit reads an empty
    testcase as PASSED. Measured on the same run: tests="26" failures="1", zero <skipped> tags,
    24 empty testcases — 24 checks that never ran, reported green.

    Now: skips get their own icon () with the reason they already carried, the footer reads
    N passed, N failed, N warnings, N skipped so the rows add up, the table is ordered
    failures → warnings → passes → skips (twenty always-red rows at the top is how a report becomes
    unread), and JUnit emits <skipped message="…"/> with a skipped= count on the suite.

    Skips still do not gate: allOk remains failed === 0 && (!failOnWarning || warnings === 0).
    Only what the report says changed. The machine-read surface was already correct — annotations
    only ever fired for fail/warn — which is the worse way round, since the human is the one
    who concludes "twenty things are broken here".


Full changelog: https://github.com/sandstream/kit/blob/v6.9.0/CHANGELOG.md

Verify this release:

git tag -v v6.9.0
npm audit signatures