kit 5.2.0
Added
-
kit skill test --runtimenow proves negative controls HELD, not just detects violations.
A forbidden action a gate denied never runs, so it is absent from the transcript — it lands
in.kit-audit.jsonlas agate-egress/gate-fsdeny. The gate now stamps the PreToolUse
session_idon that deny event as a join key, and the runtime audit attributes each deny
to the skill run that was active when it fired (session + span time-window — precise and
session-bounded, not a global timestamp guess). The negative-control check can now report
"control held — N forbidden attempt(s) denied" with evidence, closing the path that was
deferred in the previous increment for lack of a join key. Best-effort: no audit log → no
denial evidence (never breaks the audit); a denied action isdeniedForbidden, never counted
as a violation that ran. Deterministic, zero-LLM. -
kit skill test --runtimenow checks egress/fs target-scope, not just tool-scope.
When a signed broker scope is present, each span-attributed action is enriched with a
broker verdict via the SAME decisions the gate-egress / gate-fs enforcers apply
(checkEgress/checkFsWrite): aBashcommand's hosts, aWebFetchURL, and a
Write/Editfile_pathare checked against the signed[scope]. This catches a skill
that stays within its declared tools but uses an allowed tool (Bash/WebFetch) to reach
an off-scope host or write outside the allowed roots — the exfil-via-allowed-tool vector
— folding it into the existingadherence/negativeverdicts. Without a verified scope,
tool-scope adherence still applies (verdicts stay undefined). Deterministic, zero-LLM. -
kit skill test --runtime(experimental) — the recorded-run audit that closes two
of the checks P1 disclaimed. kit never runs a skill (zero-LLM); it audits what the
agent already recorded. The memory transcript index (tool_uses) reconstructs each
skill run as a span — aSkillinvocation opens it, following tool calls until the
nextSkillcall or session end are that skill's actions — and two checks now decide
from that evidence: adherence (an out-of-declared-scope tool that actually ran →
fail; all in-scope across N observed runs →pass; no recorded run → honestskip)
and negative controls (a forbidden action that succeeded →fail; none attempted →
skip"not exercised", never a false-green pass). Low-confidence attribution downgrades
a would-be fail to an inconclusiveskip— kit never blames a skill for an action it
cannot attribute. Off by default;--runtimeopts in. Honest coverage limit surfaced:
verdicts cover observed runs, not all runs; denial-based "control held" evidence and
egress/fs target-scope adherence are later increments.rubricstays OUT forever (LLM —
delegated). Design: kit-research skill-test-p2-runtime-adherence note.
Full changelog: https://github.com/sandstream/kit/blob/v5.2.0/CHANGELOG.md
Verify this release:
git tag -v v5.2.0
npm audit signatures