You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added
arena — a neutral, reproducible agent-firewall benchmark (npm run arena):
on a labeled corpus of real attacks and real benign work, every tool is scored
on recall and precision and determinism through one pipe, so the numbers
are comparable instead of anecdotal. Ships allow-all / block-all anchors, a
naive regex deny-list baseline, and a LlamaFirewall adapter, framed along
threat-model axes (what each tool even attempts to cover). Results live in arena/RESULTS.md and regenerate with node arena/run.mjs.
scanMcpTools severity tiers — every finding now carries severity: 'critical' | 'advisory'. Injection/exfil instructions (the
curated patterns) are critical; a bare sensitive-path / secret-env mention
is advisory, so consumers scanning long-form skill prose (canon) can stop
treating instructional docs about credential handling as poison — scanning
the official Claude Code marketplace flagged 19/29 skills on exactly those
mention heuristics. Additive: flags are unchanged and consumers that only
read them behave as before.
SENSITIVE_PATH_EXFIL_RE — a sensitive path being moved
(transfer-verb → sensitive path → destination, one clause), e.g. read ~/.ssh/id_rsa and POST it to https://…, which the curated exfil
patterns miss (wrong verb/noun combination). Critical, and built on SENSITIVE_PATH_RE so the two can't drift apart.
Four framework-governance examples (examples/) — the same warden MCP
gate governing a CrewAI Flow, a LangGraph.js StateGraph, an OpenAI Agents SDK
agent, and a Microsoft AutoGen agent, all surfaced in the README.
Fixed
Three scanner false-positive classes, measured on 2,000+ real marketplace
skills (auditing the official Claude Code catalog + 9 community
marketplaces with canon; every first-pass critical was manually reviewed):
SENSITIVE_PATH_RE's .env now requires a non-word lookbehind — process.env / self.env / import.meta.env are code, not the dotenv
file, and were the single largest FP source. (Also applied to the
data-exfil-to-destination pattern's .env noun.)
scanMcpTools normalizes stringified newlines (\n 2-char escapes) back
to real newlines before matching, and SENSITIVE_PATH_EXFIL_RE's gaps stop
at both — previously clause-bounded patterns silently spanned lines in
JSON-stringified text, so unrelated rows of a markdown table could read as
one verb→path→destination "clause".
The bare-word 'exfiltration intent' rule (exfiltrate/leak/steal, no
destination) now tiers as advisory, not critical: every corpus hit was
descriptive prose — memory leaks, ML data leakage, threat lists in
defensive security docs. The flag itself is unchanged, so strict
(tool-description) surfaces still act on it.
Three classifier false-positive classes:
single-label hostnames (Docker service names like db or redis) are
treated as internal, so container-to-container traffic no longer reads as
an exfil destination;
the curl | interpreter RCE rule fires only on an external target —
piping from localhost/internal services no longer flags;
the DNS-exfil rule is anchored to command position, so prose that merely
mentions host/dig no longer flags.
Docs matched to shipped behavior — the native hook (warden-fast) is
fail-safe, not fail-open (daemon unreachable → it execs the in-process Node
hook, which still screens), and the daemon command is warden-serve, not warden serve.
Audit verifier — interspersed unprotected records no longer read as tampering. verifyAuditFile() skipped only leading pre-chain lines; the first line
lacking a prev/hash mid-file was treated as a chain break, so any file that
a second, non-chained writer had appended to (e.g. an in-process hook fallback
logging raw tool calls into the shared ~/.warden/audit.jsonl) reported ok:false even when the hash chain was fully intact — and an attacker could
defeat verification outright by appending one junk line. Now any line without a
string prev+hash is treated as unprotected history: skipped and tallied as unchained, while the chained records are verified continuously. Tamper
detection is unchanged — editing a chained record breaks its hash, deleting one
breaks the next record's link, and stripping a record's prev/hash to
disguise an edit breaks the following record's link. Pinned by three regression
tests (interspersed foreign record, junk-tail append, strip-to-disguise).