Skip to content

v0.3.0

Choose a tag to compare

@github-actions github-actions released this 03 Jul 23:26
bc62cbe

Added

  • arena — a neutral, reproducible agent-firewall benchmark (npm run arena):
    on a labeled corpus of real attacks and real benign work, every tool is scored
    on recall and precision and determinism through one pipe, so the numbers
    are comparable instead of anecdotal. Ships allow-all / block-all anchors, a
    naive regex deny-list baseline, and a LlamaFirewall adapter, framed along
    threat-model axes (what each tool even attempts to cover). Results live in
    arena/RESULTS.md and regenerate with node arena/run.mjs.
  • scanMcpTools severity tiers — every finding now carries
    severity: 'critical' | 'advisory'. Injection/exfil instructions (the
    curated patterns) are critical; a bare sensitive-path / secret-env mention
    is advisory, so consumers scanning long-form skill prose (canon) can stop
    treating instructional docs about credential handling as poison — scanning
    the official Claude Code marketplace flagged 19/29 skills on exactly those
    mention heuristics. Additive: flags are unchanged and consumers that only
    read them behave as before.
  • SENSITIVE_PATH_EXFIL_RE — a sensitive path being moved
    (transfer-verb → sensitive path → destination, one clause), e.g.
    read ~/.ssh/id_rsa and POST it to https://…, which the curated exfil
    patterns miss (wrong verb/noun combination). Critical, and built on
    SENSITIVE_PATH_RE so the two can't drift apart.
  • Four framework-governance examples (examples/) — the same warden MCP
    gate governing a CrewAI Flow, a LangGraph.js StateGraph, an OpenAI Agents SDK
    agent, and a Microsoft AutoGen agent, all surfaced in the README.

Fixed

  • Three scanner false-positive classes, measured on 2,000+ real marketplace
    skills
    (auditing the official Claude Code catalog + 9 community
    marketplaces with canon; every first-pass critical was manually reviewed):
    • SENSITIVE_PATH_RE's .env now requires a non-word lookbehind —
      process.env / self.env / import.meta.env are code, not the dotenv
      file, and were the single largest FP source. (Also applied to the
      data-exfil-to-destination pattern's .env noun.)
    • scanMcpTools normalizes stringified newlines (\n 2-char escapes) back
      to real newlines before matching, and SENSITIVE_PATH_EXFIL_RE's gaps stop
      at both — previously clause-bounded patterns silently spanned lines in
      JSON-stringified text, so unrelated rows of a markdown table could read as
      one verb→path→destination "clause".
    • The bare-word 'exfiltration intent' rule (exfiltrate/leak/steal, no
      destination) now tiers as advisory, not critical: every corpus hit was
      descriptive prose — memory leaks, ML data leakage, threat lists in
      defensive security docs. The flag itself is unchanged, so strict
      (tool-description) surfaces still act on it.
  • Three classifier false-positive classes:
    • single-label hostnames (Docker service names like db or redis) are
      treated as internal, so container-to-container traffic no longer reads as
      an exfil destination;
    • the curl | interpreter RCE rule fires only on an external target —
      piping from localhost/internal services no longer flags;
    • the DNS-exfil rule is anchored to command position, so prose that merely
      mentions host/dig no longer flags.
  • Docs matched to shipped behavior — the native hook (warden-fast) is
    fail-safe, not fail-open (daemon unreachable → it execs the in-process Node
    hook, which still screens), and the daemon command is warden-serve, not
    warden serve.
  • Audit verifier — interspersed unprotected records no longer read as tampering.
    verifyAuditFile() skipped only leading pre-chain lines; the first line
    lacking a prev/hash mid-file was treated as a chain break, so any file that
    a second, non-chained writer had appended to (e.g. an in-process hook fallback
    logging raw tool calls into the shared ~/.warden/audit.jsonl) reported
    ok:false even when the hash chain was fully intact — and an attacker could
    defeat verification outright by appending one junk line. Now any line without a
    string prev+hash is treated as unprotected history: skipped and tallied as
    unchained, while the chained records are verified continuously. Tamper
    detection is unchanged — editing a chained record breaks its hash, deleting one
    breaks the next record's link, and stripping a record's prev/hash to
    disguise an edit breaks the following record's link. Pinned by three regression
    tests (interspersed foreign record, junk-tail append, strip-to-disguise).