Skip to content

AgentLock v1.4.0: Selective Action-Class Gating

Choose a tag to compare

@webpro255 webpro255 released this 10 Jul 14:07

v1.3 proved the security property. v1.4 makes it deployable.

The v1.3 uniform write-gate achieved 0% defense-effective attack success on AgentDojo's write-trailing-read threat models, at a utility cost the paper stated plainly. v1.4 attacks that cost where attacking it is sound, and refuses where it is not.

Selective action-class gating

Instead of blocking every consequential write on session taint, tools declare which action classes they belong to. Value-free classes (deletions, membership changes) stay taint-gated, because their malice needs no distinctive parameter value for lineage to catch. Value-carrying writes can be released to parameter lineage, which provably covers them (paper, section 7). The polarity rule is structural: declarations that add gating are free, and the one declaration that removes gating (is_value_carrying) can only come from the tool's trusted registration block, never from a caller. A wrong addition over-blocks; a wrong removal fails open.

Benchmark results, pre-registered

Predictions were committed before any run launched (PREDICTION_v14.md). All runs: gpt-4o-mini, tool_knowledge attack, AgentDojo 0.1.35, fixed adapter classification.

suite undefended, attacked uniform gate (v1.3) selective gate (v1.4) benign ceiling
travel 36.43% 30.00% 51.43% 65.00%
slack 52.38% 4.76% 4.76% 76.19%

Travel: selective gating restores utility to 79% of the benign ceiling, from 46%, and the defended agent outperforms the undefended agent under attack. Defense-effective ASR went from 2.14% (persuasion residual) to 0.00%. The mechanism is visible at parameter level in the decision logs: the attacker's injected reservation was denied by parameter lineage in the same episode where the user's own reservation committed.

Slack: no utility recovered, by design. Slack has no honestly declarable value-carrying tool, and the control run confirms selective gating refuses to relax where relaxing is unsound. Stated plainly: on slack-shaped workloads (benign read-then-write over attacker-reachable content) the defense floors utility at 4.76% whether or not an attack is present, and costs more utility than the attack it prevents. The full four-cell decomposition is in the attached report. v1.4 ships audit tooling so you can measure which case your agent is in before deploying.

Full benchmark report: MORNING_REPORT_v14.md

Two defects found and fixed before release

Both caught by our own verification gates during v1.4 development, neither ever shipped in a release:

  • Schema version comparison was lexicographic, so "1.10" compared below "1.3" and would have silently disabled all lineage defenses at that version. Found by the audit tooling auditing its own coverage rule. Affects v1.3.x only if a permission block hand-declares a version of 1.10 or higher, which no published schema defines. Fixed with numeric comparison that fails closed on unparseable versions.
  • The deferred-commit re-decision ignored action-class declarations entirely, making gate_consequential=False inert whenever deferred commit was enabled. Found by a pre-registered verification gate before any benchmark run launched (PREDICTION_v14_addendum.md). Fixed by extracting the gating disjunct into a single shared predicate used by both enforcement points, so they cannot drift again.

Also in this release

  • Novel lineage: per-target trusted/untrusted/novel classification, exact-token, placed above the coarse taint gate
  • Action-class audit: audit_action_classes() reports every lineage-enabled tool as DECLARED, UNDECLARED, or NOT_COVERED, with two-tier suggestions (observed traffic beats lexical heuristics) and a hard guarantee that the gating-removing declaration is never suggested without requiring human confirmation
  • needs_approval surfaced at the gate boundary: step-up no longer degrades to a hard deny; denial status becomes approval_required while DecisionType is unchanged, so no existing control flow breaks
  • The register_tool hazard warning is removed, replaced by the audit report. Note for strict callers: code running under -W error::UserWarning no longer gets a registration-time raise; assert on audit_action_classes() instead
  • schema/agentlock-v1.4.json published; SCHEMA_VERSION is 1.4; v1.3 blocks remain valid
  • README rewritten, feature history moved to docs/history.md, SECURITY.md gains a stated support policy

1041 tests passing under -W error::UserWarning. Full details in CHANGELOG.md.

Benchmark integrity notes

AgentDojo scores some blocked-but-attempted calls as attack successes (reported upstream as ethz-spylab/agentdojo#168), so we report defense-effective ASR alongside official ASR, with every excluded episode justified in the attached report. Separately, the benchmark never applies temperature=0 due to a falsy-zero guard, so single-run rates carry sampling noise; we measured the spread across replicates (also in the attached report) and the mechanism-level claims are stable across all draws.

Paper: Provenance-Based Pre-Action Authorization for LLM Agents (DOI: 10.5281/zenodo.21270300)

License: AGPL-3.0-or-later with commercial licenses available (see COMMERCIAL.md). Versions 1.2.x and earlier remain Apache 2.0.

PREDICTION_v14.md
PREDICTION_v14_addendum.md
MORNING_REPORT_v14.md