Skip to content

v0.2.0 — IFC propagation (dual-lattice, plan-first)

Latest

Choose a tag to compare

@amurlaniakea amurlaniakea released this 29 Aug 07:29
e9be8b1

Changelog

All notable changes to agent-harness-defense are documented here. The format
is loosely based on Keep a Changelog and the
project adheres to Semantic Versioning.

[0.2.0] — 2026-08-29

Breaking changes

  • run_admission now requires a Plan (declarative) as its third argument;
    the v0.1 proposed_files parameter is removed. The mission string is now
    part of the Plan. Migration:
    # v0.1
    run_admission(repo, label, "mission text", proposed_files=[...])
    # v0.2
    run_admission(repo, label, Plan(mission="mission text", steps=[...]))
  • ahd run PATH is now ahd run REPO --plan PATH (YAML Plan). The plan is
    the unit of admission, not the on-disk materialized state.

Added

  • Dual-lattice IFC (confidentiality + integrity) per arXiv:2608.27234 (SPA,
    Girrens & Wang). SourceTag, Label (componentwise join: confidentiality
    = max, integrity = min so "taint does not wash"), PlanStep, Plan,
    PlanVerdict, evaluate_plan, plan_from_yaml in
    agent_harness_defense.ifc.
  • Propagation by depends_on (control-flow) and value_source (data-flow):
    a write driven by an UNTRUSTED read inherits UNTRUSTED integrity and is
    denied (no-upgrade); a SECRET value written to a public sink is denied
    (no-downgrade). Both axes reported separately.
  • Reads classified by path (_classify_read_path): repo text → UNTRUSTED,
    system paths → SYSTEM. Pure, no filesystem access.
  • INCIDENT_REPORT_INJECTION scenario (AC-EVAL-1) + assert_plan_matches_ materialized guard against Plan↔materialized drift.
  • 23 tests (was 6 v0.1): 7 v0.2 IFC (3 lattice + 4 propagation) + 3 conftest
    • 2 false-positive guards + 7 v0.1-adapted (3 IPI + 2 monitor + 1 regression
    • 1 loop-state). The v0.1 regression guard now monkey-patches
      evaluate_plan to prove the eval is non-vacuous.

Changed

  • The v0.1 keyword heuristic (ESCALATION_TRIGGERS + LoopStateMonitor) is
    now a SECOND signal (flagged_by_keyword / cross_iteration_signal), not
    the admission decision. The IFC verdict is authoritative.
  • InstructionLevel and _level_of_file removed (superseded by the
    dual-lattice in ifc.py).

Closed limitations (see KNOWN_ISSUES.md §1)

  • AC-IFC-1 (lattice enforcement, both axes)
  • AC-IFC-2 (data-flow propagation)
  • AC-IFC-3 (control-flow propagation, both axes)
  • AC-IFC-4 (false-positive guards)
  • AC-IFC-5 (no-regression: v0.1 scenarios adapted, teeth guard added)
  • AC-IFC-6 (eval is not vacuous: v0.1 missed AC-EVAL-1, v0.2 catches)
  • AC-IFC-7 (offline, <1s/test)

Remaining (v0.3, see ROADMAP.md)

  • Label-preserving persistence between iterations (arXiv:2608.27234
    §label-preserving).
  • AgentDojo / AgentDojo-MQ benchmark wiring.
  • The 6 real coding-agent harnesses from arXiv:2608.27299.

[Unreleased] — v0.1.0 (2026-08-28)

First public release. Open admission-layer defense for LLM agent harnesses.

Added

  • run_admission() with forbidden-path quarantine gated by a cross-iteration
    escalation monitor (arXiv:2608.27141).
  • LoopStateMonitor retaining state across loop iterations; observes ALL
    untrusted repo text (planted input) separately from the admission decision.
  • InstructionLevel enum and _level_of_file() for provenance of the
    escalation decision (full label propagation is roadmap work; see
    KNOWN_ISSUES.md §1 and ROADMAP.md).
  • CLI: ahd run PATH and ahd eval.
  • Bundled adversarial scenarios re-modeling the public Signetry/eval IPI
    corpus: ipi.readme_deploy_and_exfil and ipi.claude_md_scope_expansion.
    The agent's obey() step WRITES the malicious artifacts to disk so the eval
    exercises the real defense.
  • Three independent guard rails proving the eval is non-vacuous:
    • Teeth assert in test_admission.py — fails with SCENARIO INCOMPLETE
      if obey() does not materialize the attack artifact.
    • Regression guard in test_eval_catches_regression.py — a defense that
      admits the forbidden artifact reports trust_boundary_clean=False.
    • Monitor signal pins in test_monitor_signal.py — the cross-iteration
      monitor's signal is asserted at concrete values; the planted-input
      invariant is verified by re-introducing the original bug and seeing the
      test fail with REGRESSION:.

Security

  • AGPL-3.0-or-later license, verbatim text from
    https://www.gnu.org/licenses/agpl-3.0.txt (the legal body from the FSF
    Copyright (C) 2007 Free Software Foundation line onwards is byte-identical,
    658 lines, diff empty). The author attribution is added as an FSF-style
    header notice (the pattern recommended in the license's own "How to Apply
    These Terms" section).
  • SPDX headers in every .py source file under agent_harness_defense/ and
    tests/.
  • gitleaks scan in CI to catch accidental secret leaks.

CI / infra

  • pip install -e .[dev] verified on a clean runner (/tmp/ahd-clean):
    • ruff check . → All checks passed.
    • ruff format --check . → 12 files already formatted.
    • pytest → 6 passed.
    • bandit -r agent_harness_defense -ll → Low: 0, Med: 0, High: 15
      (legitimate subprocess usage in admission.py for git rev-parse and
      git diff --stat; non-fatal with -ll).
  • gitleaks/gitleaks-action@v2 with fetch-depth: 0 (shallow clones break
    the action's diff scan).
  • [tool.hatch.metadata] allow-direct-references = true so the eval extra
    (which uses a direct git URL for signetry-eval) installs without
    ERROR: Direct references are not allowed.

Known limitations (see KNOWN_ISSUES.md for the full list)

  • Taint propagation is NOT implemented. The taint field in
    AdmissionReport records each file's label for provenance but does not
    propagate labels across data/control flows. The direction is SPA
    (arXiv:2608.27234); see ROADMAP.md.
  • llm-guard is optional ([detect] extra) and not yet wired into
    run_admission. It is offered as a second signal for callers who want it.
  • Cross-iteration state retention is the caller's responsibility. The CLI
    does not drive a real agent loop; the harness that integrates
    run_admission must reuse the same LoopStateMonitor instance.
  • Eval is narrow. Only the two IPI scenarios are re-modeled; the
    skill_poison and minja Signetry scenarios and the 6 real coding-agent
    harnesses from arXiv:2608.27299 are roadmap work.

Audits

Two independent external audits (Claude) on 2026-08-28:

  • Round 1 flagged that the original eval was vacuous (the obey() step
    was missing) and that the README claimed taint propagation that did not
    exist. Both fixed in the initial round of commits: port the real Signetry
    obey(), add the teeth assert, document the v0.1 scope honestly in
    KNOWN_ISSUES.md.
  • Round 2 flagged (a) a regression introduced by the proposed_files
    refactor — the cross-iteration monitor was scanning only the agent's
    output, so the signal came from a coincidental substring in the agent's
    output rather than from the planted injection; (b) pip install -e .[dev]
    failed on a clean runner for three independent reasons (hatchling direct
    references, gitleaks not being a pip package, bandit exiting 1 on
    legitimate Low-severity findings); (c) the LICENSE lost its author
    attribution when the verbatim swap removed it. All three fixed in commits
    37e4b44, 693de48, and d05723c.