You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added
bench: block-rate comparison vs LlamaFirewall + Invariant (benchmarks/blockrate/). One shared tool-call corpus (210 calls: ToolPrivBench-derived over-privileged tool selection + a public injected-tool-call set + benign controls). agent-airlock's deny-by-default presets are re-run deterministically (block-rate + p50/p95 latency); Meta LlamaFirewall and Invariant Guardrails are model-in-the-loop systems this in-process harness does not execute, so their coverage is cited, not re-run — no fabricated competitor number. Honest headline: 100% block / 0% false-positive on a self-curated corpus at p50 ~2µs/decision, contrasted categorically (not as a head-to-head score) with model-in-the-loop detection. Deterministic block-rate folds into the drift-gated BENCHMARK.md; wall-clock latency lives in benchmarks/blockrate/RESULTS.md. Run: python -m benchmarks.blockrate.
Fixed
test-badge version drift. The README badge sat at v0.8.5 while the package shipped v0.8.39; update_test_badge.py already read the live __version__ but had never been re-run. Badge refreshed (3,280 tests · 85.32% · v0.8.39) and a new CI guard (tests/test_badge_version.py) fails if the badge version ever diverges from the package version again. (#107)