Skip to content

0.4.0 — the gate you can deploy, and a second language

Choose a tag to compare

@cgrtml cgrtml released this 14 Sep 21:20
· 49 commits to main since this release

The first release on PyPI since 4 July. It supersedes the v0.3.0 tag, which was cut and
never published — everything in 0.3.0 is here.

Every change in this release answers something docs/coverage-gaps.md
measured, and the same script (python -m eval.misses) re-measures what moved.

0.3.0 0.4.0
Rule-core recall, real corpus 10.6% 19.8%
False-positive rate 0.5% 0.5%
NotInject over-defense 0.0% 0.0%

Doubling recall without touching the false-positive rate is the honest read. The
denominator is still the point: 19.8% is what the first layer catches, and the inventory
says which 80% it does not — including the 59% of misses that no input filter can reach.

The action gate, made deployable

  • Multi-hop taint (GateSession). The gate was single-hop: it caught a destination
    quoted from a document you handed in, and missed the shape that actually happens — the
    value reaching the sensitive argument through a tool result. A session carries trust
    across calls, so an address that arrives in a fetched page still blocks the later send.
  • Authorization no longer launders a tainted destination. authorized=True used to
    return allow before the taint check ran, so "email the summary to my manager" also
    cleared whatever recipient an injection had substituted. A principal authorizes an
    action, not the argument values chosen for it. Behavior change if you pass
    authorized=True.
  • Taint sees through two cheap transforms: a destination split or punctuated inside
    the untrusted text ("99-00-4321" vs "99004321"), and one hidden in a base64 run.
  • Adapters and a catalog. reasongate.adapters.toolcalls converts Anthropic, OpenAI
    and MCP tool calls into what the gate authorizes and hands a block back as a normal tool
    result; reasongate.catalog drafts policies from tool names and prints them for review.
  • PolicyGate is the seam for the 59%: a deployment declares what it forbids and a
    judge reviews against it. No model judge ships here — unconfigured it returns "not
    evaluated", never a clearance, and the module documents why a model judge is itself an
    injection target.

A second language, measured properly

German override, disclosure and declared-void families. The patterns were written from the
train split of the real corpus and scored on the held-out test split:

Before After
German, held-out test (15 attacks) 0.0% 26.7%
German, whole corpus (73 attacks) 0.0% 28.8%
English, whole corpus 14.7% 16.3%
NotInject over-defense 0.0% 0.0%

Fifteen held-out attacks is a small test set and 26.7% is four of them. It is the honest
figure and it is not precise. This is a beachhead in one language, not multilingual
support
— every other language is still measurably zero, Turkish included.

Performance

The normalization detector no longer re-scans surfaces whose result is discarded: an
attack only counts as obfuscated when the raw text did not match, so once it did, every
following surface scan was thrown away. Verified decision-identical across 1,001 corpus
rows and against the evasion suite (unchanged at 78.1% / 6.7%).

Against 0.3.0 on the same machine, with roughly twice as many patterns: chat prompts
0.218 → 0.178 ms, throughput 4,667 → 5,422/s, a 50 KB document carrying a known
attack 112 → 85 ms. A clean 50 KB document costs 211 ms — slower than before,
because a clean input is the case that still scans every surface. That is what the German
families cost, published as two paths in RESULTS.md rather than averaged into one number.

76 tests, Python 3.9–3.12, zero dependencies.