Skip to content

Threat Coverage

Fabrizio Salmi edited this page Sep 6, 2026 · 1 revision

Threat coverage

The layers, and what each one is for

Layer Catches Fails on
ASGI firewall Known injection signatures, including obfuscated ones: URL, Unicode, Base64, hex and ROT13, decoded iteratively through chains Anything not in the signature set. It is a signature engine, with a signature engine's blind spot
SecurityShield Threat scoring over regex patterns, multi-turn trajectory detection, and a cross-session ledger Single benign-looking turns that only add up to an attack across sessions it cannot correlate
Semantic analyzer Trigram similarity against a pattern corpus in 20+ languages, with leetspeak normalisation and Cyrillic/Greek confusable mapping Novel phrasings far from the corpus. Bounded by a 5-second timeout, so an expensive input degrades rather than blocks the loop
PII detection Presidio NLP entity types, or a regex fallback covering email, phone, SSN, credit card, IBAN, IP and API keys The fallback only knows those formats. Presidio is not installed by default
Response sanitisation Entropy anomalies, steganography (bidi overrides, zero-width characters, homoglyphs), and prompt leaks in what comes back Content that is simply wrong or harmful without being anomalous
Audit ledger Nothing, by design. It records, with a SHA-256 hash chain and tamper detection Being a detection layer. It is evidence, after the fact

The useful mental model: signatures catch the known, scoring catches the shaped, semantics catch the rephrased, and none of them catch the genuinely novel. Layers help because attackers rarely evade all four at once, not because any one of them is complete.

The measured numbers

An adversarial corpus runs as a regression test on every build, and the results are published rather than claimed:

Category Coverage
LLM01, prompt injection 100% of 27 corpus variants, including base64, hex, zero-width, leetspeak, role-play, suffix injection, seven languages, jailbreak framing and tool-call injection
LLM02, sensitive information 100%: email, SSN, Visa, Amex, IBAN, phone, API keys
LLM07, system prompt leakage 100%: direct, indirect, continuation, translation, meta-instruction, persona rebase
Benign false positives 6% of 18 controls

Read those honestly, and the project itself does:

  • 100% is against this corpus, not against prompt injection as a problem. A regression suite proves the known attacks stay caught, which is exactly what it is for and is not a claim about the unknown ones
  • The 6% false-positive rate is a real cost. The tripping control is meta-discussion of attacks, so "explain how prompt injection works" is refused. If your users are security people, that will happen to them
  • The number excludes the AI escalation path deliberately. Deterministic checks only, because the gray-zone escalation depends on an upstream model being available, and a number that depends on someone else's uptime is not a regression number

Full per-entry results, the known gaps and reproduction steps are in docs/OWASP_LLM_COVERAGE.md. Regenerate with pytest tests/test_owasp_corpus.py.

What is out of scope

LLM03, 04, 06, 08, 09 and 10 are documented as not applicable to a proxy: they live at build time, training time, in the caller, or in the model. That is an accurate boundary rather than a gap, and it is worth quoting when someone asks whether this covers the OWASP LLM Top 10. It covers three of them, thoroughly, and says so.

Two things that are not on by default

  • Presidio is not installed unless you install it. Without it, PII detection falls back to regex, which is narrower. pip install presidio-analyzer presidio-anonymizer
  • tiktoken is not installed either, so token counting is approximate until it is. That matters if you use budgets, since a budget enforced on an estimate is enforced approximately

Clone this wiki locally