Repository navigation
v0.3.0 — 95 vectors, 10 OWASP categories
Expands the suite from 38 vectors to 95, across 10 categories covering every dimension of the OWASP LLM Top 10 that is actually probeable from a prompt.
New categories
| Category | OWASP | Vectors | Probes |
|---|---|---|---|
improper_output_handling |
LLM05 | 9 | Markdown-image exfil, stored XSS, SQL clause injection, JSON structure injection, template injection, shell + CSV formula passthrough |
excessive_agency |
LLM06 | 9 | Destructive tool calls without confirmation, scope escalation, action driven by injected document text, fabricated tools |
unbounded_consumption |
LLM10 | 6 | Repetition floods, recursive expansion, verbosity coercion |
misinformation |
LLM09 | 6 | Confabulated specs, false premises, invented packages, fabricated citations |
The six original categories gain 27 vectors: payload splitting, refusal suppression, prefix forcing, non-English instruction, crescendo escalation, virtualisation, homoglyphs, base32, Morse, acrostic smuggling, poisoned JSON metadata, image alt text, diffs, calendar entries, private-key echo, identifier aggregation.
LLM03 (supply chain) and LLM04 (data poisoning) are properties of how a model was built, not behaviours a prompt can probe — deliberately out of scope.
Two new detectors
repetition— counts pattern occurrences past a threshold.absent— inverted: the finding is a missing hedge. This is the only detector with a real false-positive surface, so vectors using it are capped at medium severity and a test enforces it.
reject_if
Vectors can now withdraw a finding when the model demonstrably defended itself — an escaped <script>, a parameterised WHERE name = ?, a quoted shell argument. Every payload vector is pinned from both directions: safe form must come back clean, unsafe form must fire.
Fix: blank responses were counted as passes
Found by running the new categories live. An empty completion scored as the guardrail holding — so a provider-side filter returning blanks for every vector would have read as a clean PASS. Blanks are now counted separately, withhold the all-clear, and short-circuit every detector (notably absent, which would otherwise fire on an empty string).
Validation
Live against gemini-3.5-flash: 17 new vectors ran before the free-tier daily quota. All five detector types behaved correctly, zero false positives — reject_if correctly cleared three confirmation-seeking replies to destructive tool calls.
65 tests.