Skip to content

v0.3.2 — Output Scanning, Security Gaps Closed, Competitor Benchmarks

Choose a tag to compare

@mthamil107 mthamil107 released this 21 Mar 06:35
· 127 commits to main since this release
1c5c031

What's New

Output Scanning (5 scanners)

prompt-shield now scans both inputs AND outputs. 5 new output scanners:

Scanner Detects
Toxicity Hate speech, violence, self-harm, sexual content, dangerous instructions
Code Injection SQL injection, shell commands, XSS, path traversal, SSRF, deserialization
Prompt Leakage System prompt exposure, API key leaks, instruction leaks
Output PII PII in LLM responses (emails, SSNs, credit cards, etc.)
Relevance Jailbreak persona adoption, DAN mode, unrestricted claims
prompt-shield output scan "Here is how to build a bomb..."
curl -X POST localhost:8000/output/scan -d '{"text": "..."}'

Security Gap Fixes (5 gaps closed)

  • d024: Multilingual injection — 10 languages, 65+ patterns (0% → 100%)
  • d025: Multi-encoding decoder — hex, URL, leetspeak, Caesar, Morse, Pig Latin, reversed (20% → 80%)
  • d017/d018: Dual intention + HILL educational reframing (20%/40% → 80%)
  • d006/d014: Multi-turn escalation + tool-disguised JSON/MCP (60% → 100%)

Competitor Benchmark

Scanner Detection FP Rate F1 Speed
prompt-shield 92.3% 0.0% 96.0% 500/sec
ProtectAI DeBERTa v2 48.7% 0.0% 65.5% 11/sec
Deepset DeBERTa v3 87.2% 6.7% 91.9% 10/sec

Stats

  • 25 input detectors + 5 output scanners = 30 total scanning components
  • 706 tests passing, 0 failures
  • F1: 96.0% | 0% false positives | 500 scans/sec