Skip to content

v3.6.0: Prompt Injection Bypass Hardening

Choose a tag to compare

View all tags
@sinewaveai sinewaveai released this 16 Feb 02:53
v3.6.0
b444296

🛡️ Major Security Enhancements

This release dramatically improves prompt injection detection by closing 5 critical bypass vectors identified in product evaluation. Detection rates improved from 5/10 to comprehensive coverage across all attack categories.

🎯 Bypass Fixes

# Bypass Vector Fix
1 Code block delimiter confusion Extract content from ~~~, <code>, <!-->, <pre>, CDATA blocks
2 Pattern fragmentation Collapse string concatenation ("ign" + "ore") and C-style comment separators
3 Multi-encoding bypass Cascade decoder supporting base64 (nested), hex, URL encoding, ROT13
4 Multi-turn shallow detection Scan all previous messages, detect Crescendo frame-setting, cross-turn boundary patterns
5 Composite threshold gaming Co-occurrence matrix, orthogonal dimension scoring, low-signal accumulation

✨ New Features

Unicode Normalization Pipeline

  • NFKC normalization
  • Cyrillic/Greek homoglyph canonicalization (40+ mappings)
  • Zero-width character stripping
  • Zalgo diacritics removal
  • Invisible Unicode detection as obfuscation indicator

Enhanced Encoding Detection

  • Multi-encoding cascade replaces base64-only decoder
  • Supports nested base64, hex, URL encoding
  • Indicator-gated ROT13 (avoids false positives)
  • Lowered thresholds: 40→20 chars, 70%→55% printability

Composite Scoring Improvements

  • Category co-occurrence matrix: 12 suspicious pairs, +40% cap
  • Orthogonal dimension scoring: 7 attack dimensions, +40 flat bonus
  • Low-signal accumulation: Multiple LOW-confidence findings now trigger

Garak Integration (Optional)

  • NVIDIA Garak LLM vulnerability scanner support via deep_scan parameter
  • Encoding probes (base64, base16, rot13)
  • Latent injection pattern detection
  • Graceful degradation when Garak not installed

PromptFoo Red-Team Suite

  • 13 automated test cases with custom MCP provider
  • Continuous bypass detection validation
  • Run with: npm run test:redteam

📝 New Rules

3 new YAML security rules:

  • generic.prompt.security.whitespace-fragmentation - Letter-spaced keyword evasion
  • generic.prompt.security.crescendo-setup - Persistent behavioral rule injection
  • generic.prompt.security.leetspeak-injection - Character substitution obfuscation

🧪 Testing

  • 28 new test cases covering all bypass vectors
  • False positive regression suite for benign prompts
  • All 102 tests passing ✅
  • Average test time: ~2.7 seconds per test

📊 Impact

  • Detection improvement: 5/10 → comprehensive coverage
  • Code changes: +1,099 additions / -85 deletions
  • Files modified: 7 (3 new)
  • Breaking changes: None (strictly additive)

🔧 What Changed

Modified:

  • mcp-server/src/tools/scan-prompt.js - 612 lines of enhanced logic (was 486 lines)
  • mcp-server/rules/prompt-injection.security.yaml - 68 new lines
  • mcp-server/package.json - Added test:redteam script

New Files:

  • mcp-server/src/tools/garak-bridge.js - 209 lines
  • mcp-server/tests/promptfoo/promptfooconfig.yaml - 101 lines
  • mcp-server/tests/promptfoo/scan-provider.js - 54 lines
  • mcp-server/tests/scan-prompt.test.js - 137 new test cases

📦 Installation

npm install -g agent-security-scanner-mcp@3.6.0
# Or update existing installation
npm update -g agent-security-scanner-mcp

🔗 Related

  • Addresses findings from internal security evaluation
  • Inspired by Garak Buff normalization techniques
  • Implements PromptFoo red-team methodology
  • Builds on Crescendo multi-turn attack research

Full Changelog: v3.5.2...v3.6.0

Assets 2
Loading