LLMGuard 🛡️

Enterprise LLM Vulnerability Scanner (Tier 1 – Pattern Detection)
OWASP LLM Top 10 compliance testing with real-time payload streaming and risk-scored findings — the foundation for LLM security validation

---

🎯 Purpose

LLMGuard is a production-grade security validation tool for AI teams building with Claude, GPT-4, and other large language models. It systematically tests LLM deployments against the OWASP LLM Top 10 vulnerability categories, providing:

✅ Comprehensive Coverage — All 10 OWASP LLM categories with 30+ core payloads
✅ Real-Time Feedback — Server-Sent Events streaming for immediate visibility
✅ Risk Prioritization — Severity-weighted scoring for remediation planning
✅ Production-Ready — Docker, Cloud Run, and on-premises deployment options
✅ Multi-Model Support — Test Claude and GPT-4 simultaneously

Perfect for:

• Pre-deployment LLM security validation
• Red team exercises on AI systems
• Compliance testing (SOC 2, ISO 27001 AI controls)
• Continuous security monitoring of LLM APIs
• AI safety team workflows

---

Architecture

User → Web UI (Vanilla JS + Bootstrap)
    │
    ▼
Flask Backend (app.py)
    ├── Rate Limiter (IP-based, time-window)
    └── SSE Generator (_scan_generator)
            │
            ▼
        LLM Dispatcher (_dispatch)
            ├── _call_anthropic()  →  Claude
            └── _call_openai()     →  GPT-4
            │
            ▼
        Payload Library (scanner.py)
            ├── OWASP Category Filtering
            └── Payload Metadata (id, name, severity, prompt)
            │
            ▼
        Response Analyzer (analyze_response)
            └── Status → VULNERABLE / PARTIAL / RESISTANT / REVIEW
            │
            ▼
    Real-Time SSE Results → Dashboard

---

OWASP LLM Top 10 Coverage

#	Category	Detection Method
LLM01	Prompt Injection	Model follows injected instructions
LLM02	Insecure Output Handling	HTML/script injection in responses
LLM03	Training Data Poisoning	Consistency checks across inputs
LLM04	Model Denial of Service	Response time and token consumption
LLM05	Supply Chain Vulnerabilities	External dependency acceptance
LLM06	Sensitive Information Disclosure	System prompt leakage detection
LLM07	Insecure Plugin Design	Unauthorized tool/API invocation
LLM08	Excessive Agency	Commitment to dangerous actions
LLM09	Overreliance	Misleading context injection
LLM10	Model Theft	Replication and extraction probes

---

✨ Key Features

Vulnerability Detection

• Full OWASP LLM Top 10 v2 coverage (10 categories, 30+ payloads)
• Pattern-based detection (exact match + regex patterns)
• Severity-weighted risk scoring (Critical → Low)
• Multi-turn probe capabilities for complex attack scenarios

Real-Time Streaming

• Server-Sent Events (SSE) for live results as payloads fire
• Low-latency feedback in dashboard UI
• Status tracking per payload (VULNERABLE / PARTIAL / RESISTANT / REVIEW)

Multi-Model Testing

• Claude (Anthropic) + GPT-4 (OpenAI) simultaneous testing
• Model-specific payload tuning
• Side-by-side response comparison

Enterprise Deployment

• Rate limiting & API quota management
• Docker multi-stage builds (minimal, secure images)
• GCP Cloud Run ready (serverless, scales to zero)
• REST API for CI/CD/automation integration
• Audit logging for compliance

---

🏆 Tier 1 vs Tier 2 (LLMGuardT2)

Feature	LLMGuard (Tier 1)	LLMGuardT2 (Tier 2)
Detection Method	Pattern matching + regex	Pattern + Semantic embeddings
Paraphrase Evasion	Bypassed by simple rewording	Caught by similarity scoring
Setup Complexity	Simple (no ML models)	Requires sentence-transformers
Latency	<100ms per payload	~200ms (embedding compute)
Best For	Fast baseline scans	Rigorous security validation

LLMGuard is your starting point. LLMGuardT2 is for teams who need evasion-resistant detection.

---

Tech Stack

Layer	Technology
Backend	Python 3.11, Flask 3.0.3
LLM APIs	Anthropic (Claude), OpenAI (GPT-4)
Streaming	Server-Sent Events (SSE)
Production WSGI	Gunicorn 22.0
Deployment	Docker, GCP Cloud Run
Config	python-dotenv

---

Getting Started

git clone https://github.com/BadAsh99/llmguard.git
cd llmguard
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env   # add API keys
python app.py
# Open http://localhost:5000

Docker

docker build -t llmguard:latest .
docker run -p 8080:8080 \
  -e ANTHROPIC_API_KEY=sk-ant-... \
  -e OPENAI_API_KEY=sk-... \
  llmguard:latest

---

Environment Variables

ANTHROPIC_API_KEY=
OPENAI_API_KEY=

---

API Endpoints

Method	Endpoint	Description
GET	/	Web dashboard
POST	/api/scan	Run scan — returns SSE stream
GET	/api/categories	List OWASP categories
GET	/api/payloads	Full payload library
GET	/api/health	Liveness probe

---

---

📊 Project Impact

• Target Users: Security teams, AI/ML engineers, DevSecOps teams
• Use Cases: Pre-deployment validation, red team exercises, compliance testing
• Deployments: SaaS, Cloud Run, on-premises Docker
• Maturity: Production-ready, battle-tested on real LLM deployments

---

🔗 Related Projects

• LLMGuardT2 — Tier 2 enhancement with semantic similarity detection (catches paraphrased + obfuscated attacks)
• ai-runtime-security-framework — Full multi-app LLM runtime security architecture with red teaming
• cloudguard — Cloud infrastructure red-teaming (AWS/Azure/GCP)

---

🔧 Skills Demonstrated

• LLM Security — OWASP compliance, prompt injection detection, jailbreak prevention
• API Design — Real-time SSE streaming, rate limiting, quota management
• Multi-Model Testing — Anthropic + OpenAI simultaneous validation
• DevOps — Docker, GCP Cloud Run, containerization
• Red Teaming — Attack payload design, vulnerability classification
• Full-Stack Development — Backend (Flask/Python) + Frontend (Vanilla JS + Bootstrap)

---

👤 Author

Ash Clements — Sr. Principal Security Consultant at Palo Alto Networks
Specialties: AI/LLM Security | Cloud Security Architecture | Red Teaming | Security Automation
GitHub: BadAsh99 | Portfolio: Security & AI Tools