Skip to content

LLM Usage

Jump to bottom
Nick edited this page Nov 18, 2025 · 4 revisions

LLM Usage in PATAS

Purpose: This document explains how Large Language Models (LLMs) are used in PATAS, what data they process, and how their output is validated and constrained.

LLM Role: Pattern Discovery, Not Classification (Optional / External)

Critical Point: PATAS Core does NOT require or ship with LLM. LLM integration (if used) is optional and external, running inside Telegram's infrastructure.

If LLM is integrated: LLMs are NOT used for online per-message classification. They are used offline for pattern discovery and hypothesis generation.

What LLMs Do

LLMs play a crucial role in significantly improving pattern analysis accuracy through deep semantic understanding:

  1. Semantic Pattern Discovery: LLMs analyze aggregated spam signals to identify semantic patterns that go beyond simple keyword matching. They understand context, meaning, and variations in spam messages.

  2. Deep Semantic Analysis: Unlike rule-based methods that only match exact keywords or URLs, LLMs understand:

    • Semantic variations (e.g., "earn money" → "make cash" → "get income")
    • Contextual patterns (e.g., recognizing commercial spam intent across different phrasings)
    • Paraphrased content (e.g., LLM-generated spam variations)
    • Cross-language patterns (understanding spam intent across languages)

  3. Intelligent SQL Rule Generation: LLMs don't just generate SQL queries—they:

    • Understand the semantic meaning of discovered patterns
    • Propose SQL rules that catch semantic variations, not just exact matches
    • Generate rules that are both precise (low false positives) and comprehensive (catch variations)
    • Consider context and intent when creating rules

  4. Pattern Quality Assessment: LLMs evaluate the quality and safety of discovered patterns before they become rules, identifying potential false positive risks.

  5. Offline Only: All LLM processing happens during pattern mining, not during message evaluation.

What LLMs Do NOT Do

  1. ❌ NOT used for real-time message classification
  2. ❌ NOT making ban/unban decisions
  3. ❌ NOT processing individual messages online
  4. ❌ NOT making final enforcement decisions

LLM Usage Flow

1. Pattern Mining Pipeline
   ↓
2. Aggregate spam signals (URLs, keywords, examples)
   ↓
3. Send aggregated summary to LLM (NOT individual messages)
   ↓
4. LLM performs deep semantic analysis:
   - Understands semantic patterns and variations
   - Identifies contextual spam intent
   - Generates pattern hypotheses with semantic understanding
   - Proposes SQL rules that catch semantic variations
   ↓
5. LLM Rule Validation (Optional):
   - Validates rule adequacy before production
   - Checks for potential false ban risks
   - Assesses semantic correctness
   ↓
6. Offline evaluation: test SQL rules on historical data
   ↓
7. Quality filtering: tier classification (SAFE_AUTO / REVIEW_ONLY / FEATURE_ONLY)
   ↓
8. Safety profiles: Conservative / Balanced / Aggressive
   ↓
9. Only then: rules can be used in production (with safety constraints)

Data Privacy & LLM

Sensitive Information Protection

Before sending data to LLM, PATAS encrypts sensitive information:

  • PII Redaction: Personal Identifiable Information (emails, phone numbers, credit cards) is automatically redacted or masked before sending to LLM
  • Data Encryption: Sensitive data is encrypted using configurable encryption methods
  • Minimal Data Exposure: Only aggregated signals and anonymized examples are sent to LLM
  • Configurable Privacy Levels: Different privacy modes control what data is sent and how it's protected

What Data LLM Sees

During Pattern Mining:

  • Aggregated signals only: Top URLs, top keywords, sample spam messages (limited to 10 examples)
  • Anonymized examples: User identifiers removed, PII redacted
  • NOT individual user messages in real-time
  • NOT full message history
  • NOT user identifiers (sender IDs, user names, etc.)

Example of aggregated data sent to LLM (after encryption/redaction):

{
  "total_spam": 1000,
  "total_ham": 200,
  "url_patterns": ["https://spam-site.com", "https://scam-link.net"],
  "keyword_patterns": ["earn money", "get rich", "work from home"],
  "spam_examples": [
    {"text": "Earn money fast! Click here...", "pii_redacted": true},
    {"text": "Get rich quick! Join now...", "pii_redacted": true}
  ]
}

Privacy Guarantees

  1. On-Prem Deployment: PATAS runs on Telegram's infrastructure by default
  2. Configurable Provider: LLM provider is configured by operator (internal endpoint or external)
  3. No Hardcoded External Calls: PATAS does not hardcode sending data to external services
  4. Strict Privacy Mode: Can disable external LLM providers entirely
  5. Sensitive Data Encryption: PII and sensitive information encrypted before LLM processing

See Privacy-and-Data-Protection for details.

LLM Configuration

Provider Configuration

LLM provider is configurable by the operator:

# In app/config.py
llm_provider: str = "openai"  # or "none", "disabled", "local"
llm_model: str = "gpt-4o-mini"

Options:

  • "openai": Use OpenAI API (requires API key)
  • "local": Use local/on-prem LLM endpoint (configure endpoint URL)
  • "none" or "disabled": Disable LLM entirely (pattern mining uses rule-based methods only)

Privacy Mode

In STRICT privacy mode:

  • External LLM providers are disabled by default
  • Only internal/on-prem LLM endpoints are allowed
  • Message texts are not stored in logs
  • Sensitive information is encrypted before any processing

See Privacy-and-Data-Protection for privacy mode details.

LLM Prompt Safety

Prompt Constraints

The LLM prompt explicitly instructs:

  1. Narrow Patterns: Focus on explicit commercial/abusive spam, not broad categories
  2. No Sensitive Attributes: Never base rules on politics, religion, race, ethnicity, gender, sexual orientation
  3. Safe SQL Only: Generate only SELECT queries with whitelisted columns
  4. No Match-Everything: Rules must not match >80% of messages
  5. Interpretable: Patterns must be human-readable and explainable

Prompt Example

See app/v2_llm_engine.py for the full prompt. Key sections:

CRITICAL SAFETY REQUIREMENTS:
1. Generate NARROW, INTERPRETABLE patterns
2. Avoid over-broad rules
3. NEVER base rules on sensitive attributes
4. SQL rules MUST be safe SELECT queries only

LLM Output Validation

All LLM-generated rules go through multiple validation layers:

1. SQL Safety Validation

  • ✅ Only SELECT queries
  • ✅ Only whitelisted tables/columns
  • ✅ No UPDATE/DELETE/INSERT/ALTER
  • ✅ No subqueries, no semicolons
  • ✅ No "match everything" patterns

See app/v2_sql_safety.py for implementation.

2. Coverage Check

  • ✅ Rules matching >80% of messages are rejected
  • ✅ Prevents over-broad rules from LLM

3. LLM Rule Validation (Before Production)

Before rules are deployed to production, LLM can validate them for adequacy and false ban risks:

  • ✅ Adequacy Check: LLM validates that the rule correctly identifies the intended spam pattern
  • ✅ False Ban Risk Assessment: LLM analyzes whether the rule could lead to false bans of legitimate users
  • ✅ Semantic Correctness: LLM verifies that the rule's semantic understanding matches the pattern intent
  • ✅ Edge Case Analysis: LLM identifies potential edge cases where the rule might fail
  • ✅ Token-efficient: ~200-400 tokens per validation
  • ✅ Graceful degradation: Skips if LLM unavailable (rules still go through other validation layers)

How it works: After deterministic validation passes, an optional LLM validation step analyzes the SQL rule for:

  • Adequacy: Does the rule correctly identify the spam pattern?
  • False Ban Risks: Could this rule accidentally ban legitimate users?
  • Semantic Correctness: Does the rule's logic match the semantic intent?

Rules flagged as "high risk" for false bans are rejected; "medium risk" rules are logged but allowed to proceed with manual review.

This validation step provides an additional safety layer before rules enter production, leveraging LLM's semantic understanding to catch potential issues that deterministic checks might miss.

See SQL_LLM_VALIDATION.md for detailed documentation.

4. Offline Evaluation

  • ✅ Rules tested on historical data
  • ✅ Precision, recall, ham hit rate calculated
  • ✅ Rules classified into tiers (SAFE_AUTO / REVIEW_ONLY / FEATURE_ONLY)

5. Safety Profiles

  • ✅ Only SAFE_AUTO rules in Conservative profile
  • ✅ Balanced/Aggressive profiles require manual review
  • ✅ Safety guardrails prevent unsafe auto-promotion

LLM vs Machine Learning

Important: PATAS does NOT use Machine Learning for classification.

Why not ML?:

  • ML models are black boxes, hard to interpret
  • ML requires large training datasets and continuous retraining
  • ML models can degrade over time without clear explanation
  • ML adds complexity without clear benefit for pattern-based spam detection

What PATAS uses instead:

  • Rule-based pattern matching: Explicit SQL rules that are interpretable
  • LLM for pattern discovery: LLM helps discover patterns with deep semantic understanding, but rules are explicit and auditable
  • Statistical evaluation: Rules are evaluated on historical data with clear metrics

LLM Limitations & Mitigations

Limitations

  1. LLM can generate unsafe rules: Mitigated by SQL safety validation
  2. LLM can generate over-broad patterns: Mitigated by coverage checks and quality filtering
  3. LLM can hallucinate patterns: Mitigated by offline evaluation on real data

Mitigations

  1. Multi-layer validation: SQL safety → Coverage check → LLM rule validation → Offline evaluation → Tier classification
  2. Human review: REVIEW_ONLY and FEATURE_ONLY tiers require manual review
  3. Safety profiles: Conservative profile only includes SAFE_AUTO rules
  4. Rollback capability: Rules can be deprecated if quality degrades

Summary

LLM Role in PATAS:

  • ✅ Offline pattern discovery (not online classification)
  • ✅ Deep semantic analysis (understands context, meaning, variations)
  • ✅ Significantly improves accuracy (catches semantic variations that rule-based methods miss)
  • ✅ Hypothesis generation (not final decisions)
  • ✅ Aggregated data only (not individual messages, with PII encryption/redaction)
  • ✅ Configurable provider (internal or external, operator's choice)
  • ✅ Multi-layer validation (safety checks, LLM rule validation, evaluation, tiering)
  • ✅ Rule validation before production (adequacy and false ban risk assessment)

LLM Does NOT:

  • ❌ Classify messages in real-time
  • ❌ Make ban/unban decisions
  • ❌ Process individual user messages online
  • ❌ Send data to external services by default

Privacy & Security:

  • ✅ On-prem deployment by default
  • ✅ LLM provider configurable by operator
  • ✅ Strict privacy mode available
  • ✅ No hardcoded external calls
  • ✅ Sensitive information encrypted before LLM processing
  • ✅ PII redaction and anonymization

Last Updated: 2025-11-18

Clone this wiki locally