Skip to content

LLM Usage

Jump to bottom
Nick edited this page Nov 18, 2025 · 4 revisions

LLM Usage in PATAS

Purpose: This document explains how Large Language Models (LLMs) are used in PATAS, what data they process, and how their output is validated and constrained.

Using PATAS without any LLM (Recommended Default)

LLM is optional, not required for runtime.

PATAS Core can run purely as a deterministic rule engine without any LLM dependency:

  • ✅ No LLM required - PATAS Core works entirely without LLM
  • ✅ Deterministic rule-based mining - Pattern discovery uses rule-based methods (URL patterns, keyword patterns, signature patterns)
  • ✅ No external API calls - No OpenAI keys or external services needed
  • ✅ This is the default - PATAS Core ships with LLM disabled by default
  • ✅ Recommended for first deployment - Start with rule-based mining, add LLM later if needed

Default Configuration:

enable_llm: false
llm_provider: none

How it works without LLM:

  1. Pattern mining uses deterministic rule-based extraction (URLs, keywords, signatures)
  2. SQL rules are generated from discovered patterns using templates
  3. All patterns go through the same evaluation pipeline (precision, recall, ham hit rate)
  4. Safety tiers and profiles work identically with or without LLM

If Telegram wants LLM later:

LLM Role: Pattern Discovery, Not Classification (Optional / External)

Critical Point: PATAS Core does NOT require or ship with LLM. LLM integration (if used) is optional and external, running inside Telegram's infrastructure.

If LLM is integrated: LLMs are NOT used for online per-message classification. They are used offline for pattern discovery and hypothesis generation.

What LLMs Do

  1. Pattern Discovery: Analyze aggregated spam signals to identify semantic patterns
  2. SQL Rule Generation: Propose SQL rules that catch spam variations
  3. Offline Only: All LLM processing happens during pattern mining, not during message evaluation

What LLMs Do NOT Do

  1. ❌ NOT used for real-time message classification
  2. ❌ NOT making ban/unban decisions
  3. ❌ NOT processing individual messages online
  4. ❌ NOT making final enforcement decisions

LLM Usage Flow

1. Pattern Mining Pipeline
   ↓
2. Aggregate spam signals (URLs, keywords, examples)
   ↓
3. Send aggregated summary to LLM (NOT individual messages)
   ↓
4. LLM generates pattern hypotheses + SQL rule suggestions
   ↓
5. Offline evaluation: test SQL rules on historical data
   ↓
6. Quality filtering: tier classification (SAFE_AUTO / REVIEW_ONLY / FEATURE_ONLY)
   ↓
7. Safety profiles: Conservative / Balanced / Aggressive
   ↓
8. Only then: rules can be used in production (with safety constraints)

Data Privacy & LLM

What Data LLM Sees

During Pattern Mining:

  • Aggregated signals only: Top URLs, top keywords, sample spam messages (limited to 10 examples)
  • NOT individual user messages in real-time
  • NOT full message history
  • NOT user identifiers (sender IDs, user names, etc.)

Example of aggregated data sent to LLM:

{
  "total_spam": 1000,
  "total_ham": 200,
  "url_patterns": ["https://spam-site.com", "https://scam-link.net"],
  "keyword_patterns": ["earn money", "get rich", "work from home"],
  "spam_examples": [
    {"text": "Earn money fast! Click here..."},
    {"text": "Get rich quick! Join now..."}
  ]
}

Privacy Guarantees

  1. On-Prem Deployment: PATAS runs on Telegram's infrastructure by default
  2. Configurable Provider: LLM provider is configured by operator (internal endpoint or external)
  3. No Hardcoded External Calls: PATAS does not hardcode sending data to external services
  4. Strict Privacy Mode: Can disable external LLM providers entirely

See Privacy-and-Data-Protection for details.

LLM Configuration

Provider Configuration

LLM provider is configurable by the operator:

# In app/config.py
llm_provider: str = "openai"  # or "none", "disabled", "local"
llm_model: str = "gpt-4o-mini"

Options:

  • "openai": Use OpenAI API (requires API key)
  • "local": Use local/on-prem LLM endpoint (configure endpoint URL)
  • "none" or "disabled": Disable LLM entirely (pattern mining uses rule-based methods only)

Privacy Mode

In STRICT privacy mode:

  • External LLM providers are disabled by default
  • Only internal/on-prem LLM endpoints are allowed
  • Message texts are not stored in logs

See Privacy-and-Data-Protection for privacy mode details.

LLM Prompt Safety

Prompt Constraints

The LLM prompt explicitly instructs:

  1. Narrow Patterns: Focus on explicit commercial/abusive spam, not broad categories
  2. No Sensitive Attributes: Never base rules on politics, religion, race, ethnicity, gender, sexual orientation
  3. Safe SQL Only: Generate only SELECT queries with whitelisted columns
  4. No Match-Everything: Rules must not match >80% of messages
  5. Interpretable: Patterns must be human-readable and explainable

Prompt Example

See app/v2_llm_engine.py for the full prompt. Key sections:

CRITICAL SAFETY REQUIREMENTS:
1. Generate NARROW, INTERPRETABLE patterns
2. Avoid over-broad rules
3. NEVER base rules on sensitive attributes
4. SQL rules MUST be safe SELECT queries only

LLM Output Validation

All LLM-generated rules go through multiple validation layers:

1. SQL Safety Validation

  • ✅ Only SELECT queries
  • ✅ Only whitelisted tables/columns
  • ✅ No UPDATE/DELETE/INSERT/ALTER
  • ✅ No subqueries, no semicolons
  • ✅ No "match everything" patterns

See app/v2_sql_safety.py for implementation.

2. Coverage Check

  • ✅ Rules matching >80% of messages are rejected
  • ✅ Prevents over-broad rules from LLM

3. LLM Quality Validation (Optional)

  • ✅ Assesses false positive risks (low/medium/high)
  • ✅ Rejects high-risk rules before creation
  • ✅ Logs warnings for medium-risk rules
  • ✅ Token-efficient (~200-400 tokens per validation)
  • ✅ Uses same LLM client as pattern mining (no additional API key needed)
  • ✅ Graceful degradation (skips if LLM unavailable)

How it works: After deterministic validation passes, an optional LLM validation step analyzes the SQL rule for semantic false positive risks. The validator uses a compact prompt focused on false positive scenarios, requiring minimal tokens. Rules flagged as "high risk" are rejected; "medium risk" rules are logged but allowed to proceed.

See SQL_LLM_VALIDATION.md for detailed documentation.

4. Offline Evaluation

  • ✅ Rules tested on historical data
  • ✅ Precision, recall, ham hit rate calculated
  • ✅ Rules classified into tiers (SAFE_AUTO / REVIEW_ONLY / FEATURE_ONLY)

5. Safety Profiles

  • ✅ Only SAFE_AUTO rules in Conservative profile
  • ✅ Balanced/Aggressive profiles require manual review
  • ✅ Safety guardrails prevent unsafe auto-promotion

LLM vs Machine Learning

Important: PATAS does NOT use Machine Learning for classification.

Why not ML?:

  • ML models are black boxes, hard to interpret
  • ML requires large training datasets and continuous retraining
  • ML models can degrade over time without clear explanation
  • ML adds complexity without clear benefit for pattern-based spam detection

What PATAS uses instead:

  • Rule-based pattern matching: Explicit SQL rules that are interpretable
  • LLM for pattern discovery: LLM helps discover patterns, but rules are explicit and auditable
  • Statistical evaluation: Rules are evaluated on historical data with clear metrics

LLM Limitations & Mitigations

Limitations

  1. LLM can generate unsafe rules: Mitigated by SQL safety validation
  2. LLM can generate over-broad patterns: Mitigated by coverage checks and quality filtering
  3. LLM can hallucinate patterns: Mitigated by offline evaluation on real data

Mitigations

  1. Multi-layer validation: SQL safety → Coverage check → Offline evaluation → Tier classification
  2. Human review: REVIEW_ONLY and FEATURE_ONLY tiers require manual review
  3. Safety profiles: Conservative profile only includes SAFE_AUTO rules
  4. Rollback capability: Rules can be deprecated if quality degrades

Summary

LLM Role in PATAS:

  • ✅ Offline pattern discovery (not online classification)
  • ✅ Hypothesis generation (not final decisions)
  • ✅ Aggregated data only (not individual messages)
  • ✅ Configurable provider (internal or external, operator's choice)
  • ✅ Multi-layer validation (safety checks, evaluation, tiering)

LLM Does NOT:

  • ❌ Classify messages in real-time
  • ❌ Make ban/unban decisions
  • ❌ Process individual user messages online
  • ❌ Send data to external services by default

Privacy:

  • ✅ On-prem deployment by default
  • ✅ LLM provider configurable by operator
  • ✅ Strict privacy mode available
  • ✅ No hardcoded external calls

Last Updated: 2025-01-15

Clone this wiki locally