← Back to README · Architecture

• Threat Model
• Guardrails Pipeline
• Security Layers
• Dataset Field Policy
• Data Ingestion Pipeline
• Evaluation

Three categories of attack are defended against:

Every query passes through five sequential gates before a response is returned.

flowchart TD
    Q([User Query]) --> INJ

    INJ{"Prompt injection\ndetected?"}
    INJ -->|yes| B1([🚫 Block — injection])
    INJ -->|no| RES

    RES{"Restricted field\nrequested?"}
    RES -->|yes| B2([🚫 Block — restricted data])
    RES -->|no| INT

    INT{"RESTRICTED\nintent classified?"}
    INT -->|yes| B3([🚫 Block — intent])
    INT -->|no| RET

    RET["Retrieve docs\nALLOWED_RETURN_FIELDS only"]
    RET --> LLM["LLM generation"]

    LLM --> SAN{"Response contains\nrestricted terms?"}
    SAN -->|yes| RDC["🔒 Redact → [redacted]"]
    SAN -->|no| OK

    RDC --> OK([✅ Safe response returned])

Regex patterns match known injection vectors before any retrieval or LLM call:

• ignore (previous|above|all) instructions
• you are now, act as, pretend (you are|to be)
• <system>, [INST], ### instruction delimiters
• DAN, jailbreak, developer mode and similar tokens

If matched → RAGResponse(blocked=True, block_reason="prompt_injection") is returned immediately.

A keyword blocklist rejects queries that ask for internal fields:

supplier · margin · internal notes · warehouse · profit · cost price

Pattern is case-insensitive. If matched → blocked before retrieval.

The classifier assigns one of:

RESTRICTED intent fires on queries asking for confidential operational data even when the phrasing does not match Layer 2 keywords.

The retriever only returns fields in ALLOWED_RETURN_FIELDS. Raw index records contain all fields including any that were present before ingestion cleaning — this layer guarantees none leave the retrieval layer regardless.

ALLOWED_RETURN_FIELDS = {
    "product_id", "item_name", "category", "country",
    "price_local", "currency", "technical_specs", "score"
}

After the LLM generates a response, a final scan replaces any remaining restricted terms with [redacted]:

supplier · margin · internal notes · warehouse · profit margin

This is a defence-in-depth backstop. The primary protection is that Internal_Notes is stripped at ingestion and therefore never enters the index or LLM context.

Internal_Notes is removed during ingestion — before embedding — so it cannot be retrieved by any query, including adversarial ones.

flowchart LR
    A["📄 products_data_3000.csv\n(raw)"] --> C
    B["📊 Task Data .xlsx\n(optional merge)"] --> C

    subgraph C["pipelines/ingestion/clean_data.py"]
        direction TB
        C1["Drop Internal_Notes"] --> C2["Build internal_notes\n(name + specs + category)"]
    end

    C --> D["📄 data/processed/\ncleaned CSV"]
    D --> E

    subgraph E["pipelines/indexing/build_vector_index.py"]
        direction TB
        E1["Sentence Transformer\nall-MiniLM-L6-v2\n(384-dim)"] --> E2["FAISS IndexFlatIP\n+ L2 normalise"]
    end

    E --> F["🗄️ vector_store/faiss_index/\nindex.faiss\nmetadata.json"]

metadata.json stores only ALLOWED_RETURN_FIELDS — the ingestion pipeline never writes Internal_Notes to this file.

evaluation/test_queries.py verifies all security guarantees automatically.

# Run full suite (requires OPENROUTER_API_KEY)
uv run --group dev python evaluation/test_queries.py

# Skip LLM calls (CI-safe)
EVAL_MOCK_LLM=1 uv run --group dev python evaluation/test_queries.py