feat: PII/secret masking in LLM traffic

[thejoeejoee]

Complement to #93 (credential proxy). Even with API keys hidden from the agent, the agent can still leak sensitive data from the codebase (credentials found in source, PII from databases, internal URLs, etc.) to LLM providers via request bodies.

Problem

# Agent reads a config file containing a database password:
DB_PASSWORD=s3cret123

# Agent sends it to the LLM as part of the conversation:
POST https://api.anthropic.com/v1/messages
{"messages": [{"content": "I found this config: DB_PASSWORD=s3cret123 ..."}]}

The LLM provider now has the database password. Even if the agent didn't intend to leak it, the secret is in the training/logging pipeline.

Goal

Scan LLM request/response bodies for secrets and PII — replace with reversible placeholders before sending to the provider, restore originals in responses. The agent works normally but sensitive data never leaves the container.

Prior art

Detailed research in #93 comment. Top candidates:

Tool	Language	Latency	Reversible	Notes
CloakPipe	Rust	3.2ms	✅	AES-256-GCM vault, 5-layer detection, F1 0.94, streaming, MCP server
PasteGuard	TS/Bun	97ms	✅	Microsoft Presidio, 30+ PII types, 24 languages, dashboard
mirage-proxy	Rust	<1ms	✅ (fakes)	Plausible fake values instead of tokens, 129 patterns
LLM Guard	Python	varies	❌	Most mature (2.7K★), 20+ scanners, not reversible
LiteLLM + Presidio	Python	varies	✅	Multi-provider routing, most flexible
AegisGate	Python	varies	✅	PII + prompt injection + response sanitization

Integration with OpenCode

OpenCode supports baseURL override per provider and HTTP_PROXY/HTTPS_PROXY env vars — both work for routing traffic through a masking proxy. See integration research.

Possible approach

PII masking proxy as a sidecar container (same as credential proxy in #93, could be the same proxy process):

1. Proxy receives LLM request from OpenCode
2. Scans request body for secrets (API keys, passwords, tokens) and PII (names, emails, IPs)
3. Replaces with reversible placeholders ([[SECRET_1]], [[EMAIL_1]])
4. Forwards to provider (after credential injection from feat: credential proxy — agent uses API keys without seeing them #93)
5. On response, reverses placeholders so the agent sees original values

Relationship to other issues

• feat: credential proxy — agent uses API keys without seeing them #93 — Credential proxy (hides API keys from agent) — independent, higher priority
• #XX — General-purpose proxy with SSL bump — this could be the transport layer for both credential injection and PII masking

[thejoeejoee]

Research: PII/secret masking proxies

These tools mask PII and secrets in LLM API traffic (agent → provider), preventing sensitive data from codebase/context from leaking to LLM providers. Complementary to credential proxy (#93) — that hides API keys from the agent, this hides codebase data from the LLM provider.

Landscape overview

The space splits into three tiers:

Tier	Tools	Language	Latency	Approach
High-performance	CloakPipe, mirage-proxy	Rust	<5ms	Regex + ONNX NER, plausible fakes
Feature-rich	LLM Guard, AegisGate, LiteLLM+Presidio	Python	50–200ms	NLP + ML, prompt injection detection
Simple proxy	PasteGuard, ctxproxy, CloakLLM	TS/Python	50–100ms	Presidio/spaCy, mask/route modes

---

Top candidates for jailoc sidecar

1. CloakPipe (Rust) — best performance

• Repo: rohansx/cloakpipe · MIT · 21★
• Latency: 3.2ms avg, 4.8ms P99
• Detection: 5-layer pipeline (regex → financial → ONNX NER → fuzzy → custom), 33+ entity types, F1 0.94
• Masking: Deterministic within session (same entity → same token), non-deterministic across sessions
• Vault: AES-256-GCM encrypted, session-scoped, local-only
• Streaming: SSE rehydration
• Docker: ghcr.io/cloakpipe/cloakpipe:latest, single binary, no external deps
• Policies: DPDP, GDPR, HIPAA, PCI-DSS built-in
• MCP server: 6 tools

2. PasteGuard (TypeScript/Bun) — simplest setup

• Repo: sgasser/pasteguard · Apache 2.0 · 583★
• Engine: Microsoft Presidio (Docker service, port 5002) for PII + custom regex for secrets
• Latency: ~97ms avg (Presidio: 50–80ms, secrets regex: 5–20ms, masking: 5–10ms)
• Memory: ~4GB (Presidio NLP models)
• Entities: 30+ PII types (24 languages), 8 secret types (API keys, JWT, private keys, env vars)
• Modes: Mask (placeholder round-trip) or Route (PII → local LLM, clean → cloud)
• Streaming: Real-time SSE unmasking with partial placeholder buffering
• Providers: OpenAI, Anthropic, any OpenAI-compatible
• Dashboard: Web UI at /dashboard with request logs (SQLite)
• Docker: ghcr.io/sgasser/pasteguard:en (English) / :eu (8 European languages)
• Limitation: 4GB memory, ~100ms overhead, provider sees [[PERSON_1]] placeholders

3. mirage-proxy (Rust) — plausible fakes

• Repo: chandika/mirage-proxy · MIT · 34★
• Latency: Sub-millisecond (regex + entropy only)
• Approach: Replaces secrets with plausible fakes (AWS key → valid-looking AWS key), not tokens
• Detection: 129 patterns (Gitleaks + secrets-patterns-db)
• Vault: AES-256-GCM, Argon2id key derivation, persist across restarts
• Sensitivity levels: low (secrets only) → medium (+ PII) → high → paranoid
• Limitation: Regex-only — won't catch secrets in natural language

4. AegisGate (Python) — most comprehensive

• Repo: ax128/AegisGate · MIT · 34★
• Features: PII redaction (50+ patterns) + prompt injection detection (bilingual, no GPU) + dangerous response sanitization (shell commands, SQL injection) + MCP support
• Pipeline: PII redaction → exact-value redaction → request sanitizer → RAG poison guard → (LLM) → injection detector → anomaly detector → privilege guard → tool call guard → restoration
• Routing: Token-based multi-upstream, Anthropic ↔ OpenAI protocol conversion
• Web UI: Admin console for policy management
• Docker Compose: Full stack with SQLite/Redis/PostgreSQL + Caddy TLS

5. LLM Guard (Python) — most mature

• Repo: protectai/llm-guard · MIT · 2,782★ · 2.5M+ downloads
• Scanners: 20+ input + 20+ output scanners (PII, secrets via detect-secrets, prompt injection, toxicity, ban topics/code/competitors)
• Latency: 50–200ms (spaCy NER)
• Docker: protectai/llm-guard:latest
• Limitation: Not reversible (masks but doesn't unmask in responses)

6. LiteLLM + Presidio — most flexible

• Repo: BerriAI/litellm · MIT · 10K+★
• Approach: LiteLLM proxy with Presidio guardrails, multi-provider routing (15+ providers)
• Features: PII masking, secret detection, cost tracking, rate limiting, per-request PII controls
• Deployment: 3 containers (LiteLLM + Presidio analyzer + Presidio anonymizer)
• Best for: Already using LiteLLM or needing multi-provider support

---

Agent security frameworks (broader scope)

These go beyond PII masking — they enforce policies, isolate credentials, and sandbox agent actions:

Tool	Focus	Key feature
Wardgate (117★)	Credential isolation gateway	API gateway where agents never see real credentials + isolated execution environments
nono	Phantom token pattern	Agents get session tokens, real credentials in system keystore, kernel-level enforcement (Landlock/Seatbelt)
AgentGuard	MCP + LLM proxy layer	Intercepts agent-OS interactions + scans prompts for secrets/PII, zero prompt engineering
Guardplane	Security kernel	Workspace sandboxing + MCP tool routing + policy enforcement, default-deny capability model
agentguardhq/agentguard	Governance kernel	24 built-in safety invariants, 93 destructive pattern detectors, 4 enforcement modes

---

Comparison matrix (sidecar candidates)

Tool	Lang	Latency	F1	Reversible	Memory	Streaming	Prompt injection	License
CloakPipe	Rust	3.2ms	0.94	✅	Low	✅	❌	MIT
mirage-proxy	Rust	<1ms	~0.61	✅	Low	✅	❌	MIT
PasteGuard	TS	~97ms	~0.84	✅	~4GB	✅	❌	Apache 2.0
AegisGate	Python	<50ms	—	✅	Medium	✅	✅	MIT
LLM Guard	Python	50–200ms	0.82	❌	Medium	✅	✅	MIT
LiteLLM+Presidio	Python	50–200ms	0.84	✅	High (3 containers)	✅	❌	MIT

---

Recommendation for jailoc

CloakPipe as an optional sidecar — <5ms overhead, single binary, reversible masking, encrypted vault, Docker-ready. Falls naturally into the existing compose stack as a third service alongside opencode and dind:

services:
  cloakpipe:
    image: ghcr.io/cloakpipe/cloakpipe:latest
    environment:
      - CLOAKPIPE_UPSTREAM_URL=https://api.openai.com
    volumes:
      - cloakpipe-vault:/data/vault

Agent's OPENAI_BASE_URL points to http://cloakpipe:3100/v1 — transparent, no SDK changes needed.

This is complementary to the credential proxy (#93) — credential proxy hides API keys from the agent, PII proxy hides codebase data from the LLM provider. Both can coexist as middleware in the general-purpose proxy (#95).

[crp4222]

Interesting timing — we shipped something similar as an OpenAI-compatible proxy. Key learnings: spaCy NER alone produces too many false positives on code (method calls like logging.getLogger match as URLs, tech terms as locations). We ended up filtering SpacyRecognizer results from non-primary languages and adding contextual pattern recognizers. Details + benchmark: https://github.com/crp4222/PrivAiTe

[crp4222]

This is exactly why we added the OpenAI privacy-filter model as an ONNX detector — it catches passwords, API keys, and secrets that regex-based tools miss entirely. Combined with Presidio for structured PII (emails, phones, cards), it covers both the "data from databases" and "credentials in source" cases you describe. The proxy approach means it works regardless of the agent framework: https://github.com/crp4222/PrivAiTe