Skip to content

5. PII Guard

Hemant Kohli edited this page Sep 23, 2026 · 4 revisions

PII guard

Regex checks run alongside the selected token-classification model. Regex covers email, Australian phone and identifier patterns, and credit-card-like numbers.

Mode Default model
ner (default) Xenova/bert-base-NER
classifier openai/privacy-filter

NER maps person entities to NAME. Classifier mode aggregates spans and maps account_number, private_address, private_email, private_person, private_phone, private_url, private_date and secret to ACCOUNT_NUMBER, ADDRESS, EMAIL, NAME, PHONE, URL, DATE and SECRET respectively.

Fine-tuned labels

Place these options in your shared configuration:

pii: {
  mode: "classifier",
  model: "your-org/your-pii-model",
  labelMappings: {
    LABEL_0: null,
    LABEL_1: "NAME",
    customer_id: "CUSTOMER_ID"
  }
}

Mappings work in both modes and override built-in labels. Keys are case insensitive and ignore BIOES prefixes such as B-. Use unprefixed keys. Unmapped labels retain built-in behavior; unknown labels are ignored. A null mapping suppresses that model label but does not disable regex detection. Token types must contain only uppercase letters and underscores.

Classifier spans use valid character offsets first, otherwise a matching word. NER retains its previous person-token handling unless mapped. Use Transformers.js-compatible token-classification models; classifier mode currently loads q4 weights. Label mapping does not convert or fine-tune models.

Masking and storage

Email alice@example.com becomes Email [[EMAIL_<namespace>_0]]. Responses and tool inputs restore the original value. The reversible option remains accepted; current hooks restore tokens regardless of its value.

The default vault is in memory. Provide createRedisPiiVaultStorage(redis, { keyPrefix: "app:pii", ttlSeconds: 3600 }) as pii.vault.storage and choose a pii.vault.scopeId appropriate to your application. fallbackToMemory: true enables a process-local TTL mirror during Redis failures; the default is false. Custom adapters use createPiiVaultStorage({ get, set, entries, getByToken }).

Cross-turn recovery uses opaque token lookup. Keep tenant authorization in the application and storage adapter and choose appropriate storage boundaries.

Token namespaces use cryptographic randomness. Masking handles longer values before shorter overlapping values and ignores empty matches. Tokens are cached only after their vault writes succeed. Vault access through middleware/tokenizers fails with sanitized GuardOperationalError (VAULT_UNAVAILABLE); direct adapter calls retain adapter errors.

Scopes and opaque tokens are not tenant authorization. Cross-turn lookup can recover tokens across scopes in the same backend; isolate adapters or Redis prefixes and token indexes across trust boundaries. Vault values are not encrypted by the adapter. In-memory storage has no automatic expiry; Redis TTL expires keys and refreshes on writes, rather than setting a per-entry retention deadline. See 12.-Security-and-Operational-Errors.

Clone this wiki locally