[Bug] Presidio guardrail: end-to-end PII masking broken with Anthropic native API

[firestaerter3]

Summary

When using the Anthropic native API path (not OpenAI compat) with Presidio guardrail and output_parse_pii: true, several bugs prevent end-to-end PII masking from working. PII is masked on input but never unmasked in responses, and using tools (e.g. via Claude Code) causes a 400 error.

Symptoms

1. 400 error — Input tag 'function' found using 'type' does not match expected tags: 'bash_20250124', 'text_editor_20250124', ...
   Triggered when any tool is present in the request.

2. PII never unmasked in non-streaming responses — <PERSON> appears in output instead of the original name.

3. PII never unmasked in streaming responses on Anthropic native path — same issue, different code path.

Root Causes

1. Tool key contamination: apply_guardrail() returns the entire mutated inputs dict including OpenAI-converted tool keys (tools, structured_messages, model, images) added by process_input_messages(). On the Anthropic native path this overwrites native tool definitions with the OpenAI type:"function" format.

2. No unmask path in apply_guardrail(): Only a masking path exists — when called with input_type="response" and output_parse_pii=True, PII tokens are never restored.

3. Abstract interface out of sync: All process_output_response() implementations accept request_data but the abstract base (BaseGuardrailTranslation) does not declare the param, causing a type/mypy mismatch.

4. Streaming hook handles only ModelResponseStream: The async_post_call_streaming_iterator_hook unmasking path iterates chunks expecting ModelResponseStream objects. With the Anthropic native API path, response chunks are raw bytes (SSE format) — these pass through unprocessed, so unmasking never runs.
   Additionally, AnthropicMessagesHandler.process_output_response() creates a fresh request_data dict, discarding any pii_tokens stored during input masking.

Config

model_list:
  - model_name: claude-sonnet
    litellm_params:
      model: anthropic/claude-sonnet-4-5  # native, not openai-compat

guardrails:
  - guardrail_name: pii_masking
    litellm_params:
      guardrail: presidio
      default_on: true
      output_parse_pii: true

Provider: anthropic/ native (not via OpenAI compat)

Fix PRs

Each root cause is addressed in a separate PR (will link once submitted):

• PR A — fix/guardrail-request-data-passthrough: Sync abstract BaseGuardrailTranslation.process_output_response() signature with all implementations (root cause 3)
• PR B — fix/presidio-native-tool-stripping: Strip OpenAI-converted keys before returning from apply_guardrail() (root cause 1)
• PR C — fix/presidio-output-unmask: Add output unmask path to apply_guardrail() (root cause 2)
• PR D — fix/presidio-anthropic-sse-unmask: Handle Anthropic native SSE bytes in streaming unmask hook + fix AnthropicMessagesHandler (root cause 4)

[firestaerter3]

Update: The previous PRs (#22879–#22882, #23037) have been closed and superseded by two fresh PRs that address all Greptile review feedback:

• fix(guardrail): pass request_data to output response handlers #24290 — fix(guardrail): pass request_data to output response handlers (plumbing, 6 tests)
• fix(presidio): PII token round-trip masking/unmasking #24291 — fix(presidio): PII token round-trip masking/unmasking (all 6 root causes fixed, 13 tests)

Key improvements over the previous PRs:

• 19 tests total (vs 0 before)
• True streaming unmask with carry buffer (preserves real-time SSE streaming)
• Safer fallback logic (removed UUID-suffix matching that was flagged as false-positive risk)
• All existing upstream tests pass

[someswar177]

This issue appears to be the same root cause described in PR #5068 and Issue #6247 —
the pii_tokens offset bug in presidio.py where left-to-right replacement without
index recalculation corrupts the string.

PR #5068 proposed a right-to-left fix in August 2024 but was closed as stale without
being merged. Linking here for visibility.

[crp4222]

Ran into similar issues with in-framework Presidio — ended up moving PII handling to a standalone proxy upstream. Decouples anonymization from the LLM gateway entirely, avoids provider-specific bugs: https://github.com/crp4222/PrivAiTe

[crp4222]

Ran into similar issues with in-framework Presidio — ended up moving PII handling to a standalone proxy upstream. Decouples anonymization from the LLM gateway entirely, avoids provider-specific bugs. You can sit it in front of litellm without changing anything on the litellm side: https://github.com/crp4222/PrivAiTe

[crp4222]

The proxy-level approach avoids this entirely — PII is stripped before it reaches any provider, so Anthropic vs OpenAI API differences don't matter. We ran into the same Presidio issues (byte/char offset mismatch on non-ASCII, MISC entity misclassification) and fixed them with contextual recognizers and a name validation filter. Open source: https://github.com/crp4222/PrivAiTe