Skip to content

llm-shield-proxy 1.7.1

Latest

Choose a tag to compare

@ninadphalak ninadphalak released this 12 Nov 22:01
· 7 commits to main since this release
3a2c719

llm-shield-proxy 1.7.1 is a security release. It fixes two cases where streamed responses could pass sensitive values unredacted when response redaction is enabled, and it ships the hardening from a full audit (#214). Please upgrade.

pip install --upgrade llm-shield-proxy

Streamed response redaction (#223)

These two cases affect only deployments that set ENABLE_RESPONSE_PII_REDACTION=true (it is off by default). They are present in every release since 1.6.0, up to and including 1.7.0. Non-streamed responses through the proxy and request-side redaction were not affected.

A reply to a request that carried no personal data

Before: when the request carried no personal data, nothing was masked, and the streamed reply skipped the response scan. Only the last held-back word was scanned at the end of the stream. A value the model wrote anywhere before it reached the client unredacted, even when it arrived in one event.
After: the streamed reply is scanned whether or not the request carried personal data.

A value printed with spaces, delivered in pieces

Before: the hold-back kept only the text after the last space. A card number or IBAN printed in groups (4111 1111 1111 1111) could leave one group at a time when events ended at those spaces, as a provider that sends about one token per event does, and no group matched a detector on its own.
After: the last RESPONSE_PII_SCAN_WINDOW characters (64 by default) are held, never cutting a word or a detected value, so a value up to that length is scanned whole.

Also in this fix

  • Streamed reasoning (delta.reasoning_content, delta.reasoning) and Anthropic thinking deltas get their own hold-back window, like the answer text. Before, they were scanned one event at a time.
  • A caller that passes its last text together with the end-of-stream flag gets the held tail scanned before it leaves.

Stand-ins that are no longer ordinary words

With synthetic masking, values of some types (encoded values, digit runs, medical record numbers, and any type without its own fake format) were replaced by a dictionary word such as where or call. Restoring the reply then put the caller's value wherever the model wrote that word. Those types now get opaque stand-ins (ref_ and 12 characters, 9 and 11 digits, or QQ and 6 digits and a letter for UK National Insurance numbers). Sessions saved before the upgrade keep their stored stand-ins and still restore.

Hardening from a full audit (#214)

  • Adding "jsonrpc":"2.0" to a request no longer skips normalization, blast-radius limits, SCRUB/HMAC masking or response redaction. PUT, PATCH and DELETE bodies are redacted; non-POST replies and replies whose JSON root is not an object are scanned when response redaction is on.
  • A malformed client field returns 400 instead of letting FAIL_OPEN forward the raw body. Luhn-valid card numbers sent as JSON integers, and PII used as object keys, are redacted.
  • Client-chosen upstreams get a proxy-held key only over HTTPS, and failover no longer carries the client-chosen provider's key to the fallback host. Client paths can no longer leave a path-scoped upstream base.
  • Per-key policy: roles are resolved before the credential swap and looked up by the configured key; an empty policy map no longer disables the FAIL_CLOSED gate. The role settings enforced_masking_mode and allowed_masking_modes are enforced.
  • STATELESS_CRYPTO tokens are bound to the virtual key and session ([ENC_v2_]). The guard API (/v1/guard/*) applies the same per-key policy, identity check, rate limit and body cap as the proxy. Vault eviction is fair per tenant.
  • /v1/mcp requires a valid virtual key, authorizes resources/read (deny by default, new allowed_resources policy key), scrubs the upstream error member, and fails closed on egress-scan depth overflow and ambiguous URL forms. Audit records store key fingerprints, never raw keys.
  • Hot reload revokes keys and flags removed from .env or config.yaml; policies.yaml changes are detected by content. The agent-identity enforcer requires an audience; the DPoP replay window covers the full freshness window; optional PUBLIC_BASE_URL binds DPoP htu.

Behaviour changes

  • [ENC_v1_] tokens minted before this release no longer restore.
  • resources/read on /v1/mcp needs allowed_resources in the policy, and /v1/mcp needs a valid virtual key unless OVERRIDE_CLIENT_AUTH is set.
  • The agent-identity enforcer needs ALLOWED_AUDIENCES or a role-level allowed_audiences; without one every request gets 401.
  • The guard API applies policies.yaml: an unmapped key under FAIL_CLOSED with no default_role gets 403.
  • With response redaction on, streamed text is held back about 64 characters (the documented window, at most twice that while a long match is still arriving) instead of one word, so the client sees it slightly later.

If you cannot upgrade yet

Have clients that rely on response redaction send "stream": false. Non-streamed replies are scanned whole and were not affected.

pii-leak-benchmark 0.7.0 and chunk-invariance 0.3.0 are released alongside.