bug: output guardrail does not run on streaming responses (silent bypass via stream:true)

[moonming]

Summary

The output guardrail (hook_point: "output") only fires on the non-streaming path. When a caller requests stream: true, the gateway forwards SSE chunks from upstream verbatim and never invokes state.guardrails.check_output(...) on the assembled assistant response. As a result, a forbidden literal that the model emits inside a streamed completion reaches the caller verbatim — bypassing the output guardrail entirely.

Where the gap is

crates/aisix-proxy/src/chat.rs has exactly one check_output call site, in the non-streaming branch around line 793 (post-#196). The streaming branch (build_sse_stream in the same file) wraps the upstream chunk stream and forwards rendered chunks but never accumulates the assistant text and never calls check_output. The same gap exists for /v1/responses streaming.

Severity

MEDIUM-HIGH for any deployment that has output guardrails configured AND serves streamed completions:

• A customer enabling output guardrails reasonably expects them to apply uniformly across response shapes. There's no UI signal in the admin API saying "this guardrail is off when stream=true".
• The gap is invisible: the guardrail is configured, fires correctly on non-streaming responses, and silently does nothing on streaming. Detection requires writing the kind of e2e regression that doesn't exist today.
• For kind: "keyword" with literal/regex match, streaming makes the bypass trivial: any caller can just add stream: true to their request to skip the policy.

What "correct" looks like

Two reasonable shapes:

A. Buffer-then-check (simplest): accumulate the assistant content across chunks, run check_output once at end-of-stream, and if it blocks: emit an SSE event: error chunk and close (per docs §5 for abnormal termination). This loses TTFB advantages of streaming but matches the semantics of the non-streaming path.

B. Streaming guardrail (more invasive): invoke an output guardrail per-chunk on the partial accumulator. Block on first match, emit error chunk, close. Lower latency to first token, but content already streamed before the match still reaches the caller — for "keep forbidden content from caller" this can leak the prefix that triggered the match.

Most customer expectations align with A: "if I configured an output guardrail, I want forbidden content blocked, I'll trade TTFB for safety." Worth a config flag if shape B is also desired.

Repro shape (for the held-back e2e)

// Mock upstream emits SSE chunks; one of them carries FORBIDDEN_WORD
upstream = await startOpenAiUpstream({
  streamEvents: [
    `{"id":"x","object":"chat.completion.chunk","model":"gpt-4o","choices":[{"delta":{"role":"assistant"},"finish_reason":null}]}`,
    `{"id":"x","object":"chat.completion.chunk","model":"gpt-4o","choices":[{"delta":{"content":"sure here it is: ${FORBIDDEN_WORD}"},"finish_reason":null}]}`,
    `{"id":"x","object":"chat.completion.chunk","model":"gpt-4o","choices":[{"delta":{},"finish_reason":"stop"}]}`,
    "[DONE]",
  ],
});
// Configure output keyword guardrail on FORBIDDEN_WORD.
// Caller streams.
// Expected: stream throws (or emits SSE error event) BEFORE FORBIDDEN_WORD reaches the iterator.
// Today: the FORBIDDEN_WORD is delivered to the caller verbatim. Bug.

Surfaced by

Independent audit on PR #203 (#153 redaction fix). Audit MEDIUM-1: "streaming output path has no guardrail and no redaction test". This issue is out of scope for the redaction fix (which is about the error envelope shape) but is the same product surface — output guardrails — and affects the same security contract.

Companion

• bug: output guardrail error message echoes the matched forbidden literal back to caller #153 (closed by fix(proxy): redact matched literal from guardrail-block error envelope #203): wire-level redaction of the matched literal in the error envelope.
• enhancement: optional strict-mode error envelope sanitization (OpenAI-conformant taxonomy) #199: system-wide error-envelope leakage (BridgeError Display + taxonomy).

[coderabbitai]

🔗 Related PRs

api7/ai-gateway-stash#67 - refactor: metrics hook [merged]
api7/api7-ee-3-gateway#1310 - fix: handle misaligned HTTP chunk and SSE event boundaries in AI proxy streaming [merged]
api7/api7-ee-3-gateway#1319 - fix: prevent apisix_llm_active_connections gauge leak when plugin exits early via ngx.exit() [merged]
api7/api7-ee-3-gateway#1458 - fix(ai-proxy): return 502 when streaming converter receives mismatched response format [merged]
#90 - feat(provider): move into sub crate [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[moonming]

Triage update — confirmed real bug (HIGH priority)

Verified against the de-facto reference implementation in this category. Conclusion: this is a real bug — streaming output guardrails are baseline industry-standard, not optional.

Evidence

Streaming output guardrails are first-class in the reference implementation. The proxy's central dispatcher chains every guardrail's streaming hook around the upstream response iterator on every streaming chat-completion request — guardrails are not bypassed when stream: true:

# Equivalent pattern (reference: ProxyLogging.async_post_call_streaming_iterator_hook):
async def async_post_call_streaming_iterator_hook(response, ...):
    for callback in litellm.callbacks:
        if has_streaming_hook(callback):
            response = await callback.async_post_call_streaming_iterator_hook(response, ...)
    return response

This wraps the response iterator unconditionally for streaming requests. Multiple guardrail kinds implement this hook — confirming it is the canonical extension point, not an exotic feature: bedrock_guardrails, aim, presidio, model_armor, noma, enkryptai, dynamoai, panw_prisma_airs, unified_guardrail, ibm_guardrails, tool_permission.

Reference framework supports both strategies — pick per guardrail kind

The reference exposes a single hook (async_post_call_streaming_iterator_hook) and lets each guardrail pick its evaluation cadence:

• Buffer-then-check (Bedrock guardrails) — collect all chunks, assemble a full response, run the check once, replace with blocked-message stub or apply masking before re-emitting.
• Per-chunk / streaming evaluation (AIM via WebSocket) — forward each chunk to a streaming scanner that yields verified payloads only.

What the caller sees on a block

A clean SSE error event, not a raw cut. The reference catches StreamingCallbackError in the proxy's stream handler, JSON-encodes a ProxyException, and yields:

data: {"error": {"message": "...", "type": "..."}}\n\n

The stream terminates gracefully on a SSE-shaped error payload that downstream OpenAI-compatible clients can parse. (This shape is also already what api7-cloud emits for streaming abnormal termination after #198.)

Recommended fix shape for api7-cloud

Buffer-then-check is the more defensible default for kind: "keyword" deny-lists:

• Per-chunk evaluation on a sliding accumulator is wrong for blocking guardrails: by the time chunk N matches the forbidden literal, chunks 1..N-1 have already been emitted to the caller — the secret leaks regardless. Per-chunk only makes sense for masking/anonymization guardrails (where partial leakage is repaired by rewriting tokens) or for low-latency providers that own the wire and can hold chunks server-side until verified.
• Keyword scanning is cheap; the latency cost of buffering an entire completion is bounded.

Implementation outline:

1. In crates/aisix-proxy/src/chat.rs::build_sse_stream, introduce an output-guardrail wrap around the upstream chunk stream (mirror the existing errored flag pattern from fix(proxy): omit terminal [DONE] when stream terminates abnormally #198).
2. Accumulate chunk.delta.content across the stream into a String buffer.
3. At end-of-stream (clean completion path, NOT abnormal-termination path), call state.guardrails.check_output(...) on a synthesized ChatResponse with the accumulated content.
4. On Block { reason }: emit an SSE event: error frame with the OpenAI-shape envelope (parallel to fix(proxy): omit terminal [DONE] when stream terminates abnormally #198's error_frame_payload), then close WITHOUT [DONE]. Apply the same redaction as bug: output guardrail error message echoes the matched forbidden literal back to caller #153's non-streaming path ("response blocked by content policy") and emit the rich reason via tracing::warn!.
5. v2 follow-up could add a per-chunk evaluation mode for masking-style guardrails.

Disposition

• Severity: bug + vulnerability (matches initial framing — the security control's contract is silently violated under stream: true).
• Priority: HIGH (security control bypass).
• E2E shape: caller sets stream: true, mock upstream emits chunks where chunk N contains the forbidden literal, assert iterator throws + final SSE error event shape + accumulated delta.content does NOT contain the forbidden literal (or partial — depending on chosen strategy).