Expose Original Client Headers to Policies via Downstream/Upstream Context #2735
Replies: 1 comment 2 replies
-
Decision: Option 1 — Keep Phase-Sequential, Add Structured ContextOption 2 carries behavioral regression risk for the existing policy base and requires restructuring the ext_proc callback orchestration across both buffered and streaming paths. Option 1 preserves full backward compatibility, has lower implementation risk, and can be shipped independently of any execution model change. The original framing of Option 1 — a flat SDK Context ExtensionsProblem being solvedIn phase-sequential execution, by the time The fix is to give every context type a stable reference to the headers as they arrived at chain entry, before any policy touched them. We extend this to upstream information as well, so policies have a consistent, structured view of both sides of the connection. New Types
|
Beta Was this translation helpful? Give feedback.
-
|
Let's go with just fields instead of functions to be consistent with other fields. "Upstream": { // NEW
"URL": "string ",
"BasePath": "string",
"Response": {
"Headers": "Headers"
}
} |
Beta Was this translation helpful? Give feedback.
-
|
Previously |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
When an API operation has a policy chain A → B → C, the Policy Engine executes them in two separate phases today:
This phase-sequential model was straightforward to implement, but it creates a class of bugs that is hard to detect and hard to fix without understanding the internals.
The Two Personas
Before discussing options, it helps to anchor the problem in two distinct audiences this system serves.
API Designer — applies policies to REST APIs, LLM providers, and MCP endpoints. They think in terms of: "I put A before B, so A runs first." They do not know (or need to know) that internally the engine splits execution into header and body phases. Their mental model is a simple sequential pipeline.
Policy Developer — writes the Go policy implementations. They need to understand exactly when their handler functions are called, what state they can rely on, and what mutations from other policies are visible. They read the SDK contracts and understand the execution model explicitly.
The key design question: should the execution model that the engine actually uses match the API Designer's mental model?
The Bug That Surfaced This
Consider a policy chain where:
Authorizationheader. Validation logic lives inOnRequestBody(because it needs the body to extract the MCP tool call context alongside the token).Authorizationheader with a backend credential (rewrites it inOnRequestHeaders).With the current phase-sequential model:
Step 3 fails or validates the wrong token. The bug is invisible from the API Designer's perspective — they applied Auth before Set Header, which looks correct. The ordering feels right. The engine just doesn't match the mental model.
Option 1: Keep Phase-Sequential, Add Structured Context
Leave the execution order unchanged. Introduce structured
DownstreamandUpstreamsub-contexts that snapshot headers before any policy mutations. MCP Auth reads fromctx.Downstream.Headersinstead of the livectx.Headers.Execution order (unchanged):
Fix for MCP Auth:
Pros:
Cons:
Downstream.Headerstrick for this — B.body hasn't run when C.headers runs. A new workaround field would be needed for every similar case discovered in the future.Option 2: Per-Policy Sequential Execution
Change the execution model so each policy runs completely — headers then body — before the next policy starts.
Request (forward order):
Response (reverse order, unwrapping preserved):
The MCP Auth + Set Header chain now executes:
Correct. No special context fields needed.
Pros:
Cons:
Streaming Mode Consideration
The Policy Engine has two response processing modes:
StreamingResponsePolicy. Required for LLM streaming responses (SSE/chunked).The concern: An LLM provider can respond in either streaming or buffered mode at runtime, depending on the request. The same policy chain could execute under different modes for different requests.
Does this break Option 2's consistency guarantee?
The mechanics differ between modes, but the high-level ordering contract — A runs before B — holds in both. The API Designer never needs to know which mode is active.
For the Policy Developer, the two modes are explicitly documented contracts:
Additionally: any policy that does meaningful body inspection (auth, content moderation, transformation) cannot implement
StreamingResponsePolicy— it needs the full body. Such chains are forced buffered regardless of what the LLM does. Streaming mode only applies to lightweight pass-through chains where the per-chunk model is the correct semantic.Implementation Notes (Option 2)
The ext_proc protocol constraint: Envoy delivers request headers in one callback (
processRequestHeaders) and the request body in a separate callback (processRequestBody). These are two separate gRPC messages. The body is not available during the headers callback.Proposed approach — deferred execution:
processRequestHeaders(): Store the header context. Do not execute any policies yet.processRequestBody(): Run the full per-policy sequential loop:processRequestHeaders().Same pattern applies symmetrically to the response path.
Tradeoff: Header-only short-circuit (rejecting a request based on headers alone, before the body arrives) is deferred until the body arrives in buffered mode. For most security policies, this is acceptable since they need the body anyway. For header-only policies (e.g., API key validation in headers), they do not declare
RequestBodyMode = BodyModeRequired, so the per-policy loop can short-circuit after the headers phase for those policies and skip the body phase.Key files to change:
internal/executor/chain.go— newExecuteRequestPoliciesSequential()andExecuteResponsePoliciesSequential()methodsinternal/kernel/execution_context.go— restructureprocessRequestHeaders/processRequestBody/processResponseHeaders/processResponseBodyorchestrationOpen Questions
Migration: Are there existing policies in the platform that depend on the current phase-sequential behavior (i.e., they expect to see all prior policies' header mutations before their body runs)? A sweep of existing policy implementations is needed before merging.
Header-only short-circuit in buffered mode: With deferred execution (Option 2), a malformed request body must be received before a header-based rejection fires. Is this acceptable for all header-only policies (e.g., API key, rate limit by header)?
Streaming mode contract documentation: The per-chunk model for streaming is a distinct contract from per-policy-complete for buffered. Should the SDK surface this distinction explicitly, or document it as an implementation detail that Policy Developers encounter only when implementing
StreamingPolicy?Response reverse order: The current reverse-order response execution (C → B → A) is an intentional unwrapping pattern. Does per-policy sequential for responses (C.(headers→body) → B.(headers→body) → A.(headers→body)) preserve all cases where this unwrapping behavior is relied upon?
Testing surface: The new execution path needs integration tests covering: (a) header mutation visibility across policies, (b) body mutation visibility across policies, (c) short-circuit behavior mid-chain, (d) bodyless request handling, and (e) streaming path unchanged behavior.
Beta Was this translation helpful? Give feedback.
All reactions