Feature request: per-request dynamic headers for Streamable HTTP MCP clients

[ninetyhe-90]

Problem

DeepSeek Harness supports configuring headers for HTTP MCP servers, including through ACP session configuration. This works well for static values, but some headers must be generated from the active invocation context immediately before each outbound MCP HTTP request.

Examples include:

• W3C Trace Context headers such as traceparent and tracestate
• short-lived or rotating credentials
• tenant, routing, and correlation headers
• request-specific policy metadata

A header captured when an MCP transport or session is created can become stale. With reused transports or concurrent sessions, reusing a request-scoped header may also attach a call to the wrong trace or tenant.

Current workaround

We currently pass a fresh traceparent through mcpServers[].headers on session/new or session/resume, and close the turn-scoped runtime afterward. This uses existing DSH APIs and preserves end-to-end trace IDs, but it requires lifecycle churn and cannot select the exact active span for every individual MCP HTTP request.

Proposed capability

Could DSH expose an async per-request header provider or hook for Streamable HTTP MCP clients, for example:

mcpRequestHeaders(context): Promise<Record<string, string>>

Useful context would include:

• DSH session ID
• MCP server name
• tool name, when available
• MCP method
• existing static headers
• an invocation-scoped context or metadata carrier

The returned headers would be merged case-insensitively immediately before the final HTTP request is sent. Static mcpServers[].headers should continue to work, with documented precedence and protection for protocol-managed headers.

Requirements

• safe isolation across concurrent sessions and tool calls
• async providers for token refresh or context lookup
• support for all HTTP requests made by Streamable HTTP transport, including initialization and reconnects
• no effect on stdio transport
• clear merge and overwrite rules for Authorization, MCP protocol headers, and user headers
• failures should fail the request explicitly rather than silently reuse stale headers

Related ecosystem patterns

This pattern exists in other agent/MCP implementations:

• Hermes Agent uses an mcp_request_headers hook for per-request W3C traceparent propagation.
• Google ADK Kotlin exposes a per-request headerProvider.
• IBM ContextForge dynamically injects trace context for outbound MCP calls and documents the risks of pinning request context to pooled transports.

Would the maintainers be open to a request-level hook/provider in the official MCP client? If there is already a preferred Cordis hook or transport extension point for this, guidance would be appreciated.

[argszero]

I looked this up in the pinned SDK and in dsh-mcp-client, because "is there already a preferred Cordis hook?" has a definite answer: there is no hook today — but the SDK half already exists, which makes the harness change small.

Findings, all verified against @modelcontextprotocol/client@2.0.0 (the exact version dsh-mcp-client pins) and the harness at ddefc45fbc.

1. The SDK is already request-scoped. The staleness is entirely on the harness side.

The transport stores the init object once and re-reads it on every outbound request:

• dist/index.cjs:4700 — this._requestInit = opts?.requestInit (construction, once)
• dist/index.cjs:4720 — _commonHeaders() does normalizeHeaders(this._requestInit?.headers) per request
• dist/index.cjs:4837 — _send() builds const init = { ...this._requestInit, method: 'POST', headers, body, signal } per request
• dist/index.cjs:4731 — the GET/SSE stream path (EventSource fetch wrapper in _startOrAuth) also calls await this._commonHeaders() per (re)connect

So both POSTs and the stream re-read the live object every time — exactly the "all HTTP requests, including initialization and reconnects" requirement. An object that is mutated between requests is picked up by the next one, with no SDK change.

What freezes it in dsh is simply that no such object is ever retained:

• packages/mcp/mcp-client/src/transport.ts:40-44 — new StreamableHTTPClientTransport(new URL(config.url), { requestInit: { headers: config.headers } }). The { headers } literal is created at construction and not kept anywhere.
• packages/mcp/mcp-client/src/index.ts:136 — headers: z.dict(String).default({}), a plain string map resolved once from plugin config.

Net effect: MCP headers are load/generation-scoped, not request-scoped. (Your workaround works for the same reason in reverse: createTransport runs once per connection attempt — connection.ts:248 "fresh transport + client", connection.ts:307 — so a new generation does pick up fresh headers. cordis.yml !!js under plugin config is also load-scoped, so it does not help here.)

2. There is no Cordis hook, and the transport is deliberately unreachable.

• ConnectionHandle (packages/mcp/mcp-client/src/connection.ts:103-121) exposes exactly resources, instructions(), ready, dispose(). No transport, client, or init.
• registerServerContext (packages/mcp/mcp-client/src/server-context.ts:28-39) publishes only the resource provider and one system-prompt section.
• createTransport is module-local; the package exports map (. + ./src/*) gets a plugin the function, never the instance.

So a third-party plugin cannot supply fetch, cannot mutate requestInit, and there is no mcp/* waterfall event to intercept. I looked for a plugin-side path first, since that is normally the cheapest route; here the missing piece is genuinely in core. The only plugin-shaped option is reimplementing the whole supervised connection (reconnect policy, tool sync, resource provider, instruction snapshot) to add one header — a bad trade.

3. Your merge/overwrite question has a concrete answer, and it matters for Authorization.

_commonHeaders() (:4716-4730) builds headers SDK-first, user-last:

if (token) headers['Authorization'] = `Bearer ${token}`
if (this._protocolVersion) headers['mcp-protocol-version'] = this._protocolVersion
const extraHeaders = normalizeHeaders(this._requestInit?.headers)
return new Headers({ ...headers, ...extraHeaders })   // user headers win

…but content-type and accept are set afterwards, by the callers:

• _send (:4835): headers.set('content-type', 'application/json')
• stream (:4733): headers.set('Accept', 'text/event-stream')

So today: content-type/accept are SDK-authoritative; Authorization and mcp-protocol-version are user-overridable — a static mcpServers[].headers.Authorization silently overrides the authProvider/OAuth bearer token. If you add a provider, decide explicitly whether computed headers keep that override power or are merged under the protocol-managed names; that is the "clear merge and overwrite rules" item in your requirements.

Second consequence: because the headers are read from the stored object, a provider that throws must fail that request rather than leave the previous value in place — otherwise you get exactly the silent stale-header reuse you want to eliminate.

Minimal change I would propose (all inside dsh-mcp-client, no SDK fork):

1. Retain the init object per connection generation instead of building it inline in createTransport.
2. Add one provider to the streamable-http config, e.g. requestHeaders?: (ctx: { serverName; toolName?; method? }) => Promise<Record<string, string>> | Record<string, string>, and refresh init.headers from it immediately before each request. Per finding 1 the SDK then applies it to POSTs and the stream alike.
3. Fail the request when the provider rejects — never fall back to the previous headers.
4. Stdio needs nothing: createTransport already branches on config.transport.

I would rather not prepare a patch until the config-provider shape is settled, since mcp-client config is a public surface and the naming/precedence rules are yours to pick. Happy to write it against whatever shape you prefer.

[ninetyhe-90]

Thank you for checking the pinned SDK and harness sources. Your explanation also confirms why our current reconnect workaround works.

I omitted an important part of our deployment model in the original post. DSH is not embedded in our web process; we run it as a server-side agent runtime behind our own gateway:

Browser / company website
  -- HTTP traceparent and selected custom headers --> Java gateway
  -- ACP JSON-RPC over stdio --> DSH runtime
  -- Streamable HTTP --> downstream MCP server

The browser headers terminate at our gateway. The next hop is ACP over stdio, so there is no HTTP request header for DSH to inherit. We need a supported way to carry an allowlisted, invocation-scoped context from the ACP caller into DSH, and then apply it to outbound MCP requests. We do not want to forward all browser headers implicitly.

Our current workaround is:

1. Extract and validate traceparent at the gateway.
2. Put it in mcpServers[].headers on session/new or session/resume.
3. Close the DSH turn-scoped runtime after each prompt so the next turn cannot reuse stale headers.

This preserves the trace ID but makes the header connection-generation scoped and forces lifecycle churn.

For our server-side case, the complete upstream capability therefore seems to need two pieces:

1. The request-scoped provider/interceptor inside dsh-mcp-client that you described.
2. A serializable per-prompt/per-invocation ACP input, for example a namespaced session/prompt.params._meta value or another official extension:

{
  "ai.deepseek.dsh/trace-context": {
    "traceparent": "00-...-...-01",
    "tracestate": "..."
  }
}

DSH could bind this data to the prompt/tool execution context, and the in-process request-header provider could read it immediately before each outbound MCP request. A function-valued config alone would help in-process plugins, but an external ACP client cannot serialize or install that function, and it still needs a supported source for the current invocation context.

I agree that no SDK fork is necessary. One concurrency concern with mutating a retained shared requestInit.headers object is overlapping MCP requests. Since @modelcontextprotocol/client@2.0.0 exposes fetch?: FetchLike, a DSH-owned fetch wrapper could await the provider and construct a fresh Headers snapshot for each actual request. That would also make provider failure fail exactly that request without retaining a previous value. If mutation is used instead, the connection would need equivalent per-request serialization/isolation.

For precedence, my preferred default would be:

• SDK-managed protocol headers remain authoritative.
• Dynamic headers override static user headers for ordinary names.
• If an authProvider is configured, conflicting user/provider Authorization should be rejected rather than silently overriding OAuth.
• Provider failure fails the request; no stale fallback.

The Java gateway can provide the parent trace context for the turn. If DSH wants a distinct span for each tool call, that child span must be created inside DSH because the tool is selected and executed there.

Is there already an ACP invocation metadata extension we should use for the first hop? If not, would a namespaced per-prompt _meta field plus a Cordis runtime provider/hook be an acceptable shape?

[ninetyhe-90]

Thank you for checking the pinned SDK and harness sources. Your explanation also confirms why our current reconnect workaround works.

I omitted an important part of our deployment model in the original post. DSH is not embedded in our web process; we run it as a server-side agent runtime behind our own gateway:

Browser / company website
  -- HTTP traceparent and selected custom headers --> Java gateway
  -- ACP JSON-RPC over stdio --> DSH runtime
  -- Streamable HTTP --> downstream MCP server

The browser headers terminate at our gateway. The next hop is ACP over stdio, so there is no HTTP request header for DSH to inherit. We need a supported way to carry an allowlisted, invocation-scoped context from the ACP caller into DSH, and then apply it to outbound MCP requests. We do not want to forward all browser headers implicitly.

Our current workaround is:

1. Extract and validate traceparent at the gateway.
2. Put it in mcpServers[].headers on session/new or session/resume.
3. Close the DSH turn-scoped runtime after each prompt so the next turn cannot reuse stale headers.

This preserves the trace ID but makes the header connection-generation scoped and forces lifecycle churn.

For our server-side case, the complete upstream capability therefore seems to need two pieces:

1. The request-scoped provider/interceptor inside dsh-mcp-client that you described.
2. A serializable per-prompt/per-invocation ACP input, for example a namespaced session/prompt.params._meta value or another official extension:

{
  "ai.deepseek.dsh/trace-context": {
    "traceparent": "00-...-...-01",
    "tracestate": "..."
  }
}

DSH could bind this data to the prompt/tool execution context, and the in-process request-header provider could read it immediately before each outbound MCP request. A function-valued config alone would help in-process plugins, but an external ACP client cannot serialize or install that function, and it still needs a supported source for the current invocation context.

I agree that no SDK fork is necessary. One concurrency concern with mutating a retained shared requestInit.headers object is overlapping MCP requests. Since @modelcontextprotocol/client@2.0.0 exposes fetch?: FetchLike, a DSH-owned fetch wrapper could await the provider and construct a fresh Headers snapshot for each actual request. That would also make provider failure fail exactly that request without retaining a previous value. If mutation is used instead, the connection would need equivalent per-request serialization/isolation.

For precedence, my preferred default would be:

• SDK-managed protocol headers remain authoritative.
• Dynamic headers override static user headers for ordinary names.
• If an authProvider is configured, conflicting user/provider Authorization should be rejected rather than silently overriding OAuth.
• Provider failure fails the request; no stale fallback.

The Java gateway can provide the parent trace context for the turn. If DSH wants a distinct span for each tool call, that child span must be created inside DSH because the tool is selected and executed there.

Is there already an ACP invocation metadata extension we should use for the first hop? If not, would a namespaced per-prompt _meta field plus a Cordis runtime provider/hook be an acceptable shape?

[argszero]

Two answers: the protocol has the slot, dsh has no reader — and the second half of your model is better supported than you assumed, because the injection point you propose already exists in the SDK dsh pins.

First hop — ACP invocation metadata

_meta is real and is exactly the vehicle you are describing. In the SDK dsh pins (@agentclientprotocol/sdk@1.4.0, packages/acp/acp/package.json), PromptRequest carries:

_meta?: { [key: string]: unknown } | null;

with the spec's own comment: "The _meta property is reserved by ACP to allow clients and agents to attach additional metadata to their interactions. Implementations MUST NOT make assumptions about values at these keys." So a namespaced ai.deepseek.dsh/trace-context value on session/prompt is squarely within the protocol.

But dsh never reads it:

• grep -rn "_meta" packages/acp/acp/src/ → 0 hits.
• sessionId is used only for routing (packages/acp/acp/src/index.ts:361-364 looks the session up, then calls record.prompt(params, …)), and the session handler reads exactly one field of the request: params.prompt (packages/acp/acp/src/session.ts:283, feeding createUserMessage({ content, source: { kind: 'user' } })). Everything else on the request is dropped at that line.

There is also no extension-method escape hatch to reach for instead: the bridge registers nine handlers (packages/acp/acp/src/index.ts:379-390 — initialize, authenticate, session/{new,list,resume,close,set_config_option,prompt} as requests, plus session/cancel as a notification), and no ext/* handler. The SDK's ExtRequest/ExtNotification types exist but are opaque (ExtRequest = unknown) and dsh binds none.

So: the protocol offers exactly the carrier you want, and today nothing end-to-end can put a value in it or read one out. That makes "namespaced per-prompt _meta" not a workaround but the intended shape.

Second hop — the outbound provider

Your read of the SDK is correct, and slightly pessimistic in a useful direction: fetch injection already exists, dsh just does not spend it. All of the outbound header story today is one line — packages/mcp/mcp-client/src/transport.ts:41-44:

return new StreamableHTTPClientTransport(
  new URL(config.url),
  { requestInit: { headers: config.headers } },
)

and StreamableHTTPClientTransportOptions (SDK 2.0.0) has:

/** Custom fetch implementation used for all network requests. */
fetch?: FetchLike;   // (url: string | URL, init?: RequestInit) => Promise<Response>

So "a DSH-owned fetch wrapper could await the provider and construct a fresh Headers snapshot per request" needs no SDK fork — it needs one config field and one wiring change. Two mechanical details that decide the shape:

• Per-request headers are already rebuilt fresh, so reading is per-request; it is only the source object that is shared. _commonHeaders() does new Headers({ ...Authorization/session/version, ...normalizeHeaders(this._requestInit?.headers) }), and normalizeHeaders shallow-copies on every call. That means a mutation-based provider does work — and that is precisely the problem you flag: the transport keeps one requestInit object for its lifetime, so overlapping requests read whatever it holds at that instant. The fresh Headers copy does not isolate anything; it copies the shared object's current state.
• The wrapper shape gets a clean hand-off for free. createFetchWithInit(baseFetch, baseInit) returns baseFetch unchanged when there is no baseInit (if (!baseInit) return baseFetch). So if the provider is installed as fetch and static headers stop being passed as requestInit.headers, every actual request arrives at your wrapper with init.headers already merged by the transport — and the wrapper builds a new Headers from it per call. Provider failure = throw inside the wrapper = that one request fails, nothing stale retained. Exactly the isolation property you asked for, without any connection churn.

Precedence — I agree with your four defaults, and #1 has a hard mechanical reason

The transport does not only set mcp-session-id / mcp-protocol-version; it also derives mcp-method (and mcp-name for tools/call / resources/read) from the message body (_applyBodyDerivedHeaders), and the server cross-checks header against body: SEP-2243, error -32020 HEADER_MISMATCH, mapped to HTTP 400 by the SDK's own ladder status table. A provider that overwrites an mcp-* name therefore converts a tracing feature into a hard 400 from a modern-era server. So:

• allowlist must exclude mcp-* (case-insensitively — Headers normalizes names),
• and for your item 3, authorization is worth rejecting rather than overriding for a second reason: when authProvider is configured, the SDK consults authProvider.token() per request inside _commonHeaders() and assigns it into the fresh Headers, so a provider-supplied Authorization would be silently overwritten anyway. Rejecting surfaces the conflict instead of writing a header that never wins.

What cannot be done from a plugin (I checked, since it would be the cheapest contribution)

I looked for a plugin-mountable path and there is none, on either hop — mechanically:

• The transport is constructed inside dsh-mcp-client's own fiber from its own resolved Config (a discriminated union of stdio/streamable-http; no fetch, no authProvider, no header-provider field). There is no plugin-facing hook on that construction, and the HTTP happens inside the MCP client's request path — a tools/pre-execute-style interceptor can rewrite arguments before dispatch but never reaches the wire.
• On the input side the _meta is dropped inside the ACP request handler, and that path emits no Cordis event carrying the raw params, so a plugin cannot even observe the invocation context, let alone bind it.

⇒ both halves are core work. One adjacent piece is plugin-reachable: the per-tool-call child span. tools/pre-execute / tools/post-execute (packages/core/tools/src/index.ts:146/169) bracket each tool execution in-process, so an out-of-tree plugin can start and finish a span around every tool call today. It cannot propagate that span (propagation is the header half), so it would be local-record-only — but if you want per-tool-call telemetry before the header work lands, that route is open now.

Finally, a note on why your workaround has to be connection-scoped: packages/acp/acp/src/mcp.ts:49-70 maps mcpServers[].headers into the per-session mcp-client config (with duplicate-name and absolute-command validation), i.e. headers are fixed when the connection is created. That is the property to change — making the header set request-scoped instead of connection-scoped. Per-prompt _meta plus a fetch-level provider is the minimal two-piece shape for it.