Check for existing issues
The Feature
The MCP spec (2025-06-18, extended in 2025-11-25 which LiteLLM targets since v1.80.18) defines two server→client capabilities that the MCP Gateway does not currently support:
Today, the gateway only proxies client→server messages. When an upstream MCP server sends elicitation/create or sampling/createMessage, the message has no path back to the client and is silently dropped.
Impact differs per MCP mode
Mode A — Transparent proxy (/mcp/, /{server}/mcp — Cursor, Claude Code, FastMCP clients):
Both capabilities need bidirectional JSON-RPC relay over the existing Streamable HTTP session. This requires propagating client-declared capabilities upstream during initialize, and routing server→client messages back through the SSE channel.
Mode B — Tool bridge (/v1/chat/completions, /v1/responses with "type": "mcp" tools):
LiteLLM is the MCP client — there's no downstream MCP client to relay to.
- Sampling is a natural fit here: LiteLLM could handle
sampling/createMessage internally using its own LLM routing. It already manages 100+ providers with cost tracking and rate limiting — this is arguably the ideal position to fulfill sampling requests.
- Elicitation is harder in this mode since there's no user-facing MCP client to present a form to. Could be deferred or handled as a clear error rather than a silent drop.
Motivation, pitch
Elicitation (Mode A): We run MCP servers behind LiteLLM that use elicitation for interactive workflows (user confirmation, disambiguation, missing parameters). Deploying them behind the gateway for centralized auth and access control silently breaks elicitation. URL-mode elicitation is especially relevant for enterprise use — it lets MCP servers initiate third-party OAuth flows without credentials transiting through the client.
Sampling (Mode B): MCP servers with agentic behaviors (multi-step triage, analysis) use sampling for intermediate LLM reasoning. LiteLLM handling this internally via its own model routing would enable a powerful pattern: agentic MCP servers that work through LiteLLM without needing their own API keys.
A phased approach could make sense: sampling in Mode B first (most natural fit), then bidirectional relay in Mode A, then elicitation in Mode B if needed.
Related
What part of LiteLLM is this about?
Proxy
LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?
No
Twitter / LinkedIn details
No response
Check for existing issues
The Feature
The MCP spec (2025-06-18, extended in 2025-11-25 which LiteLLM targets since v1.80.18) defines two server→client capabilities that the MCP Gateway does not currently support:
elicitation/create. The 2025-11-25 spec adds URL-mode for secure out-of-band OAuth flows with third-party services.sampling/createMessage, enabling agentic server-side behaviors without server-side API keys.Today, the gateway only proxies client→server messages. When an upstream MCP server sends
elicitation/createorsampling/createMessage, the message has no path back to the client and is silently dropped.Impact differs per MCP mode
Mode A — Transparent proxy (
/mcp/,/{server}/mcp— Cursor, Claude Code, FastMCP clients):Both capabilities need bidirectional JSON-RPC relay over the existing Streamable HTTP session. This requires propagating client-declared capabilities upstream during
initialize, and routing server→client messages back through the SSE channel.Mode B — Tool bridge (
/v1/chat/completions,/v1/responseswith"type": "mcp"tools):LiteLLM is the MCP client — there's no downstream MCP client to relay to.
sampling/createMessageinternally using its own LLM routing. It already manages 100+ providers with cost tracking and rate limiting — this is arguably the ideal position to fulfill sampling requests.Motivation, pitch
Elicitation (Mode A): We run MCP servers behind LiteLLM that use elicitation for interactive workflows (user confirmation, disambiguation, missing parameters). Deploying them behind the gateway for centralized auth and access control silently breaks elicitation. URL-mode elicitation is especially relevant for enterprise use — it lets MCP servers initiate third-party OAuth flows without credentials transiting through the client.
Sampling (Mode B): MCP servers with agentic behaviors (multi-step triage, analysis) use sampling for intermediate LLM reasoning. LiteLLM handling this internally via its own model routing would enable a powerful pattern: agentic MCP servers that work through LiteLLM without needing their own API keys.
A phased approach could make sense: sampling in Mode B first (most natural fit), then bidirectional relay in Mode A, then elicitation in Mode B if needed.
Related
What part of LiteLLM is this about?
Proxy
LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?
No
Twitter / LinkedIn details
No response