[Feat] MCP expose streamable https endpoint for LiteLLM Proxy #11645
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull Request Overview
This PR exposes a streamable HTTPS endpoint for the LiteLLM Proxy MCP server. Key changes include replacing the MCP router with an app mount using a new BASE_MCP_ROUTE constant, implementing a health check endpoint, and refactoring MCP server initialization to use StreamableHTTPSessionManager for both HTTP and SSE requests.
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| litellm/proxy/proxy_server.py | Updated to import BASE_MCP_ROUTE and mount the new MCP app instead of including a router. |
| litellm/proxy/_experimental/mcp_server/server.py | Introduced session manager initialization/shutdown logic and new endpoints for streamable HTTP and SSE. |
| litellm/constants.py | Added the BASE_MCP_ROUTE constant to support MCP app mounting. |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
wagnerjt
approved these changes
Jun 12, 2025
wagnerjt
reviewed
Jun 12, 2025
X4tar
pushed a commit
to X4tar/litellm
that referenced
this pull request
Jun 17, 2025
…I#11645) * feat - add https mcp support * fixes for MCP http integration * fix code QA * bump mcp dep * test_mcp_server_manager_https_server * test mcp server https * fix linting error * bump mcp in poetry * fix import streamablehttp_client * fix streamablehttp_client * fix streamablehttp_client * add streamablehttp_client * add simple https server * working mounted app * working HTTPS mcp streamable * fix code QA check * feat: add MCP Server * fix - init just as fastapi app * add LITELLM_MCP_SERVER_DESCRIPTION * fix importing / init litellm app * Update litellm/proxy/_experimental/mcp_server/server.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update litellm/proxy/_experimental/mcp_server/server.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update server.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fixes based on review + code check * fix linting * test_streamable_http_mcp_handler_mock * fix python 3.13 install * fix deps test --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[Feat] MCP expose streamable https endpoint for LiteLLM Proxy
Closes #11603
This PR exposes a streamable HTTPS endpoint for the LiteLLM Proxy MCP serve
h/t @wagnerjt for the initial scoping work and help on this
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
🆕 New Feature
🚄 Infrastructure
✅ Test
Changes