feat: add support for using MCPs on /chat/completions #17747

uc4w6c · 2025-12-10T01:58:26Z

Title

feat: add support for using MCPs on /chat/completions

Relevant issues

LIT-1537

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
I have added a screenshot of my new test passing locally
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🆕 New Feature
🐛 Bug Fix
📖 Documentation
✅ Test

Changes

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-xxxxx" \
  -d '{
    "model": "gpt-4o",
    "messages": [
        {
            "role": "user",
            "content": "add 1 + 1"
        }
    ],
    "tools": [
      {
        "type": "mcp",
        "server_label": "litellm",
        "server_url": "litellm_proxy/mcp/everything",
        "require_approval": "never",
        "headers": {
          "x-litellm-api-key": "Bearer sk-xxxxx"
        }
      }
    ]
  }'

{"id":"chatcmpl-Cl3kfjQkTB3lWV4GvWaUSkbYbBnMK","created":1765331873,"model":"gpt-4o-2024-08-06","object":"chat.completion","system_fingerprint":"fp_83554c687e","choices":[{"finish_reason":"stop","index":0,"message":{"content":"The sum of 1 and 1 is 2.","role":"assistant"}}],"usage":{"completion_tokens":13,"prompt_tokens":482,"total_tokens":495,"completion_tokens_details":{"accepted_prediction_tokens":0,"audio_tokens":0,"reasoning_tokens":0,"rejected_prediction_tokens":0},"prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0}},"service_tier":"default"}

vercel · 2025-12-10T01:58:30Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
litellm	Ready	Preview	Comment	Dec 11, 2025 9:43pm

ishaan-jaff

reviewed

ishaan-jaff · 2025-12-10T01:59:43Z

litellm/main.py

        raise Exception("Mock completion response failed - {}".format(e))


+async def _call_acompletion_internal(


pelase don't add this to main.py we don'y need any more code here

Good point — I’ve updated the PR.

ishaan-jaff · 2025-12-10T02:00:14Z

litellm/main.py

    tool_choice = validate_chat_completion_tool_choice(tool_choice=tool_choice)
+
+    skip_mcp_handler = kwargs.pop("_skip_mcp_handler", False)
+    if not skip_mcp_handler:


if possible pelase can we avoid adding more code in main.py, new code can be placed in a diff file

ishaan-jaff · 2025-12-10T02:00:39Z

docs/my-website/docs/mcp.md

 </TabItem>

+#### Use MCP tools with `/chat/completions`
+


can you make it clear this works across ALL llm providers

I’ve updated the PR — could you take a look and let me know if it matches what you had in mind?

uc4w6c · 2025-12-11T21:26:42Z

litellm/responses/mcp/litellm_proxy_mcp_handler.py

                    tool_server_map[tool_name] = allowed_mcp_servers[0]
                else:
-                    tool_server_map[tool_name], _ = split_server_prefix_from_name(
+                    _, tool_server_map[tool_name] = split_server_prefix_from_name(


The tuple unpacking order here is reversed.
Because of that, tool_server_map ends up using the tool name as both the key and the value,
instead of mapping tool_name -> server_name.
This change fixes that issue.

ishaan-jaff

ishaan-jaff · 2025-12-11T21:28:32Z

litellm/main.py

    tools = validate_and_fix_openai_tools(tools=tools)
    # validate tool_choice
    tool_choice = validate_chat_completion_tool_choice(tool_choice=tool_choice)
+


does this need to be in main.py ?

…n.py

feat: add support for using MCPs on /chat/completions

ed5cbda

ishaan-jaff requested changes Dec 10, 2025

View reviewed changes

uc4w6c added 2 commits December 11, 2025 10:39

fix: separate MCP handling out of main.py

6393277

docs: clarify MCP tool support across providers

4efa21e

vercel bot deployed to Preview December 11, 2025 01:41 View deployment

uc4w6c commented Dec 11, 2025

View reviewed changes

ishaan-jaff requested changes Dec 11, 2025

View reviewed changes

fix: update MCP handler invocation to pass full call context from mai…

8cccf3e

…n.py

vercel bot deployed to Preview December 11, 2025 21:43 View deployment

uc4w6c merged commit 8899b63 into main Dec 12, 2025
47 of 59 checks passed

		raise Exception("Mock completion response failed - {}".format(e))


		async def _call_acompletion_internal(

Uh oh!

feat: add support for using MCPs on /chat/completions #17747

feat: add support for using MCPs on /chat/completions #17747

Uh oh!

Conversation

uc4w6c commented Dec 10, 2025

Title

Relevant issues

Pre-Submission checklist

Type

Changes

Uh oh!

vercel bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ishaan-jaff left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ishaan-jaff left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vercel bot commented Dec 10, 2025 •

edited

Loading