docs - expose MCPs on public internet #20626
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile OverviewGreptile Summary
Confidence Score: 3/5
|
| Filename | Overview |
|---|---|
| docs/my-website/docs/mcp_public_internet.md | Adds a new MCP doc page describing available_on_public_internet filtering based on caller IP; includes walkthrough screenshots and example API/config snippets. Potential doc build/maintenance issues due to heavy use of external images and unescaped angle-bracket placeholders in inline code blocks. |
| docs/my-website/sidebars.js | Adds mcp_public_internet to the /mcp docs sidebar category; change is localized and syntactically consistent with existing sidebar structure. |
Sequence Diagram
sequenceDiagram
participant Ext as External MCP client
participant Int as Internal MCP client
participant Proxy as LiteLLM MCP endpoints
participant IP as IP classifier (mcp_internal_ip_ranges)
participant Store as MCP server store
Ext->>Proxy: GET /mcp (registry / tools/list)
Proxy->>IP: classify(client_ip)
IP-->>Proxy: external
Proxy->>Store: load MCP servers
Store-->>Proxy: servers (public+private)
Proxy-->>Ext: only available_on_public_internet=true
Int->>Proxy: GET /mcp (registry / tools/list)
Proxy->>IP: classify(client_ip)
IP-->>Proxy: internal
Proxy->>Store: load MCP servers
Store-->>Proxy: servers (public+private)
Proxy-->>Int: all servers
Ext->>Proxy: POST /mcp (tools/call)
Proxy->>IP: classify(client_ip)
IP-->>Proxy: external
Proxy-->>Ext: allow only if target server is public
| Open ChatGPT and add a new MCP server. The endpoint to use is: | ||
|
|
||
| ``` | ||
| <your-litellm-url>/mcp | ||
| ``` |
There was a problem hiding this comment.
HTML placeholder may break
The snippet uses <your-litellm-url>/mcp inside a fenced code block. In MD/MDX, angle-bracket placeholders can be interpreted as an (invalid) HTML tag and may get stripped or rendered oddly, so readers may not see the literal placeholder. Prefer a plain-text placeholder without <...> (e.g. YOUR_LITELLM_URL/mcp) or escape the brackets (e.g. <your-litellm-url>/mcp).
| ```bash title="Create a public MCP server" showLineNumbers | ||
| curl -X POST <your-litellm-url>/v1/mcp/server \ | ||
| -H "Authorization: Bearer sk-..." \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{ | ||
| "server_name": "DeepWiki", | ||
| "url": "https://mcp.deepwiki.com/mcp", | ||
| "transport": "http", | ||
| "available_on_public_internet": true | ||
| }' | ||
| ``` | ||
|
|
||
| ```bash title="Update an existing server" showLineNumbers | ||
| curl -X PUT <your-litellm-url>/v1/mcp/server \ | ||
| -H "Authorization: Bearer sk-..." \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{ | ||
| "server_id": "<server-id>", | ||
| "available_on_public_internet": false | ||
| }' | ||
| ``` |
There was a problem hiding this comment.
Docs example endpoints inconsistent
The API examples use /v1/mcp/server for both create and update. If the actual REST endpoints are /v1/mcp/servers (plural) or a different path (common pattern elsewhere in the docs), these snippets will mislead users and cause copy/paste failures. Please confirm the exact MCP server CRUD endpoints used by the proxy and align the docs accordingly (including whether updates are PUT /v1/mcp/server/{id} vs body server_id).
|  | ||
|
|
||
| Enter the server details — name it "DeepWiki" and set the URL to `https://mcp.deepwiki.com/mcp`. | ||
|
|
||
|  | ||
|
|
||
| Select **HTTP** as the transport type. | ||
|
|
||
|  | ||
|
|
||
|  | ||
|
|
||
| Fill in the MCP Server URL. | ||
|
|
||
|  | ||
|
|
||
|  | ||
|
|
||
| #### Step 2: Enable "Available on Public Internet" | ||
|
|
||
| Expand **Permission Management / Access Control** and toggle **"Available on Public Internet"** on. This ensures external callers (like ChatGPT) can discover this server. | ||
|
|
||
|  | ||
|
|
||
|  | ||
|
|
||
| Click **"Create"** to save the server. | ||
|
|
||
|  | ||
|
|
||
| The server is now created and visible in the table with a **"Public"** badge under Network Access. | ||
|
|
||
|  |
There was a problem hiding this comment.
External screenshots are brittle
This page embeds many external colony-recorder.s3.amazonaws.com images. These links are likely to expire or become inaccessible, which will break the docs page rendering/UX. Consider moving key screenshots into the repo under docs/my-website/static/img/ (or switching to fewer, representative images) so the docs remain stable.
…ng format for Claude Code (#20631) * Add http support to custom code guardrails + Unified guardrails for MCP + Agent guardrail support (#20619) * fix: fix styling * fix(custom_code_guardrail.py): add http support for custom code guardrails allows users to call external guardrails on litellm with minimal code changes (no custom handlers) Test guardrail integrations more easily * feat(a2a/): add guardrails for agent interactions allows the same guardrails for llm's to be applied to agents as well * fix(a2a/): support passing guardrails to a2a from the UI * style(code-editor): allow editing custom code guardrails on ui + add examples of pre/post calls for custom code guardrails * feat(mcp/): support custom code guardrails for mcp calls allows custom code guardrails to work on mcp input * feat(chatui.tsx): support guardrails on mcp tool calls on playground * fix(mypy): resolve missing return statements and type casting issues (#20618) * fix(mypy): resolve missing return statements and type casting issues * fix(pangea): use elif to prevent UnboundLocalError and handle None messages Address Greptile review feedback: - Make branches mutually exclusive using elif to prevent input_messages from being overwritten - Handle case where data.get('messages') returns None to avoid passing invalid payload to Pangea API --------- Co-authored-by: Shin <shin@openclaw.ai> * [Feat] MCP Gateway - Allow setting MCP Servers as Private/Public available on Internet (#20607) * update MCPAuthenticatedUser * add available_on_public_internet for MCPs * update claude.md * init IPAddressUtils * init available_on_public_internet * add on REST endpoints * filter with IP * TestIsInternalIp * _extract_mcp_headers_from_request * init get_mcp_client_ip * _get_general_settings * allowed_server_ids * address PR comments * get_mcp_server_by_name fix * fix server * fix review comments * get_public_mcp_servers * address _get_allowed_mcp_servers * fixing user_id * [Feat] IP-Based Access Control for MCP Servers (#20620) * update MCPAuthenticatedUser * add available_on_public_internet for MCPs * update claude.md * init IPAddressUtils * init available_on_public_internet * add on REST endpoints * filter with IP * TestIsInternalIp * _extract_mcp_headers_from_request * init get_mcp_client_ip * _get_general_settings * allowed_server_ids * address PR comments * get_mcp_server_by_name fix * fix server * fix review comments * get_public_mcp_servers * address _get_allowed_mcp_servers * test fix * fix linting * inint ui types * add ui for managing MCP private/public * add ui * fixes * add to schema * add types * fix endpoint * add endpoint * update manager * test mcp * dont use external party for ip address * Add OpenAI/Azure release test suite with HTTP client lifecycle regression detection (#20622) * docs (#20626) * docs * fix(mypy): resolve type checking errors in 5 files (#20627) - a2a_protocol/exception_mapping_utils.py: Fix type ignore comment for None assignment - caching/redis_cache.py: Add type ignore for async ping return type - caching/redis_cluster_cache.py: Add type ignore for async ping return type - llms/deprecated_providers/palm.py: Add type ignore for palm.generate_text - proxy/auth/handle_jwt.py: Add type ignore for jwt.decode options argument All changes add appropriate type: ignore comments to handle library typing inconsistencies. * fix(test): update deprecated gemini embedding model (#20621) Replace text-embedding-004 with gemini-embedding-001. The old model was deprecated and returns 404: 'models/text-embedding-004 is not found for API version v1beta' Co-authored-by: Shin <shin@openclaw.ai> * ui new buil * fix(websearch_interception): convert agentic loop response to streaming format when original request was streaming Fixes #20187 - When using websearch_interception in Bedrock with Claude Code: 1. Output tokens were showing as 0 because the agentic loop response wasn't being converted back to streaming format 2. The response from the agentic loop (follow-up request) was returned as a non-streaming dict, but Claude Code expects a streaming response This fix adds streaming format conversion for the agentic loop response when the original request was streaming (detected via the websearch_interception_converted_stream flag in logging_obj). The fix ensures: - Output tokens are correctly included in the message_delta event - stop_reason is properly preserved - The response format matches what Claude Code expects --------- Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Shin <shin@openclaw.ai> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com> Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
…iohttp tracing (#20630) * Add http support to custom code guardrails + Unified guardrails for MCP + Agent guardrail support (#20619) * fix: fix styling * fix(custom_code_guardrail.py): add http support for custom code guardrails allows users to call external guardrails on litellm with minimal code changes (no custom handlers) Test guardrail integrations more easily * feat(a2a/): add guardrails for agent interactions allows the same guardrails for llm's to be applied to agents as well * fix(a2a/): support passing guardrails to a2a from the UI * style(code-editor): allow editing custom code guardrails on ui + add examples of pre/post calls for custom code guardrails * feat(mcp/): support custom code guardrails for mcp calls allows custom code guardrails to work on mcp input * feat(chatui.tsx): support guardrails on mcp tool calls on playground * fix(mypy): resolve missing return statements and type casting issues (#20618) * fix(mypy): resolve missing return statements and type casting issues * fix(pangea): use elif to prevent UnboundLocalError and handle None messages Address Greptile review feedback: - Make branches mutually exclusive using elif to prevent input_messages from being overwritten - Handle case where data.get('messages') returns None to avoid passing invalid payload to Pangea API --------- Co-authored-by: Shin <shin@openclaw.ai> * [Feat] MCP Gateway - Allow setting MCP Servers as Private/Public available on Internet (#20607) * update MCPAuthenticatedUser * add available_on_public_internet for MCPs * update claude.md * init IPAddressUtils * init available_on_public_internet * add on REST endpoints * filter with IP * TestIsInternalIp * _extract_mcp_headers_from_request * init get_mcp_client_ip * _get_general_settings * allowed_server_ids * address PR comments * get_mcp_server_by_name fix * fix server * fix review comments * get_public_mcp_servers * address _get_allowed_mcp_servers * fixing user_id * [Feat] IP-Based Access Control for MCP Servers (#20620) * update MCPAuthenticatedUser * add available_on_public_internet for MCPs * update claude.md * init IPAddressUtils * init available_on_public_internet * add on REST endpoints * filter with IP * TestIsInternalIp * _extract_mcp_headers_from_request * init get_mcp_client_ip * _get_general_settings * allowed_server_ids * address PR comments * get_mcp_server_by_name fix * fix server * fix review comments * get_public_mcp_servers * address _get_allowed_mcp_servers * test fix * fix linting * inint ui types * add ui for managing MCP private/public * add ui * fixes * add to schema * add types * fix endpoint * add endpoint * update manager * test mcp * dont use external party for ip address * Add OpenAI/Azure release test suite with HTTP client lifecycle regression detection (#20622) * docs (#20626) * docs * fix(mypy): resolve type checking errors in 5 files (#20627) - a2a_protocol/exception_mapping_utils.py: Fix type ignore comment for None assignment - caching/redis_cache.py: Add type ignore for async ping return type - caching/redis_cluster_cache.py: Add type ignore for async ping return type - llms/deprecated_providers/palm.py: Add type ignore for palm.generate_text - proxy/auth/handle_jwt.py: Add type ignore for jwt.decode options argument All changes add appropriate type: ignore comments to handle library typing inconsistencies. * fix(test): update deprecated gemini embedding model (#20621) Replace text-embedding-004 with gemini-embedding-001. The old model was deprecated and returns 404: 'models/text-embedding-004 is not found for API version v1beta' Co-authored-by: Shin <shin@openclaw.ai> * ui new buil * fix(http_handler): bypass cache when shared_session is provided for aiohttp tracing When users pass a shared_session with trace_configs to acompletion(), the get_async_httpx_client() function was ignoring it and returning a cached client without the user's tracing configuration. This fix bypasses the cache when shared_session is provided, ensuring the user's ClientSession (with its trace_configs, connector settings, etc.) is actually used for the request. Fixes #20174 --------- Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Shin <shin@openclaw.ai> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com> Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com> Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
docs - expose MCPs on public internet
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes