Check for existing issues
What happened?
Summary
When a non-bridge OAuth2 MCP server is registered through LiteLLM's MCP OAuth relay against an upstream that supports real Dynamic Client Registration (for example a pass-through server pointed at Atlassian's Rovo MCP server), the client-facing /register response returns the redirect_uris field
exactly as the upstream registration server echoed it back to LiteLLM, which is LiteLLM's own /callback URL. A spec-compliant OAuth 2.1 DCR client (in our case Open WebUI, connecting over Streamable HTTP) adopts that value for its own subsequent /authorize calls, so LiteLLM's /callback ends up
redirecting back to itself. The second hit lands on /callback carrying the client's own opaque state nonce rather than a LiteLLM relay-state handle or encrypted blob, fails to decrypt, and produces an error that reads exactly like the unrelated LIT-4197 "oauth_state ... Incorrect padding" failure
fixed in #32146, which is what first led us to suspect that already-fixed bug had resurfaced on v1.93.0-rc.2
Environment
LiteLLM image ghcr.io/berriai/litellm-database:v1.93.0-rc.2, deployed on Kubernetes, proxying an MCP server registered against an upstream OAuth2/DCR-capable MCP server (Atlassian Rovo MCP, https://mcp.atlassian.com/v1/mcp/authv2). MCP client is Open WebUI's native External Tools over Streamable HTTP
with Auth mode OAuth 2.1 (DCR)
Root cause
In register_client_with_server (litellm/proxy/_experimental/mcp_server/discoverable_endpoints.py), for a plain (non dcr_bridge) OAuth2 server, the relay always submits LiteLLM's own callback as redirect_uris when registering with the upstream:
```python
current_redirect_uri = f"{request_base_url}/callback"
...
register_data = {
"client_name": client_name,
"redirect_uris": client_redirect_uris if bridge_relay else [current_redirect_uri],
...
}
response = await async_client.post(mcp_server.registration_url, ..., json=register_data)
...
token_response = response.json()
...
return JSONResponse(token_response)
```
That's correct for the LiteLLM <-> upstream leg, since the upstream does need to send its authorization code to LiteLLM's own /callback. The bug is the last line: the upstream's registration response, redirect_uris field included, is returned unmodified to the calling MCP client. RFC 7591 has the
client use the redirect_uri(s) confirmed at registration for subsequent authorization requests, so the client, correctly per spec, starts sending redirect_uri=<litellm-host>/callback on its own /authorize calls, exactly as captured in the access logs below
/authorize's trusted-redirect check (validate_trusted_redirect_uri) allows this because it is same-origin to LiteLLM itself, so nothing rejects it before the loop plays out. When /callback completes the real upstream leg, it redirects the browser to the client_redirect_uri sealed in the encrypted
state, which is now LiteLLM's own /callback. That second hit carries the client's own state value instead of a relay-state handle, no per-flow cookie exists for it, _resolve_encoded_oauth_state falls back to treating that raw value as the encrypted blob, and decrypt_value_helper's
base64.urlsafe_b64decode throws Incorrect padding on a value that was never base64 to begin with
This is unrelated to LIT-4197 / #32146, the short relay-state-in-a-cookie fix, which is working correctly here: the first /callback hit succeeds with a clean redirect. This is a separate defect that happens to surface an identical-looking decrypt error on the second, self-inflicted hit
Evidence (scrubbed access/error logs)
```
GET /.well-known/oauth-authorization-server/atlassian -> 200
GET /atlassian/authorize?response_type=code&client_id=&redirect_uri=https://litellm.example.com/callback&state=<state_a>&resource=https://litellm.example.com/atlassian/mcp&code_challenge=<challenge_a>&code_challenge_method=S256 -> 307
GET /atlassian/authorize?response_type=code&client_id=&redirect_uri=https%3A%2F%2Flitellm.example.com%2Fcallback&state=<state_b>&resource=https%3A%2F%2Flitellm.example.com%2Fatlassian%2Fmcp&code_challenge=<challenge_b>&code_challenge_method=S256 -> 307
...
GET /callback?state=<litellm_relay_state>&code=<upstream_code> -> 302
GET /callback?code=<upstream_code>&state=<state_b> -> 200
ERROR Error decrypting value for key: oauth_state, Did your master_key/salt key change recently?
Error: Incorrect padding
File ".../litellm/proxy/common_utils/encrypt_decrypt_utils.py", line 137, in decrypt_value_helper
decoded_b64 = base64.urlsafe_b64decode(value)
binascii.Error: Incorrect padding
Upstream auth failure from MCP server atlassian: HTTP 401
```
<state_b> in the last two log lines is the exact same value the client originally sent as its own state on the second /authorize call, not a LiteLLM relay-state handle, which confirms the self-redirect
Suggested fix
Stop forwarding the upstream registration response's redirect_uris to the calling client verbatim on the non-bridge relay path. Either omit redirect_uris from the client-facing response, or report back whatever the calling client itself supplied at registration time. The dcr_bridge relay arm already
threads a client_redirect_uris value through for this same reason and could likely be generalized to cover this path too
Also observed, not yet root-caused
Two independent /authorize calls, each with a distinct client state and PKCE code_challenge, land roughly 60ms apart for what should be a single user-initiated connection attempt. Only one of the two completes end to end. This didn't cause the failure above, since each flow is self-contained, but is
worth a separate look on the client side
Steps to Reproduce
- Register an MCP server in LiteLLM pointed at an upstream that supports OAuth 2.1 with real Dynamic Client Registration and no admin-configured static client_id/client_secret, such as Atlassian's Rovo MCP server
- Connect to that server from an OAuth 2.1 DCR-compliant MCP client that performs
/register before /authorize and reuses the registration response's redirect_uris on /authorize (Open WebUI's native MCP External Tools with Auth: OAuth 2.1 (DCR) does this)
- Complete the browser consent
Expected: the browser lands on the MCP client's own redirect endpoint with the final code/state, and the client completes its token exchange
Actual: the browser bounces to LiteLLM's own /callback a second time, which logs a decrypt error and returns "Authentication incomplete," the client never receives a valid code, and the subsequent tool call to the MCP server returns 401
Relevant log output
{"message": "<client-ip>:49014 - \"GET /health/liveliness HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:34.402853", "component": "uvicorn.access", "logger": "h11_impl.py:473"}
{"message": "<client-ip>:36624 - \"GET /health/readiness HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:42.281727", "component": "uvicorn.access", "logger": "h11_impl.py:473"}
{"message": "<internal-ip>:36942 - \"GET /v1/models HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:46.642651", "component": "uvicorn.access", "logger": "h11_impl.py:473"}
{"message": "Synced 0 production policies and 0 draft/published (by ID) from DB to in-memory registry", "level": "INFO", "timestamp": "2026-07-17T13:42:46.917745", "component": "LiteLLM Proxy", "logger": "policy_registry.py:548"}
{"message": "Synced 0 attachments from DB to in-memory registry", "level": "INFO", "timestamp": "2026-07-17T13:42:46.918978", "component": "LiteLLM Proxy", "logger": "attachment_registry.py:453"}
{"message": "Found 3 MCP servers in database", "level": "INFO", "timestamp": "2026-07-17T13:42:46.926676", "component": "LiteLLM", "logger": "mcp_server_manager.py:4454"}
{"message": "Loading 0 search tool(s) into router (0 from config, 0 from database)", "level": "INFO", "timestamp": "2026-07-17T13:42:46.933641", "component": "LiteLLM Proxy", "logger": "proxy_server.py:6561"}
{"message": "ToolPolicyRegistry: synced 195 tool policies and 1 object permissions from DB", "level": "INFO", "timestamp": "2026-07-17T13:42:46.942392", "component": "LiteLLM Proxy", "logger": "tool_registry_writer.py:328"}
{"message": "Lazy-loaded optional feature 'mcp_byok_oauth' (module: litellm.proxy._experimental.mcp_server.byok_oauth_endpoints)", "level": "INFO", "timestamp": "2026-07-17T13:42:52.596276", "component": "LiteLLM Proxy", "logger": "_lazy_features.py:329"}
{"message": "<client-ip>:0 - \"GET /.well-known/oauth-authorization-server/atlassian HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:52.598443", "component": "uvicorn.access", "logger": "h11_impl.py:473"}
{"message": "<client-ip>:0 - \"GET /atlassian/authorize?response_type=code&client_id=<dcr_client_id>&redirect_uri=https://litellm.example.com/callback&state=<state_a>&resource=https://litellm.example.com/atlassian/mcp&code_challenge=<code_challenge_a>&code_challenge_method=S256 HTTP/1.1\" 307", "level": "INFO", "timestamp": "2026-07-17T13:42:52.654147", "component": "uvicorn.access", "logger": "h11_impl.py:473"}
{"message": "<client-ip>:0 - \"GET /atlassian/authorize?response_type=code&client_id=<dcr_client_id>&redirect_uri=https%3A%2F%2Flitellm.example.com%2Fcallback&state=<state_b>&resource=https%3A%2F%2Flitellm.example.com%2Fatlassian%2Fmcp&code_challenge=<code_challenge_b>&code_challenge_method=S256 HTTP/1.1\" 307", "level": "INFO", "timestamp": "2026-07-17T13:42:52.717774", "component": "uvicorn.access", "logger": "h11_impl.py:473"}
{"message": "<client-ip>:0 - \"GET /v1/mcp/server/health HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:53.286606", "component": "uvicorn.access", "logger": "h11_impl.py:473"}
{"message": "_get_tools_from_server for atlassian...", "level": "INFO", "timestamp": "2026-07-17T13:42:54.085233", "component": "LiteLLM", "logger": "mcp_server_manager.py:2528"}
{"message": "MCP client listed 36 tools from https://mcp.atlassian.com/v1/mcp/authv2: ['atlassianUserInfo', 'getAccessibleAtlassianResources', 'getConfluencePage', 'searchConfluenceUsingCql', 'getConfluenceSpaces', 'getPagesInConfluenceSpace', 'getConfluencePageFooterComments', 'getConfluencePageInlineComments', 'getConfluenceCommentChildren', 'getConfluencePageDescendants', 'getJiraIssue', 'editJiraIssue', 'createJiraIssue', 'getTransitionsForJiraIssue', 'getJiraIssueRemoteIssueLinks', 'getVisibleJiraProjects', 'getJiraProjectIssueTypesMetadata', 'getJiraIssueTypeMetaWithFields', 'addCommentToJiraIssue', 'transitionJiraIssue', 'searchJiraIssuesUsingJql', 'lookupJiraAccountId', 'addWorklogToJiraIssue', 'getIssueLinkTypes', 'createIssueLink', 'getCompassComponents', 'getCompassComponent', 'getCompassCustomFieldDefinitions', 'createCompassCustomFieldDefinition', 'createCompassComponent', 'createCompassComponentRelationship', 'getTeamworkGraphContext', 'getTeamworkGraphObject', 'addTeamworkGraphContext', 'search', 'fetch']", "level": "INFO", "timestamp": "2026-07-17T13:42:54.518506", "component": "LiteLLM", "logger": "client.py:507"}
{"message": "Successfully fetched 36 tools from server atlassian", "level": "INFO", "timestamp": "2026-07-17T13:42:54.523893", "component": "LiteLLM", "logger": "mcp_server_manager.py:3385"}
{"message": "<client-ip>:0 - \"GET /mcp-rest/tools/list?server_id=<server_id>&include_disabled_tools=true HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:54.524734", "component": "uvicorn.access", "logger": "h11_impl.py:473"}
{"message": "<client-ip>:52526 - \"GET /health/readiness HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:57.286062", "component": "uvicorn.access", "logger": "h11_impl.py:473"}
{"message": "<client-ip>:0 - \"GET /callback?state=<litellm_relay_state>&code=<upstream_authorization_code_jwt> HTTP/1.1\" 302", "level": "INFO", "timestamp": "2026-07-17T13:42:57.963309", "component": "uvicorn.access", "logger": "h11_impl.py:473"}
{"message": "<client-ip>:0 - \"GET /callback?code=<upstream_authorization_code_jwt>&state=<state_b> HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:57.992837", "component": "uvicorn.access", "logger": "h11_impl.py:473"}
{"message": "Error decrypting value for key: oauth_state, Did your master_key/salt key change recently? \nError: Incorrect padding\nSet permanent salt key - https://docs.litellm.ai/docs/proxy/prod#5-set-litellm-salt-key", "level": "ERROR", "timestamp": "2026-07-17T13:42:57.990389", "component": "LiteLLM Proxy", "logger": "encrypt_decrypt_utils.py:157", "stacktrace": "Traceback (most recent call last):\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/common_utils/encrypt_decrypt_utils.py\", line 137, in decrypt_value_helper\n decoded_b64 = base64.urlsafe_b64decode(value)\n File \"/usr/lib/python3.13/base64.py\", line 134, in urlsafe_b64decode\n return b64decode(s)\n File \"/usr/lib/python3.13/base64.py\", line 88, in b64decode\n return binascii.a2b_base64(s, strict_mode=validate)\n ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^\nbinascii.Error: Incorrect padding\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/common_utils/encrypt_decrypt_utils.py\", line 140, in decrypt_value_helper\n decoded_b64 = base64.b64decode(value)\n File \"/usr/lib/python3.13/base64.py\", line 88, in b64decode\n return binascii.a2b_base64(s, strict_mode=validate)\n ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^\nbinascii.Error: Incorrect padding"}
{"message": "MCP client list_tools was cancelled", "level": "WARNING", "timestamp": "2026-07-17T13:43:03.482089", "component": "LiteLLM", "logger": "client.py:510"}
{"message": "Timeout while listing tools from grafana", "level": "WARNING", "timestamp": "2026-07-17T13:43:03.482595", "component": "LiteLLM", "logger": "mcp_server_manager.py:3268"}
{"message": "Successfully fetched 0 tools from server grafana", "level": "INFO", "timestamp": "2026-07-17T13:43:03.482889", "component": "LiteLLM", "logger": "mcp_server_manager.py:3385"}
{"message": "_get_tools_from_server for atlassian...", "level": "INFO", "timestamp": "2026-07-17T13:43:03.483293", "component": "LiteLLM", "logger": "mcp_server_manager.py:2528"}
{"message": "Upstream auth failure from MCP server atlassian: HTTP 401", "level": "INFO", "timestamp": "2026-07-17T13:43:03.514902", "component": "LiteLLM", "logger": "mcp_server_manager.py:3280"}
{"message": "Error getting tools from atlassian: Upstream MCP server 'atlassian' returned 401", "level": "ERROR", "timestamp": "2026-07-17T13:43:03.515401", "component": "LiteLLM", "logger": "rest_endpoints.py:840", "stacktrace": "Traceback (most recent call last):\n File \"/app/.venv/lib/python3.13/site-packages/anyio/streams/memory.py\", line 117, in receive\n return self.receive_nowait()\n ~~~~~~~~~~~~~~~~~~~^^\n File \"/app/.venv/lib/python3.13/site-packages/anyio/streams/memory.py\", line 112, in receive_nowait\n raise WouldBlock\nanyio.WouldBlock\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/app/.venv/lib/python3.13/site-packages/litellm/experimental_mcp_client/client.py\", line 361, in _execute_session_operation\n init_result = await session.initialize()\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/app/.venv/lib/python3.13/site-packages/mcp/client/session.py\", line 171, in initialize\n result = await self.send_request(\n ^^^^^^^^^^^^^^^^^^^^^^^^\n ...<16 lines>...\n )\n ^\n File \"/app/.venv/lib/python3.13/site-packages/mcp/shared/session.py\", line 292, in send_request\n response_or_error = await response_stream_reader.receive()\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/app/.venv/lib/python3.13/site-packages/anyio/streams/memory.py\", line 125, in receive\n await receive_event.wait()\n File \"/app/.venv/lib/python3.13/site-packages/anyio/_backends/_asyncio.py\", line 1804, in wait\n await self._event.wait()\n File \"/usr/lib/python3.13/asyncio/locks.py\", line 213, in wait\n await fut\nasyncio.exceptions.CancelledError: Cancelled via cancel scope 78a3551beed0\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/_experimental/mcp_server/mcp_server_manager.py\", line 3264, in _fetch_tools_with_timeout\n tools = await client.list_tools(raise_on_error=True)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/app/.venv/lib/python3.13/site-packages/litellm/experimental_mcp_client/client.py\", line 504, in list_tools\n result = await self.run_with_session(_list_tools_operation, quiet_on_error=raise_on_error)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/app/.venv/lib/python3.13/site-packages/litellm/experimental_mcp_client/client.py\", line 400, in run_with_session\n return await self._execute_session_operation(transport_ctx, operation)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/app/.venv/lib/python3.13/site-packages/litellm/experimental_mcp_client/client.py\", line 383, in _execute_session_operation\n raise root_cause from in_flight_error\n File \"/app/.venv/lib/python3.13/site-packages/mcp/client/streamable_http.py\", line 565, in handle_request_async\n await self._handle_post_request(ctx)\n File \"/app/.venv/lib/python3.13/site-packages/mcp/client/streamable_http.py\", line 358, in _handle_post_request\n response.raise_for_status()\n ~~~~~~~~~~~~~~~~~~~~~~~~~^^\n File \"/app/.venv/lib/python3.13/site-packages/httpx/_models.py\", line 829, in raise_for_status\n raise HTTPStatusError(message, request=request, response=self)\nhttpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://mcp.atlassian.com/v1/mcp/authv2'\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/_experimental/mcp_server/rest_endpoints.py\", line 830, in list_tool_rest_api\n tools_result = await _get_tools_for_single_server(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n ...<6 lines>...\n )\n ^\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/_experimental/mcp_server/rest_endpoints.py\", line 502, in _get_tools_for_single_server\n tools = await global_mcp_server_manager._get_tools_from_server(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n ...<6 lines>...\n )\n ^\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/_experimental/mcp_server/mcp_server_manager.py\", line 2626, in _get_tools_from_server\n tools = await self._fetch_tools_with_timeout(client, server.name)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/_experimental/mcp_server/mcp_server_manager.py\", line 3281, in _fetch_tools_with_timeout\n raise MCPUpstreamAuthError(\n ...<3 lines>...\n ) from e\nlitellm.proxy._experimental.mcp_server.exceptions.MCPUpstreamAuthError: Upstream MCP server 'atlassian' returned 401"}
{"message": "_get_tools_from_server for <another_mcp_server>...", "level": "INFO", "timestamp": "2026-07-17T13:43:03.526050", "component": "LiteLLM", "logger": "mcp_server_manager.py:2528"}
{"message": "<client-ip>:47494 - \"GET /health/liveliness HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:43:04.403285", "component": "uvicorn.access", "logger": "h11_impl.py:473"}
{"message": "<client-ip>:48694 - \"GET /health/readiness HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:43:12.280437", "component": "uvicorn.access", "logger": "h11_impl.py:473"}
What part of LiteLLM is this about?
Proxy
What LiteLLM version are you on ?
v1.93.0-rc2
Twitter / LinkedIn details
No response
Check for existing issues
What happened?
Summary
When a non-bridge OAuth2 MCP server is registered through LiteLLM's MCP OAuth relay against an upstream that supports real Dynamic Client Registration (for example a pass-through server pointed at Atlassian's Rovo MCP server), the client-facing
/registerresponse returns theredirect_urisfieldexactly as the upstream registration server echoed it back to LiteLLM, which is LiteLLM's own
/callbackURL. A spec-compliant OAuth 2.1 DCR client (in our case Open WebUI, connecting over Streamable HTTP) adopts that value for its own subsequent/authorizecalls, so LiteLLM's/callbackends upredirecting back to itself. The second hit lands on
/callbackcarrying the client's own opaquestatenonce rather than a LiteLLM relay-state handle or encrypted blob, fails to decrypt, and produces an error that reads exactly like the unrelated LIT-4197 "oauth_state ... Incorrect padding" failurefixed in #32146, which is what first led us to suspect that already-fixed bug had resurfaced on v1.93.0-rc.2
Environment
LiteLLM image
ghcr.io/berriai/litellm-database:v1.93.0-rc.2, deployed on Kubernetes, proxying an MCP server registered against an upstream OAuth2/DCR-capable MCP server (Atlassian Rovo MCP,https://mcp.atlassian.com/v1/mcp/authv2). MCP client is Open WebUI's native External Tools over Streamable HTTPwith Auth mode OAuth 2.1 (DCR)
Root cause
In
register_client_with_server(litellm/proxy/_experimental/mcp_server/discoverable_endpoints.py), for a plain (nondcr_bridge) OAuth2 server, the relay always submits LiteLLM's own callback asredirect_uriswhen registering with the upstream:```python
current_redirect_uri = f"{request_base_url}/callback"
...
register_data = {
"client_name": client_name,
"redirect_uris": client_redirect_uris if bridge_relay else [current_redirect_uri],
...
}
response = await async_client.post(mcp_server.registration_url, ..., json=register_data)
...
token_response = response.json()
...
return JSONResponse(token_response)
```
That's correct for the LiteLLM <-> upstream leg, since the upstream does need to send its authorization code to LiteLLM's own
/callback. The bug is the last line: the upstream's registration response,redirect_urisfield included, is returned unmodified to the calling MCP client. RFC 7591 has theclient use the redirect_uri(s) confirmed at registration for subsequent authorization requests, so the client, correctly per spec, starts sending
redirect_uri=<litellm-host>/callbackon its own/authorizecalls, exactly as captured in the access logs below/authorize's trusted-redirect check (validate_trusted_redirect_uri) allows this because it is same-origin to LiteLLM itself, so nothing rejects it before the loop plays out. When/callbackcompletes the real upstream leg, it redirects the browser to theclient_redirect_urisealed in the encryptedstate, which is now LiteLLM's own
/callback. That second hit carries the client's ownstatevalue instead of a relay-state handle, no per-flow cookie exists for it,_resolve_encoded_oauth_statefalls back to treating that raw value as the encrypted blob, anddecrypt_value_helper'sbase64.urlsafe_b64decodethrowsIncorrect paddingon a value that was never base64 to begin withThis is unrelated to LIT-4197 / #32146, the short relay-state-in-a-cookie fix, which is working correctly here: the first
/callbackhit succeeds with a clean redirect. This is a separate defect that happens to surface an identical-looking decrypt error on the second, self-inflicted hitEvidence (scrubbed access/error logs)
```
GET /.well-known/oauth-authorization-server/atlassian -> 200
GET /atlassian/authorize?response_type=code&client_id=&redirect_uri=https://litellm.example.com/callback&state=<state_a>&resource=https://litellm.example.com/atlassian/mcp&code_challenge=<challenge_a>&code_challenge_method=S256 -> 307
GET /atlassian/authorize?response_type=code&client_id=&redirect_uri=https%3A%2F%2Flitellm.example.com%2Fcallback&state=<state_b>&resource=https%3A%2F%2Flitellm.example.com%2Fatlassian%2Fmcp&code_challenge=<challenge_b>&code_challenge_method=S256 -> 307
...
GET /callback?state=<litellm_relay_state>&code=<upstream_code> -> 302
GET /callback?code=<upstream_code>&state=<state_b> -> 200
ERROR Error decrypting value for key: oauth_state, Did your master_key/salt key change recently?
Error: Incorrect padding
File ".../litellm/proxy/common_utils/encrypt_decrypt_utils.py", line 137, in decrypt_value_helper
decoded_b64 = base64.urlsafe_b64decode(value)
binascii.Error: Incorrect padding
Upstream auth failure from MCP server atlassian: HTTP 401
```
<state_b>in the last two log lines is the exact same value the client originally sent as its ownstateon the second/authorizecall, not a LiteLLM relay-state handle, which confirms the self-redirectSuggested fix
Stop forwarding the upstream registration response's
redirect_uristo the calling client verbatim on the non-bridge relay path. Either omitredirect_urisfrom the client-facing response, or report back whatever the calling client itself supplied at registration time. Thedcr_bridgerelay arm alreadythreads a
client_redirect_urisvalue through for this same reason and could likely be generalized to cover this path tooAlso observed, not yet root-caused
Two independent
/authorizecalls, each with a distinct clientstateand PKCEcode_challenge, land roughly 60ms apart for what should be a single user-initiated connection attempt. Only one of the two completes end to end. This didn't cause the failure above, since each flow is self-contained, but isworth a separate look on the client side
Steps to Reproduce
/registerbefore/authorizeand reuses the registration response'sredirect_urison/authorize(Open WebUI's native MCP External Tools with Auth: OAuth 2.1 (DCR) does this)Expected: the browser lands on the MCP client's own redirect endpoint with the final code/state, and the client completes its token exchange
Actual: the browser bounces to LiteLLM's own
/callbacka second time, which logs a decrypt error and returns "Authentication incomplete," the client never receives a valid code, and the subsequent tool call to the MCP server returns 401Relevant log output
{"message": "<client-ip>:49014 - \"GET /health/liveliness HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:34.402853", "component": "uvicorn.access", "logger": "h11_impl.py:473"} {"message": "<client-ip>:36624 - \"GET /health/readiness HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:42.281727", "component": "uvicorn.access", "logger": "h11_impl.py:473"} {"message": "<internal-ip>:36942 - \"GET /v1/models HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:46.642651", "component": "uvicorn.access", "logger": "h11_impl.py:473"} {"message": "Synced 0 production policies and 0 draft/published (by ID) from DB to in-memory registry", "level": "INFO", "timestamp": "2026-07-17T13:42:46.917745", "component": "LiteLLM Proxy", "logger": "policy_registry.py:548"} {"message": "Synced 0 attachments from DB to in-memory registry", "level": "INFO", "timestamp": "2026-07-17T13:42:46.918978", "component": "LiteLLM Proxy", "logger": "attachment_registry.py:453"} {"message": "Found 3 MCP servers in database", "level": "INFO", "timestamp": "2026-07-17T13:42:46.926676", "component": "LiteLLM", "logger": "mcp_server_manager.py:4454"} {"message": "Loading 0 search tool(s) into router (0 from config, 0 from database)", "level": "INFO", "timestamp": "2026-07-17T13:42:46.933641", "component": "LiteLLM Proxy", "logger": "proxy_server.py:6561"} {"message": "ToolPolicyRegistry: synced 195 tool policies and 1 object permissions from DB", "level": "INFO", "timestamp": "2026-07-17T13:42:46.942392", "component": "LiteLLM Proxy", "logger": "tool_registry_writer.py:328"} {"message": "Lazy-loaded optional feature 'mcp_byok_oauth' (module: litellm.proxy._experimental.mcp_server.byok_oauth_endpoints)", "level": "INFO", "timestamp": "2026-07-17T13:42:52.596276", "component": "LiteLLM Proxy", "logger": "_lazy_features.py:329"} {"message": "<client-ip>:0 - \"GET /.well-known/oauth-authorization-server/atlassian HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:52.598443", "component": "uvicorn.access", "logger": "h11_impl.py:473"} {"message": "<client-ip>:0 - \"GET /atlassian/authorize?response_type=code&client_id=<dcr_client_id>&redirect_uri=https://litellm.example.com/callback&state=<state_a>&resource=https://litellm.example.com/atlassian/mcp&code_challenge=<code_challenge_a>&code_challenge_method=S256 HTTP/1.1\" 307", "level": "INFO", "timestamp": "2026-07-17T13:42:52.654147", "component": "uvicorn.access", "logger": "h11_impl.py:473"} {"message": "<client-ip>:0 - \"GET /atlassian/authorize?response_type=code&client_id=<dcr_client_id>&redirect_uri=https%3A%2F%2Flitellm.example.com%2Fcallback&state=<state_b>&resource=https%3A%2F%2Flitellm.example.com%2Fatlassian%2Fmcp&code_challenge=<code_challenge_b>&code_challenge_method=S256 HTTP/1.1\" 307", "level": "INFO", "timestamp": "2026-07-17T13:42:52.717774", "component": "uvicorn.access", "logger": "h11_impl.py:473"} {"message": "<client-ip>:0 - \"GET /v1/mcp/server/health HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:53.286606", "component": "uvicorn.access", "logger": "h11_impl.py:473"} {"message": "_get_tools_from_server for atlassian...", "level": "INFO", "timestamp": "2026-07-17T13:42:54.085233", "component": "LiteLLM", "logger": "mcp_server_manager.py:2528"} {"message": "MCP client listed 36 tools from https://mcp.atlassian.com/v1/mcp/authv2: ['atlassianUserInfo', 'getAccessibleAtlassianResources', 'getConfluencePage', 'searchConfluenceUsingCql', 'getConfluenceSpaces', 'getPagesInConfluenceSpace', 'getConfluencePageFooterComments', 'getConfluencePageInlineComments', 'getConfluenceCommentChildren', 'getConfluencePageDescendants', 'getJiraIssue', 'editJiraIssue', 'createJiraIssue', 'getTransitionsForJiraIssue', 'getJiraIssueRemoteIssueLinks', 'getVisibleJiraProjects', 'getJiraProjectIssueTypesMetadata', 'getJiraIssueTypeMetaWithFields', 'addCommentToJiraIssue', 'transitionJiraIssue', 'searchJiraIssuesUsingJql', 'lookupJiraAccountId', 'addWorklogToJiraIssue', 'getIssueLinkTypes', 'createIssueLink', 'getCompassComponents', 'getCompassComponent', 'getCompassCustomFieldDefinitions', 'createCompassCustomFieldDefinition', 'createCompassComponent', 'createCompassComponentRelationship', 'getTeamworkGraphContext', 'getTeamworkGraphObject', 'addTeamworkGraphContext', 'search', 'fetch']", "level": "INFO", "timestamp": "2026-07-17T13:42:54.518506", "component": "LiteLLM", "logger": "client.py:507"} {"message": "Successfully fetched 36 tools from server atlassian", "level": "INFO", "timestamp": "2026-07-17T13:42:54.523893", "component": "LiteLLM", "logger": "mcp_server_manager.py:3385"} {"message": "<client-ip>:0 - \"GET /mcp-rest/tools/list?server_id=<server_id>&include_disabled_tools=true HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:54.524734", "component": "uvicorn.access", "logger": "h11_impl.py:473"} {"message": "<client-ip>:52526 - \"GET /health/readiness HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:57.286062", "component": "uvicorn.access", "logger": "h11_impl.py:473"} {"message": "<client-ip>:0 - \"GET /callback?state=<litellm_relay_state>&code=<upstream_authorization_code_jwt> HTTP/1.1\" 302", "level": "INFO", "timestamp": "2026-07-17T13:42:57.963309", "component": "uvicorn.access", "logger": "h11_impl.py:473"} {"message": "<client-ip>:0 - \"GET /callback?code=<upstream_authorization_code_jwt>&state=<state_b> HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:42:57.992837", "component": "uvicorn.access", "logger": "h11_impl.py:473"} {"message": "Error decrypting value for key: oauth_state, Did your master_key/salt key change recently? \nError: Incorrect padding\nSet permanent salt key - https://docs.litellm.ai/docs/proxy/prod#5-set-litellm-salt-key", "level": "ERROR", "timestamp": "2026-07-17T13:42:57.990389", "component": "LiteLLM Proxy", "logger": "encrypt_decrypt_utils.py:157", "stacktrace": "Traceback (most recent call last):\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/common_utils/encrypt_decrypt_utils.py\", line 137, in decrypt_value_helper\n decoded_b64 = base64.urlsafe_b64decode(value)\n File \"/usr/lib/python3.13/base64.py\", line 134, in urlsafe_b64decode\n return b64decode(s)\n File \"/usr/lib/python3.13/base64.py\", line 88, in b64decode\n return binascii.a2b_base64(s, strict_mode=validate)\n ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^\nbinascii.Error: Incorrect padding\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/common_utils/encrypt_decrypt_utils.py\", line 140, in decrypt_value_helper\n decoded_b64 = base64.b64decode(value)\n File \"/usr/lib/python3.13/base64.py\", line 88, in b64decode\n return binascii.a2b_base64(s, strict_mode=validate)\n ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^\nbinascii.Error: Incorrect padding"} {"message": "MCP client list_tools was cancelled", "level": "WARNING", "timestamp": "2026-07-17T13:43:03.482089", "component": "LiteLLM", "logger": "client.py:510"} {"message": "Timeout while listing tools from grafana", "level": "WARNING", "timestamp": "2026-07-17T13:43:03.482595", "component": "LiteLLM", "logger": "mcp_server_manager.py:3268"} {"message": "Successfully fetched 0 tools from server grafana", "level": "INFO", "timestamp": "2026-07-17T13:43:03.482889", "component": "LiteLLM", "logger": "mcp_server_manager.py:3385"} {"message": "_get_tools_from_server for atlassian...", "level": "INFO", "timestamp": "2026-07-17T13:43:03.483293", "component": "LiteLLM", "logger": "mcp_server_manager.py:2528"} {"message": "Upstream auth failure from MCP server atlassian: HTTP 401", "level": "INFO", "timestamp": "2026-07-17T13:43:03.514902", "component": "LiteLLM", "logger": "mcp_server_manager.py:3280"} {"message": "Error getting tools from atlassian: Upstream MCP server 'atlassian' returned 401", "level": "ERROR", "timestamp": "2026-07-17T13:43:03.515401", "component": "LiteLLM", "logger": "rest_endpoints.py:840", "stacktrace": "Traceback (most recent call last):\n File \"/app/.venv/lib/python3.13/site-packages/anyio/streams/memory.py\", line 117, in receive\n return self.receive_nowait()\n ~~~~~~~~~~~~~~~~~~~^^\n File \"/app/.venv/lib/python3.13/site-packages/anyio/streams/memory.py\", line 112, in receive_nowait\n raise WouldBlock\nanyio.WouldBlock\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/app/.venv/lib/python3.13/site-packages/litellm/experimental_mcp_client/client.py\", line 361, in _execute_session_operation\n init_result = await session.initialize()\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/app/.venv/lib/python3.13/site-packages/mcp/client/session.py\", line 171, in initialize\n result = await self.send_request(\n ^^^^^^^^^^^^^^^^^^^^^^^^\n ...<16 lines>...\n )\n ^\n File \"/app/.venv/lib/python3.13/site-packages/mcp/shared/session.py\", line 292, in send_request\n response_or_error = await response_stream_reader.receive()\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/app/.venv/lib/python3.13/site-packages/anyio/streams/memory.py\", line 125, in receive\n await receive_event.wait()\n File \"/app/.venv/lib/python3.13/site-packages/anyio/_backends/_asyncio.py\", line 1804, in wait\n await self._event.wait()\n File \"/usr/lib/python3.13/asyncio/locks.py\", line 213, in wait\n await fut\nasyncio.exceptions.CancelledError: Cancelled via cancel scope 78a3551beed0\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/_experimental/mcp_server/mcp_server_manager.py\", line 3264, in _fetch_tools_with_timeout\n tools = await client.list_tools(raise_on_error=True)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/app/.venv/lib/python3.13/site-packages/litellm/experimental_mcp_client/client.py\", line 504, in list_tools\n result = await self.run_with_session(_list_tools_operation, quiet_on_error=raise_on_error)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/app/.venv/lib/python3.13/site-packages/litellm/experimental_mcp_client/client.py\", line 400, in run_with_session\n return await self._execute_session_operation(transport_ctx, operation)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/app/.venv/lib/python3.13/site-packages/litellm/experimental_mcp_client/client.py\", line 383, in _execute_session_operation\n raise root_cause from in_flight_error\n File \"/app/.venv/lib/python3.13/site-packages/mcp/client/streamable_http.py\", line 565, in handle_request_async\n await self._handle_post_request(ctx)\n File \"/app/.venv/lib/python3.13/site-packages/mcp/client/streamable_http.py\", line 358, in _handle_post_request\n response.raise_for_status()\n ~~~~~~~~~~~~~~~~~~~~~~~~~^^\n File \"/app/.venv/lib/python3.13/site-packages/httpx/_models.py\", line 829, in raise_for_status\n raise HTTPStatusError(message, request=request, response=self)\nhttpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://mcp.atlassian.com/v1/mcp/authv2'\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/_experimental/mcp_server/rest_endpoints.py\", line 830, in list_tool_rest_api\n tools_result = await _get_tools_for_single_server(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n ...<6 lines>...\n )\n ^\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/_experimental/mcp_server/rest_endpoints.py\", line 502, in _get_tools_for_single_server\n tools = await global_mcp_server_manager._get_tools_from_server(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n ...<6 lines>...\n )\n ^\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/_experimental/mcp_server/mcp_server_manager.py\", line 2626, in _get_tools_from_server\n tools = await self._fetch_tools_with_timeout(client, server.name)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/_experimental/mcp_server/mcp_server_manager.py\", line 3281, in _fetch_tools_with_timeout\n raise MCPUpstreamAuthError(\n ...<3 lines>...\n ) from e\nlitellm.proxy._experimental.mcp_server.exceptions.MCPUpstreamAuthError: Upstream MCP server 'atlassian' returned 401"} {"message": "_get_tools_from_server for <another_mcp_server>...", "level": "INFO", "timestamp": "2026-07-17T13:43:03.526050", "component": "LiteLLM", "logger": "mcp_server_manager.py:2528"} {"message": "<client-ip>:47494 - \"GET /health/liveliness HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:43:04.403285", "component": "uvicorn.access", "logger": "h11_impl.py:473"} {"message": "<client-ip>:48694 - \"GET /health/readiness HTTP/1.1\" 200", "level": "INFO", "timestamp": "2026-07-17T13:43:12.280437", "component": "uvicorn.access", "logger": "h11_impl.py:473"}What part of LiteLLM is this about?
Proxy
What LiteLLM version are you on ?
v1.93.0-rc2
Twitter / LinkedIn details
No response