[Bug]: custom_llm_provider deepseek - unhealthy health status / invalid base_url

[hipotures]

What happened?

A bug happened!

litellm/litellm:v1.59.8-stable (and older)
webui -> /health Models

Incorrect base_url when checking health status for deepseek models. Access from the code to this model is correct.

Relevant log output

"unhealthy_endpoints": [
    {
      "custom_llm_provider": "deepseek",
      "model": "deepseek/deepseek-reasoner",
      "cache": {
        "no-cache": true
      },
      "error": "litellm.AuthenticationError: AuthenticationError: DeepseekException - Error code: 401 - {'error': {'message': 'Incorrect API key provided: xxxxxxxxxxxxxxxxxxxx. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}\nHave you set 'mode' - https://docs.litellm.ai/docs/proxy/health#embedding-models\nstack trace: Traceback (most recent call last):\n  File \"/usr/lib/python3.13/site-packages/litellm/llms/openai/openai.py\", line 770, in acompletion\n    headers, response = await self.make_openai_chat_completion_request(\n                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n    ...<4 lines>...\n    )\n    ^\n  File \"/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/logging_utils.py\", line 131, in async_wrapper\n    result = await func(*args, **kwargs)\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/lib/python3.13/site-packages/litellm/llms/openai/openai.py\", line 418, in make_openai_chat_completion_request\n    raise e\n  File \"/usr/lib/python3.13/site-packages/litellm/llms/openai/openai.py\", line 400, in make_openai_chat_completion_request\n    await openai_aclient.chat.completions.with_raw_response.create(\n        **data, timeout=timeout\n    )\n  File \"/usr/lib/python3.13/site-packages/openai/_legacy_response.py\", line 373, in wrapped\n    return cast(LegacyAPIResponse[R],"
    },

Are you a ML Ops Team?

Yes

What LiteLLM version are you on ?

v1.59.8-stable

Twitter / LinkedIn details

No response

[krrishdholakia]

Incorrect API key provided: xxxxxxxxxxxxxxxxxxxx

This looks like it's an auth error for calling deepseek.

Does the chat completion call work? @hipotures

[hipotures]

import os 
import litellm
from litellm import completion

messages = [{ "content": "Hello, how are you?","role": "user"}]

response = completion(
    messages=messages,
    model="litellm_proxy/deepseek-chat",
    api_base = "https://litellm-proxy-xxx.app",
    api_key = "xxxx",
)

print(response)

`ModelResponse(id='3ef9efd9-1a49-40b8-a092-ea5aa4329c7e', created=1738488930, model='litellm_proxy/deepseek/deepseek-chat', object='chat.completion', system_fingerprint='xxxx', choices=[Choices(finish_reason='stop', index=0, message=Message(content="Hello! I'm just a virtual assistant, so I don't have feelings, but I'm here and ready to help you with whatever you need. How are *you* doing? 😊", role='assistant', tool_calls=None, function_call=None, provider_specific_fields={'refusal': None}))], usage=Usage(completion_tokens=39, prompt_tokens=9, total_tokens=48, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None), prompt_cache_hit_tokens=0, prompt_cache_miss_tokens=9), service_tier=None)`

The same problem with deepseek-reasoner and deepseek-chat.

[krrishdholakia]

Can you share the config yaml?

Trying to repro the setup

[hipotures]

Created from WebUI as Model DB.

{
  "model_name": "deepseek-reasoner",
  "litellm_params": {
    "custom_llm_provider": "deepseek",
    "use_in_pass_through": false,
    "model": "deepseek/deepseek-reasoner"
  },
  "model_info": {
    "id": "281a226b-661a-4eb1-8739-05096631b7d6",
    "db_model": true,
    "key": "deepseek/deepseek-reasoner",
    "max_tokens": 8192,
    "max_input_tokens": 64000,
    "max_output_tokens": 8192,
    "input_cost_per_token": 5.5e-7,
    "cache_creation_input_token_cost": null,
    "cache_read_input_token_cost": null,
    "input_cost_per_character": null,
    "input_cost_per_token_above_128k_tokens": null,
    "input_cost_per_query": null,
    "input_cost_per_second": null,
    "input_cost_per_audio_token": null,
    "output_cost_per_token": 0.00000219,
    "output_cost_per_audio_token": null,
    "output_cost_per_character": null,
    "output_cost_per_token_above_128k_tokens": null,
    "output_cost_per_character_above_128k_tokens": null,
    "output_cost_per_second": null,
    "output_cost_per_image": null,
    "output_vector_size": null,
    "litellm_provider": "deepseek",
    "mode": "chat",
    "supports_system_messages": null,
    "supports_response_schema": null,
    "supports_vision": false,
    "supports_function_calling": true,
    "supports_assistant_prefill": true,
    "supports_prompt_caching": true,
    "supports_audio_input": false,
    "supports_audio_output": false,
    "supports_pdf_input": false,
    "supports_embedding_image_input": false,
    "supports_native_streaming": null,
    "tpm": null,
    "rpm": null,
    "supported_openai_params": [
      "frequency_penalty",
      "logit_bias",
      "logprobs",
      "top_logprobs",
      "max_tokens",
      "max_completion_tokens",
      "modalities",
      "prediction",
      "n",
      "presence_penalty",
      "seed",
      "stop",
      "stream",
      "stream_options",
      "temperature",
      "top_p",
      "tools",
      "tool_choice",
      "function_call",
      "functions",
      "max_retries",
      "extra_headers",
      "parallel_tool_calls",
      "response_format"
    ]
  },
  "provider": "deepseek",
  "input_cost": "0.55",
  "output_cost": "2.19",
  "litellm_model_name": "deepseek/deepseek-reasoner",
  "max_tokens": 8192,
  "max_input_tokens": 64000,
  "cleanedLitellmParams": {
    "custom_llm_provider": "deepseek",
    "use_in_pass_through": false
  }
}

{
  "model_name": "deepseek-chat",
  "litellm_params": {
    "custom_llm_provider": "deepseek",
    "use_in_pass_through": false,
    "model": "deepseek/deepseek-chat"
  },
  "model_info": {
    "id": "d5d4783e-4d22-4a2b-bb1c-ba0a85facc1b",
    "db_model": true,
    "key": "deepseek/deepseek-chat",
    "max_tokens": 4096,
    "max_input_tokens": 128000,
    "max_output_tokens": 4096,
    "input_cost_per_token": 1.4e-7,
    "cache_creation_input_token_cost": 0,
    "cache_read_input_token_cost": 1.4e-8,
    "input_cost_per_character": null,
    "input_cost_per_token_above_128k_tokens": null,
    "input_cost_per_query": null,
    "input_cost_per_second": null,
    "input_cost_per_audio_token": null,
    "output_cost_per_token": 2.8e-7,
    "output_cost_per_audio_token": null,
    "output_cost_per_character": null,
    "output_cost_per_token_above_128k_tokens": null,
    "output_cost_per_character_above_128k_tokens": null,
    "output_cost_per_second": null,
    "output_cost_per_image": null,
    "output_vector_size": null,
    "litellm_provider": "deepseek",
    "mode": "chat",
    "supports_system_messages": null,
    "supports_response_schema": null,
    "supports_vision": false,
    "supports_function_calling": true,
    "supports_assistant_prefill": true,
    "supports_prompt_caching": true,
    "supports_audio_input": false,
    "supports_audio_output": false,
    "supports_pdf_input": false,
    "supports_embedding_image_input": false,
    "supports_native_streaming": null,
    "tpm": null,
    "rpm": null,
    "supported_openai_params": [
      "frequency_penalty",
      "logit_bias",
      "logprobs",
      "top_logprobs",
      "max_tokens",
      "max_completion_tokens",
      "modalities",
      "prediction",
      "n",
      "presence_penalty",
      "seed",
      "stop",
      "stream",
      "stream_options",
      "temperature",
      "top_p",
      "tools",
      "tool_choice",
      "function_call",
      "functions",
      "max_retries",
      "extra_headers",
      "parallel_tool_calls",
      "response_format"
    ]
  },
  "provider": "deepseek",
  "input_cost": "0.14",
  "output_cost": "0.28",
  "litellm_model_name": "deepseek/deepseek-chat",
  "max_tokens": 4096,
  "max_input_tokens": 128000,
  "cleanedLitellmParams": {
    "custom_llm_provider": "deepseek",
    "use_in_pass_through": false
  }
}

Hhmm?

[Krzemq]

Got the same error, it looks like it is trying to reach openai api instead of deepseek.

"error": "litellm.AuthenticationError: AuthenticationError: DeepseekException - Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-f2913***********************cafa. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}\nHave you set 'mode' - https://docs.litellm.ai/docs/proxy/health#embedding-models\nstack trace: Traceback (most recent call last):\n File \"/usr/local/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py\", line 917, in acompletion\n headers, response = await self.make_openai_chat_completion_request(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/usr/local/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py\", line 624, in make_openai_chat_completion_request\n raise e\n File \"/usr/local/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py\", line 612, in make_openai_chat_completion_request\n await openai_aclient.chat.completions.with_raw_response.create(\n File \"/usr/local/lib/python3.11/site-packages/openai/_legacy_response.py\", line 373, in wrapped\n return cast(LegacyAPIResponse[R], await func(*args, **kwargs))\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/usr/local/lib/python3.11/site-packages/openai/resources/chat/completions.py\", line 1661, in create\n return await self._post(\n " }

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.