[Feature]: hosted_vllm provider `reasoning_effort` support

[andresC98]

The Feature

Currently, using reasoning_effort parameter with hosted_vllm provider does not work.

For example, I am serving gpt-oss-120b model using vLLM and when I try quering it via litellm proxy and passing reasoning_effort to the OpenAI chat completions client

response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "whats 2 + 2"}],
    reasoning_effort="high",
)

gives an error

litellm.UnsupportedParamsError: hosted_vllm does not support parameters: ['reasoning_effort'], for model=gpt-oss-120b. To drop these, set litellm.drop_params=True or for proxy:\n\nlitellm_settings:\n drop_params: true\n. \n If you want to use these params dynamically send allowed_openai_params=['reasoning_effort'] in your request.

Full traceback

Traceback (most recent call last):
  File "function-calling.py", line 36, in <module>
    response = client.chat.completions.create(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/openai/_utils/_utils.py", line 287, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/openai/resources/chat/completions/completions.py", line 925, in create
    return self._post(
           ^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/openai/_base_client.py", line 1239, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/openai/_base_client.py", line 1034, in request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "litellm.UnsupportedParamsError: hosted_vllm does not support parameters: ['reasoning_effort'], for model=gpt-oss-120b. To drop these, set `litellm.drop_params=True` or for proxy:\n\n`litellm_settings:\n drop_params: true`\n. \n If you want to use these params dynamically send allowed_openai_params=['reasoning_effort'] in your request.. Received Model Group=gpt-oss-120b\nAvailable Model Group Fallbacks=None", 'type': 'None', 'param': None, 'code': '400'}}

This is my current model config in proxy config yaml

      - model_name: gpt-oss-120b
        litellm_params:
          model: hosted_vllm/gpt-oss-120b
          api_base: https://<redacted>
          api_key: os.environ/VLLM_API_KEY
        model_info:
          mode: chat
          max_input_tokens: 9999
          max_output_tokens: 10000

Motivation, pitch

Given that OpenAI has released OSS models, it makes sense to add this openai params support to hosted_vllm provider.

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

No response

[dhs-shine]

Workaround for openAI compatible endpoint. hosted_vllm was not tested:

      - model_name: gpt-oss-120b
        litellm_params:
          model: openai/gpt-oss-120b
          api_base: https://<redacted>
          api_key: os.environ/VLLM_API_KEY
          allowed_openai_params: ["reasoning_effort"]
        model_info:
          mode: chat
          max_input_tokens: 9999
          max_output_tokens: 10000