[Bug]: Incompatibility of LiteLLM's extra_body Parameter with vLLM

[sir3mat]

What happened?

I have the meta-llama/Meta-Llama-3-8b-Instruct model exposed using vLLM with the OpenAI API interface. I'm attempting to evaluate my system using Phoenix, but I need to utilize LiteLLM to interact with my vLLM model. This is the link to Phoenic docs (https://docs.arize.com/phoenix/api/evaluation-models#litellmmodel)

However, I'm encountering an issue where the litellm.completion function adds an extra_body parameter to the request, which causes an error in vLLM.

My code snippet that throws error

os.environ["OPENAI_API_KEY"] = "api_key"

eval_model = LiteLLMModel(
    model="openai/meta-llama/Meta-Llama-3-8b-Instruct",
    model_kwargs=dict(
        api_base="http://llama3-8b:8000/v1/chat/completions",
    ),
)

Function Call in LiteLLM:

litellm.completion(
    model='openai/meta-llama/Meta-Llama-3-8b-Instruct',
    messages=[{'content': 'Tell me a joke', 'role': 'user'}],
    temperature=0.0,
    max_tokens=256,
    top_p=1,
    num_retries=0,
    request_timeout=60,
    api_base='http://llama3-8b:8000/v1/chat/completions'
)

cURL Request Sent from LiteLLM:

curl -X POST \
http://llama3-8b:8000/v1/chat/completions/ \
-d '{'model': 'meta-llama/Meta-Llama-3-8b-Instruct', 'messages': [{'content': 'Tell me a joke?', 'role': 'user'}], 'temperature': 0.0, 'top_p': 1, 'max_tokens': 256, 'extra_body': {}}'

Error Response from vLLM:

{
    "object": "error",
    "message": "[{'type': 'extra_forbidden', 'loc': ('body', 'extra_body'), 'msg': 'Extra inputs are not permitted', 'input': {}}]",
    "type": "BadRequestError",
    "param": null,
    "code": 400
}

Request:
I need a way to configure or modify LiteLLM so that it does not include the extra_body parameter in the request, as this parameter is not accepted by vLLM and causes a BadRequestError.

Thank you for your assistance!

Relevant log output

No response

Twitter / LinkedIn details

No response

Hey @sir3mat just pushed a fix to let you drop extra_body with drop_additional_params https://docs.litellm.ai/docs/completion/drop_params#specify-params-to-drop

Where would this have been helpful to have on our docs?

- litellm_params:
    api_base: my-base
    model: openai/my-model
    additional_drop_params: ["extra_body"] # 👈 KEY CHANGE
  model_name: my-model