Skip to content

extra_body flattened by TransportKwargs breaks reasoning_effort param #409

@eric-tramel

Description

@eric-tramel

Bug: TransportKwargs flattens extra_body into top-level LiteLLM kwargs, breaking reasoning_effort

Problem

When a ModelConfig sets extra_body: {"reasoning_effort": "high"} in its inference parameters, TransportKwargs.from_request() merges the contents of extra_body into the top-level kwargs dict. The LiteLLM router call then receives reasoning_effort="high" as a top-level kwarg rather than inside extra_body.

LiteLLM validates top-level kwargs against a per-provider allowlist. Since reasoning_effort isn't in the openai provider's allowlist, it raises UnsupportedParamsError:

litellm.UnsupportedParamsError: openai does not support parameters: ['reasoning_effort'], for model=my-model.

Passing extra_body={"reasoning_effort": "high"} directly to litellm.completion() works fine. LiteLLM forwards extra_body contents in the HTTP request body without validation. The flattening is what breaks it.

Reproduction

  1. Configure a model with reasoning_effort in extra_body:
- alias: my-model
  model: some-model
  inference_parameters:
    generation_type: chat-completion
    extra_body:
      reasoning_effort: high
  provider: nvidia
  1. Run any generation. The health check and/or generation will fail with ModelBadRequestError wrapping the UnsupportedParamsError.

Root cause

In TransportKwargs.from_request() (clients/types.py), extra_body is popped and its keys are merged into the top-level body dict. This is documented as "mirroring how LiteLLM flattens them," but LiteLLM treats extra_body as a distinct kwarg that bypasses param validation. Flattening it defeats that bypass.

Current workaround

Nest extra_body one level deeper so that after DD flattens the outer layer, the inner extra_body survives intact for LiteLLM:

extra_body={"extra_body": {"reasoning_effort": "high"}}

This works but is not something anyone should have to discover.

Suggested fix

TransportKwargs.from_request() should preserve extra_body as extra_body in the output body dict rather than merging its contents into the top level. This matches LiteLLM's actual semantics: extra_body is forwarded in the HTTP body without param validation.

Environment

  • data-designer v0.5.2 (commit ebd3322)
  • litellm 1.80.11

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions