Bug: TransportKwargs flattens extra_body into top-level LiteLLM kwargs, breaking reasoning_effort
Problem
When a ModelConfig sets extra_body: {"reasoning_effort": "high"} in its inference parameters, TransportKwargs.from_request() merges the contents of extra_body into the top-level kwargs dict. The LiteLLM router call then receives reasoning_effort="high" as a top-level kwarg rather than inside extra_body.
LiteLLM validates top-level kwargs against a per-provider allowlist. Since reasoning_effort isn't in the openai provider's allowlist, it raises UnsupportedParamsError:
litellm.UnsupportedParamsError: openai does not support parameters: ['reasoning_effort'], for model=my-model.
Passing extra_body={"reasoning_effort": "high"} directly to litellm.completion() works fine. LiteLLM forwards extra_body contents in the HTTP request body without validation. The flattening is what breaks it.
Reproduction
- Configure a model with
reasoning_effort in extra_body:
- alias: my-model
model: some-model
inference_parameters:
generation_type: chat-completion
extra_body:
reasoning_effort: high
provider: nvidia
- Run any generation. The health check and/or generation will fail with
ModelBadRequestError wrapping the UnsupportedParamsError.
Root cause
In TransportKwargs.from_request() (clients/types.py), extra_body is popped and its keys are merged into the top-level body dict. This is documented as "mirroring how LiteLLM flattens them," but LiteLLM treats extra_body as a distinct kwarg that bypasses param validation. Flattening it defeats that bypass.
Current workaround
Nest extra_body one level deeper so that after DD flattens the outer layer, the inner extra_body survives intact for LiteLLM:
extra_body={"extra_body": {"reasoning_effort": "high"}}
This works but is not something anyone should have to discover.
Suggested fix
TransportKwargs.from_request() should preserve extra_body as extra_body in the output body dict rather than merging its contents into the top level. This matches LiteLLM's actual semantics: extra_body is forwarded in the HTTP body without param validation.
Environment
- data-designer v0.5.2 (commit ebd3322)
- litellm 1.80.11
Bug:
TransportKwargsflattensextra_bodyinto top-level LiteLLM kwargs, breakingreasoning_effortProblem
When a
ModelConfigsetsextra_body: {"reasoning_effort": "high"}in its inference parameters,TransportKwargs.from_request()merges the contents ofextra_bodyinto the top-level kwargs dict. The LiteLLM router call then receivesreasoning_effort="high"as a top-level kwarg rather than insideextra_body.LiteLLM validates top-level kwargs against a per-provider allowlist. Since
reasoning_effortisn't in theopenaiprovider's allowlist, it raisesUnsupportedParamsError:Passing
extra_body={"reasoning_effort": "high"}directly tolitellm.completion()works fine. LiteLLM forwardsextra_bodycontents in the HTTP request body without validation. The flattening is what breaks it.Reproduction
reasoning_effortinextra_body:ModelBadRequestErrorwrapping theUnsupportedParamsError.Root cause
In
TransportKwargs.from_request()(clients/types.py),extra_bodyis popped and its keys are merged into the top-level body dict. This is documented as "mirroring how LiteLLM flattens them," but LiteLLM treatsextra_bodyas a distinct kwarg that bypasses param validation. Flattening it defeats that bypass.Current workaround
Nest
extra_bodyone level deeper so that after DD flattens the outer layer, the innerextra_bodysurvives intact for LiteLLM:This works but is not something anyone should have to discover.
Suggested fix
TransportKwargs.from_request()should preserveextra_bodyasextra_bodyin the output body dict rather than merging its contents into the top level. This matches LiteLLM's actual semantics:extra_bodyis forwarded in the HTTP body without param validation.Environment