Summary
When using gemini-3-pro-preview through our LiteLLM proxy (models-proxy), requests fail with a 400 Bad Request after the first tool call in any multi-turn interaction. This manifests both through LibreChat and directly through OpenAI-compatible clients like Roo Code. Simple single-turn curl requests to the proxy work fine, which confirms the issue is specifically in how LiteLLM handles multi-turn conversation history — not in the gateway routing or auth layer.
Root Cause
Gemini 3 models introduced a new API requirement: thought signatures. When the model returns a function call, the response includes an encrypted thought_signature field attached to that function call part. This signature is an opaque representation of the model's internal reasoning state and must be passed back verbatim in the next request as part of the conversation history.
Google's API enforces this strictly on gemini-3-pro-preview and gemini-3-flash-preview: if any function call in the current turn is missing its thought_signature, the API returns:
400 INVALID_ARGUMENT
"Function call is missing a thought_signature in functionCall parts.
This is required for tools to work correctly..."
Reference: https://ai.google.dev/gemini-api/docs/thought-signatures
The failure mode in our setup is: Turn 1 succeeds (no history yet). On Turn 2, the client (LibreChat or Roo Code) sends back the assistant message from Turn 1 as conversation history. If the thought_signature that was attached to the function call in that Turn 1 response was not preserved in the message object, the Turn 2 request is rejected with a 400.
The thought_signature is stored in a non-standard field: tool_calls[].provider_specific_fields.thought_signature. Clients that reconstruct or deserialize the assistant message without preserving provider_specific_fields will silently drop it — which is exactly what happens here.
Investigation & Reproduction
What was already tried:
- Port-forwarded the LiteLLM proxy (
models-proxy) directly to bypass Envoy, Authorino, and the rest of the gateway stack.
- Sent simple curl requests to the proxy — these worked fine (single-turn, no tool history to preserve).
- Sent requests through Roo Code pointed directly at the port-forwarded proxy — same 400 error as through the full stack.
Conclusion: The gateway is not the culprit. The error originates in the LiteLLM proxy itself, specifically in how it handles or fails to handle thought signatures in streaming responses before passing them back to clients.
Relevant upstream issues:
How Thought Signatures Work (for context)
Per Google's documentation:
- Gemini 3 returns a
thoughtSignature on every function call part in a response.
- For parallel function calls, only the first function call gets a signature. All others don't.
- For sequential (multi-step) function calls, each step's first function call has its own signature.
- When sending conversation history back, the signature from each step's first function call must be included in the exact same position in the message. Google only validates the current turn (newest turn in history), not previous ones.
- Strict enforcement: omitting a signature on any step in the current turn → 400 error, no exceptions.
LiteLLM's documented fix is automatic: when stream_chunk_builder() reassembles a streamed response, it is supposed to preserve provider_specific_fields.thought_signature on tool calls. The fix for this was merged and is supposed to be active in recent versions.
The Dummy Signature Bypass (per our Project Lead)
Google's own documentation acknowledges a compatibility path for clients that don't support thought signatures natively. Sending a dummy value of skip_thought_signature_validator as the thought_signature instructs the API to skip validation for that turn. LiteLLM implements this internally when it detects a model switch mid-conversation (e.g. from gemini-2.5-flash to gemini-3-pro-preview) — it injects this dummy value to avoid breaking existing conversation history.
This same mechanism could be applied as a workaround if the real signature is being dropped: explicitly injecting skip_thought_signature_validator as the thought_signature value for any tool call that's missing one before forwarding the request to Google's API.
Note: Google warns that using dummy signatures may lead to degraded model performance (the model loses its reasoning context). It should be treated as a fallback, not a permanent solution.
Currently Deployed Version
docker.litellm.ai/berriai/litellm:main-v1.82.4-nightly
Defined in charts/models-proxy/values.yaml under global.litellm.version.
LiteLLM's streaming thought signature preservation fix was introduced in the period around the Gemini 3 launch (November 2025). It is not clear whether v1.82.4-nightly includes it — this needs to be verified.
What the Assignee Needs to Do
Step 1 — Verify whether the current version has the fix
Check the LiteLLM changelog / release notes for thought signature streaming fixes relative to v1.82.4-nightly. The fix should be in stream_chunk_builder() and in the Gemini translation layer that preserves provider_specific_fields on tool calls.
If the fix is already present: move to Step 3 — the bug may be upstream in how LibreChat/Roo Code reconstructs the assistant message before sending it back.
Step 2 — If the version is too old, upgrade
Update global.litellm.version in charts/models-proxy/values.yaml to the latest stable release. At time of writing, the latest stable is available at https://docs.litellm.ai/release_notes.
After upgrading, retest with gemini-3-pro-preview through a tool-calling client (e.g. Roo Code or a direct multi-turn curl sequence with tool calls).
Step 3 — If the version is up to date and the error persists
The issue may be that LibreChat (or the client) is receiving the assistant message with thought_signature in provider_specific_fields, but stripping it when reconstructing the message for the next request. In this case:
- Investigate whether LibreChat passes
provider_specific_fields through in its tool call history. See LibreChat#10566 for prior art.
- As a proxy-side mitigation, consider configuring LiteLLM to automatically inject the dummy
skip_thought_signature_validator signature for any tool call in an incoming request that is missing one. This will degrade reasoning quality but will unblock users.
Step 4 — Document and validate the fix
Once a fix is in place, validate with all three surfaces:
Acceptance Criteria
Relevant Files
Summary
When using
gemini-3-pro-previewthrough our LiteLLM proxy (models-proxy), requests fail with a400 Bad Requestafter the first tool call in any multi-turn interaction. This manifests both through LibreChat and directly through OpenAI-compatible clients like Roo Code. Simple single-turn curl requests to the proxy work fine, which confirms the issue is specifically in how LiteLLM handles multi-turn conversation history — not in the gateway routing or auth layer.Root Cause
Gemini 3 models introduced a new API requirement: thought signatures. When the model returns a function call, the response includes an encrypted
thought_signaturefield attached to that function call part. This signature is an opaque representation of the model's internal reasoning state and must be passed back verbatim in the next request as part of the conversation history.Google's API enforces this strictly on
gemini-3-pro-previewandgemini-3-flash-preview: if any function call in the current turn is missing itsthought_signature, the API returns:Reference: https://ai.google.dev/gemini-api/docs/thought-signatures
The failure mode in our setup is: Turn 1 succeeds (no history yet). On Turn 2, the client (LibreChat or Roo Code) sends back the assistant message from Turn 1 as conversation history. If the
thought_signaturethat was attached to the function call in that Turn 1 response was not preserved in the message object, the Turn 2 request is rejected with a 400.The
thought_signatureis stored in a non-standard field:tool_calls[].provider_specific_fields.thought_signature. Clients that reconstruct or deserialize the assistant message without preservingprovider_specific_fieldswill silently drop it — which is exactly what happens here.Investigation & Reproduction
What was already tried:
models-proxy) directly to bypass Envoy, Authorino, and the rest of the gateway stack.Conclusion: The gateway is not the culprit. The error originates in the LiteLLM proxy itself, specifically in how it handles or fails to handle thought signatures in streaming responses before passing them back to clients.
Relevant upstream issues:
thought_signaturein streaming mode causing errors on subsequent turnsHow Thought Signatures Work (for context)
Per Google's documentation:
thoughtSignatureon every function call part in a response.LiteLLM's documented fix is automatic: when
stream_chunk_builder()reassembles a streamed response, it is supposed to preserveprovider_specific_fields.thought_signatureon tool calls. The fix for this was merged and is supposed to be active in recent versions.The Dummy Signature Bypass (per our Project Lead)
Google's own documentation acknowledges a compatibility path for clients that don't support thought signatures natively. Sending a dummy value of
skip_thought_signature_validatoras thethought_signatureinstructs the API to skip validation for that turn. LiteLLM implements this internally when it detects a model switch mid-conversation (e.g. fromgemini-2.5-flashtogemini-3-pro-preview) — it injects this dummy value to avoid breaking existing conversation history.This same mechanism could be applied as a workaround if the real signature is being dropped: explicitly injecting
skip_thought_signature_validatoras thethought_signaturevalue for any tool call that's missing one before forwarding the request to Google's API.Note: Google warns that using dummy signatures may lead to degraded model performance (the model loses its reasoning context). It should be treated as a fallback, not a permanent solution.
Currently Deployed Version
Defined in
charts/models-proxy/values.yamlunderglobal.litellm.version.LiteLLM's streaming thought signature preservation fix was introduced in the period around the Gemini 3 launch (November 2025). It is not clear whether
v1.82.4-nightlyincludes it — this needs to be verified.What the Assignee Needs to Do
Step 1 — Verify whether the current version has the fix
Check the LiteLLM changelog / release notes for thought signature streaming fixes relative to
v1.82.4-nightly. The fix should be instream_chunk_builder()and in the Gemini translation layer that preservesprovider_specific_fieldson tool calls.If the fix is already present: move to Step 3 — the bug may be upstream in how LibreChat/Roo Code reconstructs the assistant message before sending it back.
Step 2 — If the version is too old, upgrade
Update
global.litellm.versionincharts/models-proxy/values.yamlto the latest stable release. At time of writing, the latest stable is available at https://docs.litellm.ai/release_notes.After upgrading, retest with
gemini-3-pro-previewthrough a tool-calling client (e.g. Roo Code or a direct multi-turn curl sequence with tool calls).Step 3 — If the version is up to date and the error persists
The issue may be that LibreChat (or the client) is receiving the assistant message with
thought_signatureinprovider_specific_fields, but stripping it when reconstructing the message for the next request. In this case:provider_specific_fieldsthrough in its tool call history. See LibreChat#10566 for prior art.skip_thought_signature_validatorsignature for any tool call in an incoming request that is missing one. This will degrade reasoning quality but will unblock users.Step 4 — Document and validate the fix
Once a fix is in place, validate with all three surfaces:
Acceptance Criteria
gemini-3-pro-previewcompletes multi-turn tool-calling interactions without a 400 errorvalues.yamlreflects the fixRelevant Files
charts/models-proxy/values.yaml— LiteLLM image version and proxy config