fix: filter thought parts client-side when include_thoughts=False on Vertex AI#2240
Open
nemetron wants to merge 1 commit intogoogleapis:mainfrom
Open
fix: filter thought parts client-side when include_thoughts=False on Vertex AI#2240nemetron wants to merge 1 commit intogoogleapis:mainfrom
nemetron wants to merge 1 commit intogoogleapis:mainfrom
Conversation
…Vertex AI On Vertex AI, ThinkingConfig(include_thoughts=False) does not suppress thought parts from the API response for image generation models like gemini-3.1-flash-image-preview. The part.thought flag is correctly set to True on thought parts by the API, so client-side filtering is safe and reliable. This change adds _filter_thought_parts(), called in all four response paths (sync/async, unary/streaming), which removes parts where part.thought is True when include_thoughts is explicitly False. The filter is a strict opt-in: it only activates when include_thoughts is set to False, leaving all other configurations unchanged. Fixes: googleapis#2239
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
|
Hi @nemetron, Thanks for reaching out us! I noticed some checks have failed. Kindly resolve the conflicts. Thanks |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On Vertex AI,
ThinkingConfig(include_thoughts=False)does not suppress thought parts from the API response for image generation models (e.g.gemini-3.1-flash-image-preview). All intermediate thought text and draft images are returned in the response despite the config, making the flag a no-op on Vertex AI.The
part.thoughtboolean IS correctly set toTrueon thought parts by the API, so the information needed for client-side filtering is available.Fixes #2239
Root Cause
The Vertex AI backend does not honor
include_thoughts=Falsefor this model family — it returns all thought parts regardless. The SDK passed them through without filtering.Fix
Added
_filter_thought_parts()inmodels.py, called in all four response paths (sync/async, unary/streaming). It inspectsThinkingConfig.include_thoughtsand removes parts wherepart.thought is Truewhen the value is explicitlyFalse.The filter is a strict opt-in:
include_thoughts=False→ thought parts strippedinclude_thoughts=True→ unchangedinclude_thoughts=None/ noThinkingConfig/ noconfig→ unchangedTests
7 new unit tests covering all branches of
_filter_thought_parts:Reproduction (from issue)