Skip to content

fix: filter thought parts client-side when include_thoughts=False on Vertex AI#2240

Open
nemetron wants to merge 1 commit intogoogleapis:mainfrom
nemetron:fix/vertex-ai-include-thoughts-not-filtering
Open

fix: filter thought parts client-side when include_thoughts=False on Vertex AI#2240
nemetron wants to merge 1 commit intogoogleapis:mainfrom
nemetron:fix/vertex-ai-include-thoughts-not-filtering

Conversation

@nemetron
Copy link
Copy Markdown

@nemetron nemetron commented Apr 7, 2026

Problem

On Vertex AI, ThinkingConfig(include_thoughts=False) does not suppress thought parts from the API response for image generation models (e.g. gemini-3.1-flash-image-preview). All intermediate thought text and draft images are returned in the response despite the config, making the flag a no-op on Vertex AI.

The part.thought boolean IS correctly set to True on thought parts by the API, so the information needed for client-side filtering is available.

Fixes #2239

Root Cause

The Vertex AI backend does not honor include_thoughts=False for this model family — it returns all thought parts regardless. The SDK passed them through without filtering.

Fix

Added _filter_thought_parts() in models.py, called in all four response paths (sync/async, unary/streaming). It inspects ThinkingConfig.include_thoughts and removes parts where part.thought is True when the value is explicitly False.

The filter is a strict opt-in:

  • include_thoughts=False → thought parts stripped
  • include_thoughts=True → unchanged
  • include_thoughts=None / no ThinkingConfig / no config → unchanged

Tests

7 new unit tests covering all branches of _filter_thought_parts:

PASSED test_include_thoughts_false_removes_thought_parts
PASSED test_include_thoughts_true_preserves_all_parts
PASSED test_include_thoughts_none_preserves_all_parts
PASSED test_no_thinking_config_preserves_all_parts
PASSED test_no_config_preserves_all_parts
PASSED test_empty_candidates_is_safe
PASSED test_no_thought_parts_in_response

Reproduction (from issue)

from google import genai
from google.genai import types

client = genai.Client(vertexai=True, project="YOUR_PROJECT", location="us-central1")

response = client.models.generate_content(
    model="gemini-3.1-flash-image-preview",
    contents="Draw a red car",
    config=types.GenerateContentConfig(
        response_modalities=["TEXT", "IMAGE"],
        thinking_config=types.ThinkingConfig(include_thoughts=False),
    ),
)

for i, part in enumerate(response.candidates[0].content.parts):
    print(f"[{i}] thought={part.thought}, has_text={bool(part.text)}, has_image={bool(part.inline_data)}")
# After fix: only the final image part is returned
# [0] thought=None, has_text=False, has_image=True

…Vertex AI

On Vertex AI, ThinkingConfig(include_thoughts=False) does not suppress
thought parts from the API response for image generation models like
gemini-3.1-flash-image-preview. The part.thought flag is correctly set
to True on thought parts by the API, so client-side filtering is safe
and reliable.

This change adds _filter_thought_parts(), called in all four response
paths (sync/async, unary/streaming), which removes parts where
part.thought is True when include_thoughts is explicitly False.
The filter is a strict opt-in: it only activates when include_thoughts
is set to False, leaving all other configurations unchanged.

Fixes: googleapis#2239
@google-cla
Copy link
Copy Markdown

google-cla bot commented Apr 7, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@Venkaiahbabuneelam Venkaiahbabuneelam self-assigned this Apr 8, 2026
@Venkaiahbabuneelam Venkaiahbabuneelam added the size:XL Code changes > 100 lines label Apr 8, 2026
@Venkaiahbabuneelam
Copy link
Copy Markdown

Hi @nemetron, Thanks for reaching out us!

I noticed some checks have failed. Kindly resolve the conflicts.

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

include_thoughts=False does not suppress thought parts for image gen models on Vertex AI

2 participants