Gemini 2.5/3 Flash thinking tokens consume maxOutputTokens, causing empty responses

[kingpanther13]

Bug Description

Gemini 2.5 Flash and Gemini 3 Flash Preview now have thinking enabled by default. The thinking tokens are counted against maxOutputTokens, which causes responses to be empty or truncated when using typical token values (e.g. 35–50).

This is a breaking change from Google's side — these models previously worked fine with low maxOutputTokens values because thinking was not enabled.

What Happens

When llmvision.video_analyzer calls Gemini 2.5 Flash with max_tokens: 50:

1. The model uses ~44–800+ tokens for internal "thinking" (counted against maxOutputTokens)
2. Only 0–6 tokens remain for actual output
3. When zero tokens are left, the API returns candidates with no content field
4. LLM Vision raises: "No content parts were returned from Google API"
5. The user receives "Couldn't generate content. Check logs for details." as their notification

This also causes the input_boolean helper (LLM busy flag) used in blueprints like sam2kb/frigate-ai-notification to get stuck, since the automation crashes before it can reset the flag.

Evidence

gemini-2.5-flash with maxOutputTokens: 50 (no thinkingConfig)

{
  "candidates": [{ "content": { "parts": [{ "text": "This security" }] }, "finishReason": "MAX_TOKENS" }],
  "usageMetadata": {
    "promptTokenCount": 282,
    "candidatesTokenCount": 2,
    "totalTokenCount": 328,
    "thoughtsTokenCount": 44
  }
}

Only 2 output tokens. With slightly more thinking overhead, zero output tokens → empty content.

gemini-3-flash-preview with maxOutputTokens: 50 (no thinkingConfig)

{
  "candidates": [{ "content": {}, "finishReason": "MAX_TOKENS" }],
  "usageMetadata": {
    "promptTokenCount": 1124,
    "totalTokenCount": 1171,
    "thoughtsTokenCount": 47
  }
}

Zero output tokens. Completely empty response.

gemini-2.5-flash with thinkingConfig: { thinkingBudget: 0 } and maxOutputTokens: 50

{
  "candidates": [{ "content": { "parts": [{ "text": "This security camera image captures a wide-angle view of a cluttered indoor space..." }] }, "finishReason": "MAX_TOKENS" }],
  "usageMetadata": {
    "promptTokenCount": 282,
    "candidatesTokenCount": 50,
    "totalTokenCount": 332
  }
}

All 50 tokens used for actual output. Works as expected.

gemini-3-flash-preview with thinkingConfig: { thinkingBudget: 0 } and maxOutputTokens: 50

{
  "candidates": [{ "content": { "parts": [{ "text": "A security camera captures a person standing in a cluttered living room and kitchen area..." }] }, "finishReason": "MAX_TOKENS" }],
  "usageMetadata": {
    "promptTokenCount": 1124,
    "candidatesTokenCount": 46,
    "totalTokenCount": 1170
  }
}

46 output tokens, no thinking overhead. Works as expected.

Suggested Fix

For Gemini 2.5+ models, add thinkingConfig to disable thinking by default in the generationConfig payload in providers.py:

payload["generationConfig"]["thinkingConfig"] = {"thinkingBudget": 0}

Ideally this could also be exposed as a user-configurable option (enable/disable thinking, set thinking budget) for users who want thinking and are willing to increase their max_tokens, but the default should be off to avoid breaking existing configurations.

Affected Code

providers.py, Google._prepare_vision_data() — the payload sent to generativelanguage.googleapis.com/v1beta does not include thinkingConfig, so Google applies its default (thinking ON for 2.5+ models).

Environment

• LLM Vision version: v1.6.0 (latest)
• Models affected: gemini-2.5-flash, gemini-3-flash-preview (likely all Gemini 2.5+ models)
• Provider: Google (direct REST API)

Workaround

Increase max_tokens to 1024+ to leave room for both thinking and output, but this increases API costs/latency unnecessarily for simple vision tasks.

[valentinfrlch]

Configurable thinking/reasoning for Google and other providers is coming next release: c38a5de

[michaelsleen]

Configurable thinking/reasoning for Google and other providers is coming next release: c38a5de

Thanks for your work on this. Any ETA for the next release to fix this issue?