Add thinking_budget parameter to ChatGoogleGenerativeAI to support Gemini 2.5 Flash

[khoa1337]

The new release of Gemini 2.5 Flash has introduced a new parameter to Google's SDK, as follows:

prompt = "Explain the Occam's Razor concept and provide everyday examples of it"
budget = 1024 # You can set this variable to any value between 0 and 24k

client = genai.Client(api_key="your-api-key")

response = client.models.generate_content(
model="gemini-2.5-flash-preview-04-17",
contents=prompt,
config=types.GenerateContentConfig(
thinking_config=types.ThinkingConfig(
thinking_budget=budget
)
)
)

print(response.text)

As seen in https://ai.google.dev/gemini-api/docs/thinking#set-budget

[pseudotensor]

I tried adding manually, but even in adaptive mode (no thinking parameter passed) the return response object does not contain thoughts_token_count attribute.

i.e. this never is filled:

        input_tokens = response.usage_metadata.prompt_token_count
        output_tokens = response.usage_metadata.candidates_token_count
        total_tokens = response.usage_metadata.total_token_count
        cache_read_tokens = response.usage_metadata.cached_content_token_count
        has_thoughts = hasattr(response.usage_metadata, "thoughts_token_count")
        if has_thoughts:
            print("Gemini thoughts token count is available")
        thoughts_token_count = response.usage_metadata.thoughts_token_count if has_thoughts else 0

My guess is that the langchain google package needs to start using the new non-deprecated google package not generativeai package.

For reference also see: https://github.com/google-gemini/cookbook/blob/main/quickstarts/Get_started_thinking.ipynb

[pseudotensor]

Note this works fine:

import os

GOOGLE_API_KEY = os.getenv('GOOGLE_API_KEY')
from google import genai
from google.genai import types

client = genai.Client(api_key=GOOGLE_API_KEY)

MODEL_ID="gemini-2.5-flash-preview-04-17" # @param ["gemini-2.5-flash-preview-04-17", "gemini-2.5-pro-preview-03-25"] {"allow-input":true, isTemplate: true}
prompt = "Explain the concept of Occam's Razor and provide a simple, everyday example."
response = client.models.generate_content(
    model=MODEL_ID,
    contents=prompt
)

print(response.text)
print(response.usage_metadata.prompt_token_count)
print(response.usage_metadata.candidates_token_count)
print(response.usage_metadata.thoughts_token_count)

gives:

Okay, let's break down Occam's Razor.

**Concept of Occam's Razor:**

Occam's Razor is a principle that suggests when you have **competing explanations** for something, the **simplest one** – the one that requires the **fewest assumptions** – is usually the **best** or most likely one to be true.

It's not a rule of logic or a scientific law proving something is true. Instead, it's a **heuristic** or a **rule of thumb** used for choosing among explanations that fit the available facts. It basically tells you: **Don't invent unnecessary complexity** when a simpler explanation will do.

It's also sometimes called the "Principle of Parsimony" (parsimony means being frugal or simple). It's named after William of Ockham, a 14th-century friar and philosopher.

**In simple terms:** If there are two ways to explain something, pick the one that assumes less weird stuff is going on.

**Simple, Everyday Example:**

Imagine you walk into your kitchen and see a glass lying on the floor, broken.

*   **Explanation 1 (Complex):** A ghost came into the kitchen, picked up the glass with invisible hands, and intentionally threw it on the floor to scare you.
*   **Explanation 2 (Simple):** The glass was on the edge of the counter, and your cat jumped onto the counter, accidentally knocking it off.

According to Occam's Razor, Explanation 2 (the cat) is much more likely.

*   **Why?**
    *   Explanation 1 requires assuming the existence of ghosts, their ability to interact with physical objects, and their motivation to break glasses. (Many assumptions).
    *   Explanation 2 requires assuming you own a cat, cats sometimes jump on counters, and objects on edges can fall. (Fewer, more common and well-understood assumptions).

While it's *possible* a ghost broke the glass, Occam's Razor points you towards the simpler, more probable explanation first – that it was knocked over by the cat. You would look for evidence of the cat being around (or just assume it, if you have a cat) before seriously considering the ghost theory.
18
1564
1083

[Cadence-GitHub]

My guess is that the langchain google package needs to start using the new non-deprecated google package not generativeai package.

This! I now have conflicts between the dependencies of the langchain package and the new google's package. I wanted to have both installed to control the thinking tokens using google's API for this particular model, and use langchain for all the other models. However, I was not able to do that due to some conflicts.

I hope the control over thinking tokens can be added to the langchain package soon. It's unfortunate that google didn't just name the thinking model differently as they did for Gemini 2.0. I suppose their marketing decided that they need to be named the same, even though they are obviously different models with wildly different pricing.

[Syazvinski]

+1

[LeonMusCoden]

+1

[windkit]

I have looked into the issue a bit, it seems one blocker is the GenerationConfig in google/ai/generativelanguage_v1beta does not have ThinkingConfig

I believe we have to wait for next release before we can add the support here Leonid Kuligin (@lkuligin)

[Meet-Workik]

+1

[ViktorJJF]

+1

[lgesuellip]

Hey everyone, is anyone currently working on this? Windkit Li (@windkit), which release are you working on?

[rekram1-node]

+1

[windkit]

A progress update, after manually adding the corresponding proto fields in google/ai/generativelanguage_v1beta a small PR here should work

model: "models/gemini-2.5-flash-preview-04-17"
contents {
  parts {
    text: "How many O\'s are in Google? Please tell me how you double checked the result"
  }
  role: "user"
}
generation_config {
  candidate_count: 1
  temperature: 0.7
  thinking_config {
    thinking_budget: 100
  }
}

candidates {
  content {
    parts {
      text: "There are **two** O\'s in the word \"Google\".\n\nHere\'s how I double-checked:\n\n1.  **First Pass:** I looked at the word \"Google\" and scanned through it letter by letter, counting the \'O\'s: G - o (1) - o (2) - g - l - e. I counted two.\n2.  **Second Pass (Verification):** I looked at the word \"Google\" again, specifically focusing *only* on the \'O\'s. I visually located the first \'o\' and the second \'o\'. There are no other \'O\'s in the word. This confirms the count of two."
    }
    role: "model"
  }
  finish_reason: STOP
  index: 0
}
usage_metadata {
  prompt_token_count: 19
  candidates_token_count: 252
  total_token_count: 271
  thoughts_token_count: 109
}
model_version: "models/gemini-2.5-flash-preview-04-17"

model: "models/gemini-2.5-flash-preview-04-17"
contents {
  parts {
    text: "How many O\'s are in Google? Please tell me how you double checked the result"
  }
  role: "user"
}
generation_config {
  candidate_count: 1
  temperature: 0.7
  thinking_config {
    thinking_budget: 0
  }
}

candidates {
  content {
    parts {
      text: "There are **two** O\'s in Google.\n\nHere\'s how I double-checked the result:\n\n1. **Visual Scan:** I looked at the word \"Google\" and counted the letters. I specifically focused on identifying the letter \"O\".\n2. **Letter by Letter Analysis:** I mentally or physically went through each letter of \"Google\" one by one and noted when I encountered an \"O\":\n    * G - Not an O\n    * o - This is an O (count 1)\n    * o - This is an O (count 2)\n    * g - Not an O\n    * l - Not an O\n    * e - Not an O\n3. **Confirmation:** Both methods consistently resulted in a count of two O\'s.\n\nThis simple word allows for straightforward double-checking through direct observation and counting."
    }
    role: "model"
  }
  finish_reason: STOP
  index: 0
}
usage_metadata {
  prompt_token_count: 19
  candidates_token_count: 181
  total_token_count: 200
}
model_version: "models/gemini-2.5-flash-preview-04-17"

I will see how to proceed from here.

[bdjafer]

+1