Skip to content

Gemini thoughts not correctly accumulated when streaming enabled #510

@OwenDavisBC

Description

@OwenDavisBC

Description
When using gemini models with thinking enabled and thoughts turned on, the thoughts are being incorrectly appended to non-thought text responses from gemini when streaming is enabled. The python-adk does not exhibit the same behavior of combing thoughts and non-thought text responses.

The python-adk explicitly builds thought text responses separately from non-thought response to avoid this issue. See - https://github.com/google/adk-python/blob/main/src/google/adk/utils/streaming_utils.py#L51-L82

It can also be seen in the python-adk, when building llm requests, that thoughts are not removed. See the python-adk llm request pre-processing - https://github.com/google/adk-python/blob/9be9cc2feee92241fd2fbf9dea3a42de5a78e9ce/src/google/adk/models/google_llm.py#L291

To Reproduce
Steps to reproduce the behavior:

  1. Setup an agent with thinking enabled, for example the hello world agent:
LlmAgent.builder()
  .name("hello_world_agent")
  .description("hello world agent that can roll a dice and check prime numbers.")
  .model("gemini-2.5-flash")
  .instruction(
      """
      You roll dice and answer questions about the outcome of the dice rolls.
      You can roll dice of different sizes.
      You can use multiple tools in parallel by calling functions in parallel(in one request and in one round).
      It is ok to discuss previous dice roles, and comment on the dice rolls.
      When you are asked to roll a die, you must call the roll_die tool with the number of sides. Be sure to pass in an integer. Do not pass in a string.
      You should never roll a die on your own.
      When checking prime numbers, call the check_prime tool with a list of integers. Be sure to pass in a list of integers. You should never pass in a string.
      You should not check prime numbers before calling the tool.
      When you are asked to roll a die and check prime numbers, you should always make the following two function calls:
      1. You should first call the roll_die tool to get a roll. Wait for the function response before calling the check_prime tool.
      2. After you get the function response from roll_die tool, you should call the check_prime tool with the roll_die result.
        2.1 If user asks you to check primes based on previous rolls, make sure you include the previous rolls in the list.
      3. When you respond, you must include the roll_die result from step 1.
      You should always perform the previous 3 steps when asking for a roll and checking prime numbers.
      You should not rely on the previous history on prime results.
      """)
  .tools(
      ImmutableList.of(
          FunctionTool.create(HelloWorldAgent.class, "rollDie"),
          FunctionTool.create(HelloWorldAgent.class, "checkPrime")))
  .generateContentConfig(
      GenerateContentConfig.builder()
          .safetySettings(
              SafetySetting.builder()
                  .category(Known.HARM_CATEGORY_DANGEROUS_CONTENT)
                  .threshold(HarmBlockThreshold.Known.OFF)
                  .build())
          .thinkingConfig(ThinkingConfig.builder().includeThoughts(true).build()) // Including thoughts is the important part
          .build())
  .build();
  1. Run the adk web server for the agent (ie. mvn exec:java -Dexec.mainClass="com.google.adk.web.AdkWebServer" -Dexec.args="--adk.agents.source-dir=src/main/java" -Dexec.classpathScope="compile")
  2. Enable Token Streaming at the top right of the adk webpage
  3. Prompt the agent
  4. See an incorrectly combined thought + non-thought response after a partial thought

Expected behavior
Align the java-adk with the python-adk by NOT combining thought and non-thought text responses. This also includes NOT removing thoughts from LLM requests.

Screenshots
Example of the issue:
Image

Example from python-adk:
Image

Desktop:

  • OS: macOS Sequoia 15.6.1
  • Java version: Microsoft build of OpenJDK 21.0.8
  • ADK version: 0.3.1, tested on main as of commit 84e755c

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions