Structured Output Returns None When max_output_tokens Exceeded

[vm7608]

Description

When using the Gemini API with structured output (i.e., response_mime_type and response_schema are set), if the generated content exceeds the max_output_tokens limit, the API response returns None for both response.text and response.parsed. This behavior prevents any further processing or continuation of the generation.

Reproduction Steps

1. Environment:

• Python 3.9+
• google-generativeai library
• pydantic library
• rich library (for rich.print in example, not essential for bug reproduction)

2. Code Example:

import os

import rich
from google import genai
from pydantic import BaseModel

API_KEY = os.getenv("GEMINI_API_KEY")


class Recipe(BaseModel):
    recipe_name: str
    ingredients: list[str]


client = genai.Client(api_key=API_KEY)
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="List a few popular cookie recipes, and include the amounts of ingredients.",
    config={
        "response_mime_type": "application/json",
        "response_schema": list[Recipe],
        "max_output_tokens": 100,  # Set a low limit to easily trigger MAX_TOKENS
    },
)
# Use the response as a JSON string.
print(response.text)  # Expected: partial JSON string, Actual: None

# Use instantiated objects.
my_recipes: list[Recipe] = response.parsed  # Expected: partial parsed object, Actual: None

rich.print(response)

3. Observed Behavior:
The GenerateContentResponse object shows finish_reason='MAX_TOKENS', but content=Content(parts=None, role='model') and parsed=None. The response.text and response.parsed attributes are both None.

Example rich.print(response) output:

GenerateContentResponse(
    candidates=[
        Candidate(
            content=Content(parts=None, role='model'),
            citation_metadata=None,
            finish_message=None,
            token_count=None,
            finish_reason=<FinishReason.MAX_TOKENS: 'MAX_TOKENS'>,
            avg_logprobs=None,
            grounding_metadata=None,
            index=0,
            logprobs_result=None,
            safety_ratings=None
        )
    ],
    create_time=None,
    response_id=None,
    model_version='gemini-2.5-flash',
    prompt_feedback=None,
    usage_metadata=GenerateContentResponseUsageMetadata(
        cache_tokens_details=None,
        cached_content_token_count=None,
        candidates_token_count=None,
        candidates_tokens_details=None,
        prompt_token_count=15,
        prompt_tokens_details=[ModalityTokenCount(modality=<MediaModality.TEXT: 'TEXT'>, token_count=15)],
        thoughts_token_count=99,
        tool_use_prompt_token_count=None,
        tool_use_prompt_tokens_details=None,
        total_token_count=114,
        traffic_type=None
    ),
    automatic_function_calling_history=[],
    parsed=None
)

Expected Behavior

When the max_output_tokens limit is reached during structured output generation, the model should ideally return the partially generated content in response.text (as a partial JSON string). This would allow developers to implement strategies for continuing the generation in a subsequent request or gracefully handle incomplete outputs.

Proposed Solution/Improvement
It would be highly beneficial if the Gemini API could provide the incomplete JSON string in response.text when MAX_TOKENS is the finish_reason. This enhancement would enable more robust error handling and allow for techniques like "continuation" or "chunking" for very large structured outputs.

[dev7608]

Same issue when I call Gemini via OpenAI SDK!

[janasangeetha]

Hey @vm7608
Thank you for reaching out. We are aware of this issue. We’ll keep you updated once we have more information.
Appreciate your patience!

[den1251]

Ability to separately set max thinking and max output text tokens would be great.

[alberthli]

Any update on this?

[riltonfranzonee]

Any updates on this? Still facing the same problem (using AI SDK with maxOutputTokens)

[jhsansom]

Hi @janasangeetha, thank you for your attention to this bug. Do you have any updates on a potential fix? Right now, I need to either (1) increase my token budget or (2) re-prompt multiple times until Gemini just-so-happens to generate a response under the max_tokens budget. Both of these charge money for increased usage beyond what I want/need (since there is no associated 400 or 500 error that cancels the billing, even when no tokens are returned).

For additional context, here is the code I currently use to do this sort of processing. Anyone else is free to use it, but beware it will bill you up to num_tries extra times!

Click to view code

def prompt_google(conversation, model, tokenizer, temperature=0, max_tokens=256, num_tries=3, **kwargs):
    client = openai.OpenAI(
        base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
        api_key=load_user_param('google_key')
    )
    for i in range(num_tries):
        response = client.chat.completions.create(
            model=model,
            messages=conversation,
            temperature=temperature,
            max_tokens=max_tokens,
        )
        if response.usage.completion_tokens > 0:
            break
        else:
            print(f'Try {i+1}/{num_tries} resulted in zero tokens!')
            conversation.append({'role':'user', 'content':'Your last generation exceeded the token limit. Try again, but make it short!'})
    return response

[janasangeetha]

Hey Users,
We've made several improvements to structured outputs. Can you please try again with the latest model and let us know what you think?