.Net: Empty completion response with gemini-2.5-flash-preview-05-20 #12334
Description
Describe the bug
Running the GoogleAIUsingChatCompletion test generates no output when using gemini-2.5-flash-preview-05-20.
When no PromptExecutionSettings is passed (ie, is null), a default is created with a MaxTokens of 256. This appears to be a mistake as in the implementation of ClientBase it says that null indicates that the model's own default value should be used.
With these thinking models it's possible to use up the MaxTokens in the thinking phase. The model then returns an empty response as it does not include the thinking phase in the output. The result from the test is no output and with a metadata "FinishReason" => "MaxTokens".
The fix is probably removing this 256 default minimum and allowing it to be null in order to use the model's own default max tokens.
Platform
- Language: C#
- Source: main branch
- AI model:
- IDE: Visual Studio
- OS: Windows
Metadata
Metadata
Assignees
Type
Projects
Status
Milestone
Relationships
Participants
Activity
dmm-l-mediehus commentedon Jun 2, 2025
This is a known Gemini issue.
https://discuss.ai.google.dev/t/finishreason-max-tokens-but-text-is-empty/81874/3
drch- commentedon Jun 2, 2025
@dmm-l-mediehus yes I understand the behaviour, but the point is that the hard-coded default of 256 in SK basically guarantees that the user will hit this. I suggest the better experience would be to leave this as null, which will then use the model's own defaults.
markwallace-microsoft commentedon Jun 3, 2025
@drch- Thanks for creating the issue, would you be interested in contributing the fix?
.Net: Making Gemini MaxTokens Optional when not provided (#12539)