Skip to content

Account for Gemini reasoning tokens in usage details#551

Open
PratikDhanave wants to merge 1 commit into
microsoft:mainfrom
PratikDhanave:fix-gemini-reasoning-token-count
Open

Account for Gemini reasoning tokens in usage details#551
PratikDhanave wants to merge 1 commit into
microsoft:mainfrom
PratikDhanave:fix-gemini-reasoning-token-count

Conversation

@PratikDhanave

Copy link
Copy Markdown
Contributor

When converting Gemini usage metadata, toUsageDetails mapped four fields but dropped ThoughtsTokenCount entirely:

return message.UsageDetails{
    InputTokenCount:       int64(u.PromptTokenCount),
    OutputTokenCount:      int64(u.CandidatesTokenCount),
    TotalTokenCount:       int64(u.TotalTokenCount),
    CachedInputTokenCount: int64(u.CachedContentTokenCount),
}

Per the genai docs, TotalTokenCount is "the sum of prompt_token_count, candidates_token_count, tool_use_prompt_token_count, and thoughts_token_count" — so thoughtsTokenCount is a separate bucket, not included in candidatesTokenCount. For a thinking model, those reasoning tokens were silently lost: InputTokenCount + OutputTokenCount + ReasoningTokenCount no longer reconciled with TotalTokenCount, and the framework’s dedicated UsageDetails.ReasoningTokenCount field (summed by UsageDetails.Add) stayed 0.

Both OpenAI providers already populate ReasoningTokenCount (chat.go, responses.go); Gemini was the outlier.

Fix

    CachedInputTokenCount: int64(u.CachedContentTokenCount),
    ReasoningTokenCount:   int64(u.ThoughtsTokenCount),

Test

TestUsageContent_ReasoningTokens returns a response with thoughtsTokenCount: 8 (prompt 10, candidates 5, total 23) and asserts ReasoningTokenCount == 8. It fails on the old code (0) and passes with the fix; the record now reconciles (10 + 5 + 8 = 23).

@PratikDhanave
PratikDhanave requested a review from a team as a code owner July 19, 2026 10:35
Copilot AI review requested due to automatic review settings July 19, 2026 10:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request updates the Gemini provider’s usage metadata conversion to correctly preserve Gemini “reasoning” (thoughts) tokens in the framework’s UsageDetails, aligning Gemini with the existing OpenAI providers’ behavior.

Changes:

  • Map Gemini ThoughtsTokenCount into message.UsageDetails.ReasoningTokenCount in toUsageDetails.
  • Add a regression test ensuring thoughtsTokenCount is surfaced and reconciles with totalTokenCount.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
provider/geminiprovider/agent.go Populate UsageDetails.ReasoningTokenCount from Gemini usage metadata (ThoughtsTokenCount).
provider/geminiprovider/agent_test.go Add test coverage verifying reasoning tokens are not dropped for Gemini responses.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

toUsageDetails dropped GenerateContentResponseUsageMetadata.ThoughtsTokenCount
entirely. Per the genai docs, thoughtsTokenCount is a distinct bucket within
totalTokenCount (alongside prompt, candidates, and tool-use prompt tokens), so
for a thinking model the reasoning tokens vanished from accounting: input +
output + reasoning no longer reconciled with the total, and the framework's
dedicated ReasoningTokenCount field stayed zero.

Map ThoughtsTokenCount to UsageDetails.ReasoningTokenCount, matching the
OpenAI chat and Responses providers.
@PratikDhanave
PratikDhanave force-pushed the fix-gemini-reasoning-token-count branch from 77ad0cb to 49f0779 Compare July 23, 2026 15:44
@github-actions

Copy link
Copy Markdown
Contributor

Parity review complete - no issues found. See full analysis below.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by Go API Consistency Review Agent · 37.2 AIC · ⌖ 4.82 AIC · ⊞ 5.7K ·

@github-actions github-actions Bot added the parity-approved Go API consistency review found no parity issues label Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parity-approved Go API consistency review found no parity issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants