fix: accumulate ChatUsage across Anthropic streaming events - #2467
fix: accumulate ChatUsage across Anthropic streaming events#2467waterWang wants to merge 1 commit into
Conversation
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
oss-maintainer
left a comment
There was a problem hiding this comment.
Code Review — PR #2467
fix: accumulate ChatUsage across Anthropic streaming events
Fixes #2094
Summary
Fixes ChatUsage being null in streaming responses from AnthropicChatModel by accumulating token usage from message_start (input tokens) and message_delta (output tokens) events.
Analysis
The approach is correct:
parseStreamEvent()already extracts usage from individual eventsparseStreamEvents()now uses a mutableChatUsage[]accumulator to merge usage across events- Content-bearing events get the accumulated usage attached before emission
The merge logic handles the case where message_start provides inputTokens first, then message_delta adds outputTokens, and subsequent content events carry the full accumulated usage.
Observations
- ✅ Correctly identifies that
message_startandmessage_deltahave empty content blocks and are filtered out - ✅ Accumulator pattern is appropriate for the stateful nature of streaming usage
- ✅ Fixes a real user-facing issue (no token usage tracking for Anthropic streaming)
- ❌ CI failed — Spotless formatting in
AnthropicResponseParser.java. Runmvn spotless:applyto fix. ⚠️ Minor: theChatUsage[]array as mutable accumulator works but could be replaced withAtomicReference<ChatUsage>for clarity in concurrent contexts (thoughflatMaphere is sequential per-event, so the array is fine).
Required Action
@waterWang Please run mvn spotless:apply in the agentscope-extensions-model-anthropic module and push the fix.
Verdict
Request Changes — logic is correct, but Spotless formatting must be fixed before merge.
Fixes #2094
Problem
When
AnthropicChatModelis used with streaming enabled, everyChatResponsehasusage == null. The Anthropic streaming protocol sends token usage inmessage_start(input_tokens) andmessage_delta(output_tokens) events, but these events have empty content blocks and are filtered out by the.filter(response -> !response.getContent().isEmpty())inparseStreamEvents().Fix
parseStreamEvent(): ParseinputTokensfrom themessage_startevent'smessage.usage.input_tokensand create aChatUsageobject.parseStreamEvents(): Use a mutable accumulator array to merge usage across events:message_start→ setsinputTokensmessage_delta→ setsoutputTokenscontent_block_delta→ attaches the accumulated usage when content is emittedThis ensures that every content-bearing event carries the complete token usage (input + output tokens), while the filter still correctly removes empty events.