Request include_usage for OpenAI chat streaming so streamed runs report token usage#609
Request include_usage for OpenAI chat streaming so streamed runs report token usage#609PratikDhanave wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the OpenAI Chat Completions streaming request path to default stream_options.include_usage=true so streamed runs receive (and can aggregate) token usage from the final stream chunk, matching the non-streaming behavior and other SDKs.
Changes:
- Default
stream_options.include_usagetotruefor streaming requests when the caller did not set it. - Preserve caller intent by not overriding an explicitly provided
include_usagevalue (includingfalse). - Extend streaming tests to assert the outbound request includes
stream_options.include_usageand to cover the caller-override case.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| provider/openaiprovider/chat.go | Sets StreamOptions.IncludeUsage to true by default in the streaming branch when unset, enabling usage reporting from the final stream chunk. |
| provider/openaiprovider/chat_test.go | Updates streaming request body expectations and adds a test ensuring a caller-supplied include_usage=false is respected. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
54284fa to
fc3dc20
Compare
The OpenAI Chat Completions API omits the final usage chunk for streamed runs unless stream_options.include_usage is set, so every streamed run reported zero token usage while the non-streaming path reported usage normally. Set stream_options.include_usage=true before starting the stream unless the caller already supplied a value, matching the token accounting the .NET and Python OpenAI chat clients get from streamed completions.
fc3dc20 to
fb4c2c8
Compare
Parity Review: ✅ ApprovedThis PR restores cross-repo parity for the OpenAI Chat Completions streaming path. Change summary: Sets Upstream alignment verified:
No new public Go API surface is introduced. No parity issues found. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
What
In the OpenAI Chat Completions streaming path, set
stream_options.include_usage=truebefore starting the stream, unless the caller already supplied a value.Why
buildCompletionParamsnever setStreamOptions.IncludeUsage, and the SDK omits the field when unset (it is aparam.Opt[bool]withomitzero). The OpenAI API only emits the final usage chunk for streamed completions whenstream_options.include_usage=true. Without it, the streaming branch'schunk.JSON.Usage.Valid()guard never fires, so every streamed run reported zero token usage — while the non-streaming path reported usage normally.This restores parity with the .NET and Python OpenAI chat clients, which enable usage reporting for streamed completions so the same token accounting is available regardless of streaming mode. The existing
addUsageaggregation already consumes the resulting usage chunk unchanged.The caller's explicit choice is respected: if
StreamOptions.IncludeUsageis already set (including tofalse), the provider does not override it. The non-streaming path is untouched.Tests
chat_test.goto assert the outbound request body now containsstream_options.include_usage=true, and that the aggregatedUsageContentcarries non-zero token counts.TestChatStreamingIncludeUsage_RespectsCallerOverride, asserting that a caller-suppliedinclude_usage=falseis not overridden.go build ./...,go vet ./provider/openaiprovider/..., andgo test ./provider/openaiprovider/...all pass.