fix(spend): estimate fallback for zero-valued stream usage + Fireworks GLM-5.2 docs - #454
Merged
Merged
Conversation
…lback Some OpenAI-compatible providers (e.g. Fireworks) emit usage objects in streaming responses with all token counts at zero. Those zeros set saw_usage and were taken as authoritative, so the estimate-mode local fallback never fired and genuinely consumed tokens were billed as $0. Only non-zero token counts now mark usage as seen, letting zero-only streams fall through to the local estimate in estimate mode while api mode still bills nothing, matching the non-streaming path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…olders Adds a Fireworks section to the providers how-to plus a complete fireworks-glm52.toml example profile (OpenAI-compatible endpoint, estimate token counting since Fireworks streams zero-valued usage). Example configs now use $ENV_VAR placeholders instead of literal your-key-here strings so they are copy-paste safe. Local grob-*.toml scratch configs are gitignored. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Same anyhow 1.0.103 / quinn-proto 0.11.15 content as main, so the pre-push cargo-deny advisories gate passes on this branch; merges cleanly as the files are identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Mutation testing (PR diff sample)Informational — never blocks merge. Full matrix runs on main.
Legend: clean (no survivors), missed (inspect artifact), timed-out (25 min cap reached). Artifact: mutants-pr-results-2dc7f2081661e98abb006341ccb2323875001794. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two related changes from the Fireworks GLM-5.2 integration work:
Fix — zero-valued streaming usage billed as $0 (
src/server/dispatch/spend_stream.rs)Some OpenAI-compatible providers (Fireworks among them) emit streaming
usageobjects with all token counts at zero. Those zeros setsaw_usageand were treated as authoritative, so the estimate-mode local fallback never fired and genuinely consumed tokens were billed as $0. Only non-zero counts now mark usage as seen:estimatemode;apimode still bills nothing, matching the non-streaming path;zero_usage_with_text_falls_back_to_estimate.docs/reference/configuration.mdupdated to describe the zero-usage case.Docs — Fireworks GLM-5.2 provider profile
docs/how-to/providers.md(OpenAI-compatible endpoint, full model ids).docs/examples/fireworks-glm52.toml(token_counting = "estimate"since Fireworks streams zero-valued usage).$ENV_VARplaceholders instead ofyour-key-hereliterals; localgrob-*.tomlscratch configs gitignored.The
Cargo.lockcommit only syncs the RUSTSEC bumps already merged via #452 (identical content, trivial merge).Test plan
cargo test --lib spend_stream— 22 passed, including the new zero-usage regression test🤖 Generated with Claude Code