Replies: 1 comment
|
This is a really good example of why cost observability probably needs to happen at the request boundary rather than only at the client/session level. One thing I'd want to capture per request is: provider + model Then reconcile that against the provider invoice separately. The interesting part here isn't just that the numbers are wrong. An 8x discrepancy can stay invisible for days if the telemetry layer itself is the thing being trusted for cost reporting. We're working on this problem in ModelGate from the proxy/gateway side, so this case is especially interesting. The long-context threshold and cache-write distinction look like exactly the kind of metadata that needs to survive all the way through the request trace. |
Uh oh!
There was an error while loading. Please reload this page.
Issue creation is currently restricted in this repository, so I am posting this as a discussion and asking a maintainer to triage it as a bug report.
Environment
Observed AWS billing
For the same three-day period, AWS Cost Explorer reports approximately:
Observed local telemetry
Codex JSONL session telemetry and ccusage produce a much lower estimate.
The dominant Codex session alone reports approximately:
Across the matching date range, the local estimate is approximately $697,
versus $5,589.26 billed by AWS.
The discrepancy is not explained solely by the documented 2x long-context
input multiplier. AWS classifies hundreds of millions of tokens as billable
long-context input that the local Codex telemetry appears to classify as
cached input.
Questions
cached_input_tokensreturned by the provider, or calculated by Codex?pricing threshold?
All reactions