fix(ai): declare gemini cache reporting as inclusive on generations - #4473
Merged
Conversation
Gemini counts `cachedContentTokenCount` inside `promptTokenCount`, but the SDK never said so, leaving ingestion to infer the accounting model from the token counts alone. That inference is unreliable here. Under explicit context caching the two counts come from separate measurements, the cache object at creation time and the prompt per request, so they can disagree by a few percent and the cache pool can land just above the input total. Set `cacheReportingExclusive` to false on generations that report cache reads, on both the streaming and non-streaming paths, and map it onto `$ai_cache_reporting_exclusive`. The property mapping checks against undefined rather than truthiness, because false is the meaningful value here. Mirrors PostHog/posthog-python#860. Generated-By: PostHog Code Task-Id: 06160e48-feb9-4d39-8b7b-3dcfd1d9ca24
Radu-Raicea
marked this pull request as ready for review
August 7, 2026 17:59
Contributor
Contributor
|
Size Change: +13.9 kB (+0.07%) Total Size: 19.1 MB 📦 View Changed
ℹ️ View Unchanged
|
…ridden The derived flag describes how the SDK's own input and cache counts relate to each other. When a caller passes their own token counts through `posthogProperties`, those override the derived counts but the flag survived alongside them, so it could describe numbers that are no longer on the event. That is wrong in the expensive direction: declaring inclusive over counts that are actually exclusive makes ingestion subtract a cache pool that was never part of the input. Suppress the derived flag whenever the caller overrides any token count. A caller who knows their own accounting model can still pass `$ai_cache_reporting_exclusive` explicitly, and that value wins. Extracts the passthrough check behind `hasTokenOverrides`, which `getTokensSource` now shares. Generated-By: PostHog Code Task-Id: 06160e48-feb9-4d39-8b7b-3dcfd1d9ca24
Contributor
Author
|
Added in cf73d3e. The derived flag is now suppressed whenever the caller overrides any token count through A caller who knows their own accounting model can still pass Two tests added: the flag is absent when token counts are overridden, and an explicitly-passed flag survives. Worth noting the same hazard exists in posthog-python#860, where |
Radu-Raicea
approved these changes
Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Cache-heavy Gemini generations can be priced wrong in AI observability, because the SDK never tells ingestion how Gemini counts cache tokens.
Gemini counts
cachedContentTokenCountinsidepromptTokenCount, so cache reads are a subset of input, not a separate pool. The SDK reports both numbers but declares nothing, so ingestion infers the accounting model from the token counts.That inference is unreliable for Gemini. Under explicit context caching the two counts come from separate measurements: the cache object is counted when it is created, and the prompt is counted per request. They describe the same tokens but can disagree by a few percent, which puts the cache pool just above the input total and makes the counts look like two separate pools. Ingestion then bills the full input at the prompt rate and the cache again at the cache rate, so a nearly fully cached prompt bills several times over.
This is the JS counterpart of PostHog/posthog-python#860.
Changes
TokenUsagegainscacheReportingExclusive?: boolean, left undefined when a provider's accounting model is not known.captureAiGenerationmaps it onto$ai_cache_reporting_exclusive.The property mapping checks
!== undefinedrather than truthiness. Every neighbouring field inadditionalTokenValuesuses a truthiness guard, which is correct for counts but would silently dropfalse, the only value this field ever carries.The flag is set only when cache reads are present, so generations without caching do not carry an extra property.
Release info Sub-libraries affected
Libraries affected
Checklist
How did you test this code?
npx jestinpackages/ai. 764 passed. Two failures intests/openai-agents-resolution.test.tsare unrelated and fail the same way on a clean checkout.tests/gemini.test.ts. The first asserts$ai_cache_reporting_exclusiveisfalsewhen cached tokens are present, which fails if the flag stops being set or if the property mapping reverts to a truthiness guard. The second asserts the property is absent when nothing was cached, which guards against sending it on every generation.npx tsc --noEmit. The only errors are missingposthog-nodedeclarations, caused by declaration-file generation failing in this sandbox. No errors in the changed files.🤖 Agent context
Autonomy: Human-driven (agent-assisted)
I (the PostHog Slack app, running Claude) traced this from a Slack thread on Gemini cost reconciliation, and a PostHog engineer directed the fix. I did not set an assignee because I could not verify their GitHub handle in session.
The Python change landed first and needed the flag threaded through three separate tagging sites. The JS side has one central property mapper, so this diff is smaller, and the only real trap was the truthiness guard described above.
Public artifact: no customer or session material reached this PR. The token values in the new tests are invented.
Created with PostHog from a Slack thread