bug(otel): OpenInference llm.token_count.prompt_details.cache_read/cache_write not recognized by usage extractor

[gragragrab]

Describe the bug

Langfuse's OTel ingestion processor does not recognize the canonical OpenInference attribute names for prompt-cache tokens. Specifically, the OpenInference semantic conventions define:

• llm.token_count.prompt_details.cache_read — tokens read from cache
• llm.token_count.prompt_details.cache_write — tokens written to cache

These names are emitted by every official OpenInference instrumentor that supports prompt caching: openinference-instrumentation-openai, openinference-instrumentation-anthropic, and openinference-instrumentation-agno.

In packages/shared/src/server/otel/OtelIngestionProcessor.ts (v3.169.0), the extractGenericGenAiUsageDetails method has an explicit allow-list of recognized cache-token sub-keys (lines 2530–2539, 2553–2560):

cache_read.input_tokens
cache_read_tokens
details.cache_read_tokens
details.cache_read_input_tokens
cache_creation.input_tokens
cache_write_tokens
details.cache_write_tokens
details.cache_creation_input_tokens

prompt_details.cache_read and prompt_details.cache_write are not in this list.

Effect

The cache token values do reach Langfuse — they end up in usageDetails as raw passthrough keys (e.g. prompt_details.cache_read: 40024). But because they don't match the resolver's recognized names:

1. They are NOT subtracted from input (which can lead to double-counted input on cached calls).
2. They are NOT normalized to input_cached_tokens / input_cache_creation, the canonical Langfuse internal keys.
3. They are NOT recognized by the cost engine's cache-aware pricing tiers (defined in worker/src/constants/default-model-prices.json).
4. They are NOT priced at all — the cost engine prices input at full base rate and silently skips the cache keys.

For a workload using claude-opus-4-6 with prompt caching active, this means Langfuse under-reports cost on cache-heavy observations by ~30% per call, and the Other usage section in the UI shows opaque keys instead of a structured "Input cache read / write" breakdown.

Evidence — live production data

From a 250-observation sample on a production Langfuse v3.169.0 instance (self-hosted), agent using agno==2.5.15 + openinference-instrumentation-agno:

• 28/250 observations (11.2%) had non-zero cache_read or cache_write
• usageDetails keys present: input, output, total, prompt_details.cache_read, prompt_details.cache_write
• Single-observation example: input=443, output=1806, prompt_details.cache_read=40024, prompt_details.cache_write=168
• Langfuse-calculated cost: $0.047365
• Anthropic actual cost: $0.068427 (with cache_read at $0.50/MTok, cache_write_5m at $6.25/MTok)
• Per-call under-report: 30.8% on heavily cached observations

Comparison with the established fix pattern (PR #12248)

PR #12248 fixes the same shape of bug for pydantic-ai by adding gen_ai.usage.details.cache_read_tokens / cache_write_tokens to the resolver. The fix proposed here follows the identical pattern for the OpenInference family of instrumentors.

Why this isn't an upstream-OpenInference bug

llm.token_count.prompt_details.cache_read is the documented OpenInference semantic-convention name (see openinference-semantic-conventions/src/openinference/semconv/trace/__init__.py:115). Changing it upstream would break compatibility with every other observability backend that already speaks the OpenInference convention (Arize Phoenix, Datadog, Honeycomb, etc.). The right place to fix this is in Langfuse's OTel ingestion processor.

Steps to reproduce

1. Add agno==2.5.15 and openinference-instrumentation-agno>=0.1.27,<0.2 to a Python environment, plus an OTel exporter targeting a self-hosted Langfuse (v3.169.0 or any release where the resolver allow-list does not include prompt_details.cache_*).
2. Configure agno to enable Anthropic prompt caching (e.g. cache_system_prompt=True plus a mixin that attaches cache_control to the last tool).
3. Run any agent flow that produces at least 4096 input tokens (Opus minimum for caching) and is invoked twice within 5 minutes (so the second call hits the cache).
4. Inspect the resulting Langfuse observation via the API: GET /api/public/observations?limit=5&from_start_time=...
5. Observe usageDetails keys: prompt_details.cache_read / prompt_details.cache_write appear as raw keys; input_cached_tokens and input_cache_creation are absent; input is not reduced by the cached amount; cost is computed only on input * base_rate + output * output_rate.

Langfuse Cloud or self-hosted?

Self-hosted

If self-hosted, what version are you running?

v3.169.0 (also confirmed by inspecting v3.173.0 source on main)

SDK and integration versions

• agno==2.5.15
• openinference-instrumentation-agno>=0.1.27,<0.2
• OpenTelemetry SDK as bundled by openinference-instrumentation-agno

Additional information

Proposed fix

Two-part change to packages/shared/src/server/otel/OtelIngestionProcessor.ts::extractGenericGenAiUsageDetails:

1. Add prompt_details.cache_read to the cacheReadTokens resolver chain (line 2530).
2. Add prompt_details.cache_write to the cacheCreationTokens resolver chain (line 2535).
3. Add both keys to the dedup allow-list at lines 2553–2560 so they're not double-emitted.

Plus a unit test asserting that an OpenInference-style span with llm.token_count.prompt_details.cache_read / cache_write attributes is normalized to input_cached_tokens / input_cache_creation correctly, with input reduced.

I've already implemented and tested this locally against main and will open a PR if a maintainer indicates the diagnosis is correct.

Related issues / PRs

• bug: Support cache_read_input_tokens / cache_creation_input_tokens in OTEL generation spans for accurate Anthropic cost calculation #12635 — broader "support cache_read_input_tokens / cache_creation_input_tokens in OTEL generation spans" — this PR addresses the OpenInference subset of that issue.
• #12248 — fix(otel): correct pydantic-ai v0.7.x+ cache token attribute names — same structural fix for a different upstream.
• Inflated token counts when using OpenTelemetry Anthropic instrumentor with streaming + images #12994 — inflated token counts with OpenTelemetry Anthropic instrumentor + streaming + images (related but different code path).

Are you interested in contributing a fix for this bug?

Yes

[gragragrab]

Filed PR #13572 with the proposed fix and a unit test (pattern-matched to the merged-style PR #12248 for pydantic-ai). The PR is mergeable and strictly additive — no behavior change for existing recognized aliases, only adds the canonical OpenInference prompt_details.cache_* names to the resolver. Ready for review whenever a maintainer has cycles.