Skip to content

[Feature]: capture Anthropic extended-cache TTL split (5m/1h) on audit + OTel span #442

Description

@initializ-mk

Goal

Parse and surface Anthropic's extended-cache TTL breakdown — the 5-minute vs 1-hour ephemeral split — on both the llm_call audit event and the OTel LLM span, extending the flat cache-token capture from #431/#432 (audit) and #441 (span).

Background

Forge currently captures only the flat prompt-cache counts (forge-core/llm/providers/anthropic.go:348-349):

CacheReadInputTokens     int `json:"cache_read_input_tokens"`
CacheCreationInputTokens int `json:"cache_creation_input_tokens"`

Anthropic also returns a nested breakdown of the creation tokens by cache TTL (the extended ttl: "1h" cache-control option splits writes into 5-minute and 1-hour tiers, billed differently):

"usage": {
    "input_tokens": 27,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0,
    "cache_creation": {
        "ephemeral_5m_input_tokens": 0,
        "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 5
}

cache_creation.* is not parsed at all today, so the 5m/1h split is invisible in both audit and traces. cache_creation_input_tokens (the flat total) equals ephemeral_5m_input_tokens + ephemeral_1h_input_tokens, but the tiers bill at different rates, so the split matters for cost attribution.

Scope

Mirror the flat-cache plumbing established by #431/#432/#441, one layer at a time:

  1. Parse (forge-core/llm/providers/anthropic.go): add a nested struct to anthropicResponse.Usage:
    CacheCreation struct {
        Ephemeral5mInputTokens int `json:"ephemeral_5m_input_tokens"`
        Ephemeral1hInputTokens int `json:"ephemeral_1h_input_tokens"`
    } `json:"cache_creation"`
  2. Types (forge-core/llm/types.go UsageInfo, forge-core/runtime/audit.go LLMUsage): add CacheCreation5mInputTokens / CacheCreation1hInputTokens fields (omitempty), threaded like the existing flat cache fields.
  3. Audit event (forge-core/runtime/audit.go EmitLLMCall + AuditEvent): emit cache_creation_5m_input_tokens / cache_creation_1h_input_tokens (omitempty when zero), alongside the existing cache_creation_input_tokens.
  4. OTel span (forge-core/runtime/loop.go + forge-core/observability/attrs.go): add gen_ai.usage.cache_creation_5m_input_tokens / gen_ai.usage.cache_creation_1h_input_tokens attribute constants and stamp them when non-zero (following the LLM span drops prompt-cache token attributes (cache_read/creation not on gen_ai span; diverges from llm_call audit) #441 pattern).
  5. Accumulator (forge-core/runtime/usage_accumulator.go): if it's useful to aggregate per-run, sum the tiers; otherwise leave the run aggregate to the flat total. Decide during implementation.

Consistency invariant

cache_creation_5m_input_tokens + cache_creation_1h_input_tokens == cache_creation_input_tokens. Keep the flat field as the source of truth for totals; the tiers are additive detail. Providers other than Anthropic (and Anthropic without the ttl:"1h" option) leave all cache_creation tiers zero → omitempty keeps the pre-existing JSON/span shape.

Acceptance

  • A response with cache_creation.ephemeral_5m_input_tokens / ephemeral_1h_input_tokens populates the new UsageInfo fields.
  • llm_call audit event carries cache_creation_5m_input_tokens / cache_creation_1h_input_tokens (omitempty).
  • LLM span carries gen_ai.usage.cache_creation_5m_input_tokens / ..._1h_input_tokens (non-zero only).
  • Non-Anthropic / non-extended-cache calls: no new fields/attrs (shape unchanged).
  • Tests: provider parse, audit emit, span attrs.
  • Docs: docs/security/audit-logging.md token table + docs/core-concepts/observability-tracing.md span table.

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions