Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 27, 2026

  • Update OpenTelemetryConsts.cs to add new constant:
    • CacheReadInputTokens = "gen_ai.usage.cache_read.input_tokens" for activity span tag
  • Update OpenTelemetryChatClient.cs to:
    • Add CachedInputTokenCount as an activity span tag using gen_ai.usage.cache_read.input_tokens
  • Update tests in OpenTelemetryChatClientTests.cs:
    • Added CachedInputTokenCount = 5 to test usage details for both sync and streaming responses
    • Added assertion to verify gen_ai.usage.cache_read.input_tokens = 5 in activity tags
  • Verify build and tests pass (all 7 OpenTelemetryChatClient tests pass across net8.0, net9.0, net10.0)

Summary

This PR adds support for propagating CachedInputTokenCount through OpenTelemetry telemetry as an activity span tag using gen_ai.usage.cache_read.input_tokens as defined in the OpenTelemetry semantic conventions registry.

Note: The histogram metric gen_ai.client.token.usage only supports "input" and "output" token types per the semantic conventions, so cached tokens are only recorded as a span attribute, not as a separate histogram entry.

Original prompt

@open-telemetry/semantic-conventions/pull/3163 adds a new kind of token to the open telemetry that maps to UsageDetails.CachedInputTokenCount. Update the OpenTelemetryChatClient (and anything else that needs updating) to propagate this value according to the convention, just as other counts are propagated.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Microsoft Reviewers: Open in CodeFlow

…lient

Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
Copilot AI changed the title [WIP] Update OpenTelemetryChatClient for new CachedInputTokenCount Propagate CachedInputTokenCount in OpenTelemetry telemetry Jan 27, 2026
Copilot AI requested a review from stephentoub January 27, 2026 14:06
@stephentoub stephentoub marked this pull request as ready for review January 27, 2026 14:08
@stephentoub stephentoub requested a review from a team as a code owner January 27, 2026 14:08
Copilot AI review requested due to automatic review settings January 27, 2026 14:08
@stephentoub stephentoub added the area-ai Microsoft.Extensions.AI libraries label Jan 27, 2026
@stephentoub stephentoub requested a review from tarekgh January 27, 2026 14:10
@stephentoub stephentoub enabled auto-merge (squash) January 27, 2026 14:10
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds support for tracking cached input tokens in OpenTelemetry telemetry following the OpenTelemetry semantic conventions. The changes propagate UsageDetails.CachedInputTokenCount through both histogram metrics and activity span tags in the OpenTelemetryChatClient.

Changes:

  • Added OpenTelemetry constants for cache_read token type and corresponding semantic convention attribute name
  • Updated OpenTelemetryChatClient to record cached input tokens in histogram metrics and activity span tags
  • Enhanced tests to verify the new cached token telemetry functionality

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
src/Libraries/Microsoft.Extensions.AI/OpenTelemetryConsts.cs Added TokenTypeCacheRead ("cache_read") constant and CacheReadInputTokens ("gen_ai.usage.cache_read_input_tokens") attribute name following OpenTelemetry semantic conventions
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/OpenTelemetryChatClient.cs Added histogram recording for cached input tokens with token type tag and activity span tag using the new semantic convention attribute, following the same pattern as input/output tokens
test/Libraries/Microsoft.Extensions.AI.Tests/ChatCompletion/OpenTelemetryChatClientTests.cs Added CachedInputTokenCount = 5 to test usage details and corresponding assertion for the activity span tag in both streaming and non-streaming test scenarios

…ic conventions

Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
auto-merge was automatically disabled January 27, 2026 14:19

Head branch was pushed to by a user without write access

Copilot AI requested a review from stephentoub January 27, 2026 14:19
…defined in semantic conventions

Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
@stephentoub stephentoub merged commit 16ae18e into main Jan 27, 2026
6 checks passed
This was referenced Feb 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-ai Microsoft.Extensions.AI libraries

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants