Create example for cached tokens and share with Slack channel.

[antonpirker]

This should check if the cost calculation for cached and reasoning tokens work. This will also give example traces for the engineers implementing the cost calculation.

[antonpirker]

I have an agent that makes two requests to the LLM:

• Chat Request 1: https://telemetry-experience.sentry.io/insights/agents/?project=4509450749345872&span=93f5d6[…]statsPeriod=1h&trace=0385f1dcbbb74b7e9dfa97b4e5bd03e6
  • this request has 1047 tokens in total (NONE cached) and a total cost of: 0.0001638
• Chat Request 2: https://telemetry-experience.sentry.io/insights/agents/?project=4509450749345872&span=80123c[…]statsPeriod=1h&trace=0385f1dcbbb74b7e9dfa97b4e5bd03e6
  • this request has 1070 tokens in total (936 cached) and a total cost of: 0.000237

[antonpirker]

Screenshot Chat Request 1:

[antonpirker]

Screenshot Chat Request 2:

[antonpirker]

The cached tokens are added to the spans, but I think the cost calculation is off. (will be fixed in another issue)

[antonpirker]

I have now also updated my test project to include an agent that does reasoning:
https://github.com/antonpirker/testing-sentry/tree/main/test-openai-agent

[antonpirker]

This now creates spans with reasoning tokens attached:
https://telemetry-experience.sentry.io/insights/agents/?project=4509450749345872&span=8bc74054b26d1670&statsPeriod=14d&trace=322b1db8fb4a45e9b8eeffc1b15a234f

[antonpirker]

Screenshot of reasoning token span:

[antonpirker]

All the examples are in place and the people working on cost calculation are informed. So this issue is done.