introduce cached tokens to usage

[agoddijn-fern]

It's useful to understand whether (and how) we are leveraging prompt caching in different models.
So far I've looked at OpenAI, Anthropic, and Gemini

https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching#how-prompt-caching-works
https://platform.openai.com/docs/guides/prompt-caching
https://ai.google.dev/gemini-api/docs/caching?lang=python

[alexmojaki]

Thanks, this looks good, especially since you seem to have fixed how we measured input tokens in Anthropic. If you run pytest --inline-snapshot=fix that should update the tests so that they pass.

[agoddijn-fern]

Note for this, a potential next step would be to implement some of the different caching strategies mentioned in

https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching#optimizing-for-different-use-cases

I'm not sure what the most ergonomic way of doing this would be

[DouweM]

This is being implemented in #1549!