How to expose vendor specific usage information

[markpollack]

OpenAI added a new usage field,

    "usage": {
        "prompt_tokens": 13,
        "completion_tokens": 7,
        "total_tokens": 20,
        "completion_tokens_details": {
            "reasoning_tokens": 0
        }
    },

this should be made available in some way via the org.springframework.ai.chat.metadata.Usage interface

See #1394

[dafriz]

OpenAI has a nested audio_tokens usage field in both the prompt_tokens_details and completion_tokens_details so we can't just add them all as key value pairs unless key is prefixed or similar.

https://platform.openai.com/docs/api-reference/chat/object

[ThomasVitale]

The prompt_tokens_details and completion_tokens_details.audio_tokens components are being introduced in this PR: #1560 I haven't propagated them up due to the challenges covered in this issue.

Having reasoning_tokens at the top level makes it challenging to support the other fields, especially audio_tokens which is present both for prompt and completion.

[markpollack]

The design suggestion currently in play is to add a Object getNativeUsage() method on the Usage interface and that would expose either the Usage object in our custom implemented client api as with Anthropic/OpenAI or the object from the vendors third party library. The user/developer would have to cast to the type of the object to access all the details. There doesn't seem to be much commonality across vendors in this space, so one can only do so much.