Bedrock token usage

[themrzmaster]

Describe the feature

I can not find a way of getting the token usage for a given request using Claude.

The token count is an important metric to monitor and log for LLM usage.

Use Case

Calculating cost of a request, limit usage, etc.

Proposed Solution

No response

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

SDK version used

boto3-1.28.57 botocore-1.31.57

Environment details (OS name and version, etc.)

linux

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[mrtj]

@themrzmaster could you tell us how is this feature implemented/completed?

[themrzmaster]

I used claude sdk https://github.com/anthropics/anthropic-bedrock-python

[isunli]

Should reopen this issue since Bedrock supports other foundation models, we need an aws api to get accurate token usage.

[krrishdholakia]

@isunli the response headers return the usage - just get it from there