Count Tokens for Bedrock

[14gaurav16]

I am getting following when I am trying to calculate tokens in bedrock.

AttributeError: 'AnthropicBedrock' object has no attribute 'count_tokens'

It seems like there is no implementation for it.

Thanks.

[rattrayalex]

That's right; we're now deprecating token-counting in the client because the tokenization algorithms in Claude 3 are different (see deprecation notes on the count_tokens() method). The new recommendation is to look at usage in the return value.

[Smuglix]

@rattrayalex will there be a way to calculate how much tokens a given text takes without making a chat completion call?

[rattrayalex]

Not at this time, unfortunately. You can continue to use count_tokens() or other third-party libraries as a rough estimate for the time being. A better solution may be added in the future.

[opyate]

or other third-party libraries as a rough estimate

Thanks for the suggestion.

Would you recommend tiktoken (and if so, cl100k_base?), sentencepiece, or the rough heuristic as per OpenAI?

I.e.

1 token ~= 4 chars in English
1 token ~= ¾ words
100 tokens ~= 75 words

[rattrayalex]

@x5a might be better positioned to answer that.