Is there support for function calls tokens

[KilianKae]

Thanks for the project. Great so far.

Is there support for counting tokens when using function calls?
https://platform.openai.com/docs/guides/gpt/function-calling

[pkoukk]

I'm sorry, open ai does not provide any explanation on how to calculate function call costs, so we currently do not know how to calculate it either.

[phillebaba]

The Java implementation of Tiktoken seems to have a util for calculating tokens for functions.
https://github.com/forestwanglin/openai-java/blob/93596f43e684b30f8712f9ba62edcef92d3f8a9b/jtokkit/src/main/java/xyz/felh/openai/jtokkit/utils/TikTokenUtils.java#L213-L254

I have no idea where they got this solution from however.

[pkoukk]

The Java implementation of Tiktoken seems to have a util for calculating tokens for functions. https://github.com/forestwanglin/openai-java/blob/93596f43e684b30f8712f9ba62edcef92d3f8a9b/jtokkit/src/main/java/xyz/felh/openai/jtokkit/utils/TikTokenUtils.java#L213-L254

I have no idea where they got this solution from however.

I tried a few simple examples following the calculation method in this code snippet, but the computed values were inconsistent with the usage amount returned by the API.

The following examples can serve as evidence:

In this example, API response prompt_tokens usage is 43
{ "model": "gpt-3.5-turbo-16k-0613", "messages": [ { "role": "assistant", "content": "Hello!" } ], "functions": [ { "name": "test", "description": "test", "parameters": { "type": "object", "properties": { "test_string": { "type": "string" } } } } ] }

In this example, API response prompt_tokens usage is 35.
{ "model": "gpt-3.5-turbo-16k-0613", "messages": [ { "role": "assistant", "content": "Hello!" } ], "functions": [ { "name": "test", "description": "test", "parameters": { "type": "object", "properties": { "test_string": { "type": "object" } } } } ] }

The only difference between the above two examples is that the type of test_string changed from string to object, but the consumption decreased by 8.

The number of tokens for the literal "string" and "object" should both be 1, and should not lead to a discrepancy.

The token consumption appears to need to be calculated in combination with the validity of the schema. Without OpenAI disclosing the rules, it is really difficult to infer through experimentation.

[pkoukk]

However, there are at least some basic rules that are certain.

We can provide an approximate calculation method to calculate usage amount when invoking the API.