| title | type | shortDescription | tags | metaDescription | redirects | freshnessValidatedDate | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
SetLlmTokenCountingCallback (.NET agent API) |
apiDoc |
Provide a callback method that determines the token count for an LLM completion |
|
New Relic .NET agent API call to provide a callback method that determines the token count for an LLM completion. |
|
never |
NewRelic.Api.Agent.NewRelic.SetLlmTokenCountingCallback(Func<string, string, int> callback);Provide a callback method that calculates the token count.
This API call is compatible with:
- Agent versions >= 10.23.0
- All app types
Set a callback method that the agent will use to determine the token count for a LLM event. In High Security Mode or when content recording is disabled, this method will be called to determine the token count for the LLM event.
The callback method must accept a two arguments of type string, and return an integer. The first string argument is the LLM model name, and the second string argument is the input to the LLM. The callback method should return the token count for the LLM event. Values of 0 or less will be ignored.
<th>
Description
</th>
</tr>
| Parameter |
|---|
`$callback`
|
Func<string, string, int> llmTokenCountingCallback = (modelName, modelInput) => {
int tokenCount = 0;
// split the input string by spaces and count the tokens
if (!string.IsNullOrEmpty(modelInput))
{
tokenCount = modelInput.Split(' ').Length;
}
return tokenCount;
};
NewRelic.Api.Agent.NewRelic.SetLlmTokenCountingCallback(llmTokenCountingCallback);