Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes a minor bug in embedding token-counting within
TokenCountingHandler
.The current implementation replaces
\n
with spaces (" "
) before sending the prompt to the model (e.g., OpenAI), but this replacement is not applied when counting tokens locally.This causes a mismatch between actual usage reported by the provider and what LlamaIndex estimates.
Fix
Added
.replace("\n", " ")
before counting tokens to align local counting with what is actually sent.Fixes: No associated issue number (happy to file one if needed).
New Package?
Did I fill in the
tool.llamahub
section in thepyproject.toml
and provide a detailed README.md for my new integration or package?Version Bump?
Did I bump the version in the
pyproject.toml
file of the package I am updating? (Except for thellama-index-core
package)Type of Change
How Has This Been Tested?
TokenCountingHandler
output after the fixSuggested Checklist: