-
Notifications
You must be signed in to change notification settings - Fork 24
Closed
Description
It is possible for a token counter to return zero tokens, leading to a division by zero error in the merge_splits function:
File "C:\Users\user\miniconda3\envs\project\Lib\site-packages\semchunk\semchunk.py", line 78, in merge_splits
average = cumulative_lengths[midpoint] / tokens if cumulative_lengths[midpoint] else average
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~
ZeroDivisionError: division by zero
Metadata
Metadata
Assignees
Labels
No labels