You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fixed the UserWarning: The parameter --tokens is used more than once warning shown by recent versions of Click. #20, #24
--allow-special can now be used when counting or truncating tokens, not just with --encode or --tokens. This means files-to-prompt . | ttok --allow-special now works. #17
New ttok --list-models option for listing the models supported by your installed version of tiktoken. #18
Documentation now lists newer models such as gpt-4o, gpt-4.1, gpt-5, o1, o3 and o4-mini, all of which use the o200k_base encoding. #15