Skip to content

0.4

Choose a tag to compare

@simonw simonw released this 08 Oct 23:34
· 4 commits to main since this release
c4e4643
  • Fixed the UserWarning: The parameter --tokens is used more than once warning shown by recent versions of Click. #20, #24
  • --allow-special can now be used when counting or truncating tokens, not just with --encode or --tokens. This means files-to-prompt . | ttok --allow-special now works. #17
  • New ttok --list-models option for listing the models supported by your installed version of tiktoken. #18
  • Documentation now lists newer models such as gpt-4o, gpt-4.1, gpt-5, o1, o3 and o4-mini, all of which use the o200k_base encoding. #15
  • ttok now requires Python 3.10 or higher.
  • Packaging moved from setup.py to pyproject.toml.