Skip to content

Choose a tag to compare

@simonw simonw released this 09 Oct 00:34
  • ttok now defaults to the o200k_base tokenizer used by the GPT-5 and GPT-6 families. #25
  • The ttok --list-models command now also shows how prefixes like gpt-5* are handled by the underlying tiktoken library.