#

tiktoken

Here is 1 public repository matching this topic...

aallam / ktoken

Kotlin multiplatform BPE tokenizer library for OpenAI models

kotlin tokenizer openai gpt bpe byte-pair-encoding tiktoken binary-p

Updated Aug 14, 2024
Kotlin

Improve this page

Add a description, image, and links to the tiktoken topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the tiktoken topic, visit your repo's landing page and select "manage topics."