#

tiktoken

Here are 4 public repositories matching this topic...

pkoukk / tiktoken-go

go version of tiktoken

go golang openai gpt-4 chatgpt tiktoken gpt-35-turbo

Updated May 21, 2024
Go

openshield

openshieldai / openshield

OpenShield is a new generation security layer for AI models

python golang security ai guardian firewall tokenizer models owasp artificial-intelligence openai llama security-tools openai-api llm tiktoken probllama

Updated Nov 7, 2024
Go

hupe1980 / go-tiktoken

✂️ OpenAI's tiktoken tokenizer written in Go

golang tokenizer openai bpe gpt2 tiktoken

Updated May 20, 2024
Go

peterheb / gotoken

Gotoken is a pure-Go implementation of the Python library openai/tiktoken.

go openai chatgpt tiktoken

Updated Apr 19, 2023
Go

Improve this page

Add a description, image, and links to the tiktoken topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the tiktoken topic, visit your repo's landing page and select "manage topics."