Specify a github or local repo, github pull request, arXiv or Sci-Hub paper, Youtube transcript or documentation URL on the web and scrape into a text file and clipboard for easier LLM ingestion

The ChatGPT Long Term Memory package is a powerful tool designed to empower your projects with the ability to handle a large number of simultaneous users and external sources.

Predict the exact openai token usage of functions

A custom GPT based on [Zero To Hero](https://karpathy.ai/zero-to-hero.html) utilizing tiktoken with the intent to augment AI Transformer-model education and reverse engineer GPT models from scratch.

The Tiktoken API is a tool that enables developers to calculate the token usage of their OpenAI API requests before sending them, allowing for more efficient use of tokens.

🧰 The AutoTokenizer that TikToken always needed -- Load any tokenizer with TikToken now! ✨

Calculate the token count for GPT-4, GPT-3.5, GPT-3, and GPT-2.

Count tokens in a text file.

Docker container to expose the OpenAI tokenizer as a REST service

Simple wrapper around tiktoken to use it in your favorite language.

tiktoken fastapi for encoding and decoding tokens for OpenAI ChatGPT GPT-3 GPT-3.5 GPT-4 models

Logseq Markdown Tokenizer is a Python application that tokenizes and estimates prices for one to many markdown files.

Tiktoken-counter as standalone API

Extracts Google Sheets to JSONL for fine-tuning, estimates task costs with tiktoken.

Este projeto consiste em uma aplicação que utiliza IA generativa para enviar recomendações de produtos com base nos perfis de compra de cada cliente

console based game based on a llm

This project harnesses the power of Tiktoken and the OpenAI API to create a Python Streamlit web application with a primary focus on token counting and price estimation.

The OpenAI tiktoken library as a service. For counting the number of tokens in a message to an LLM like GPT.

В рамках хакатона был создан телеграм бот для консультации с клиентами с сфере обслуживания магазина "Ашан"