FlashMLA: Efficient MLA decoding kernels

AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data…

Code for the book "High Performance Python 2e" by Micha Gorelick and Ian Ozsvald with OReilly

LLM101n: Let's build a Storyteller

Unsupervised text tokenizer for Neural Network-based text generation.

Hackable and optimized Transformers building blocks, supporting a composable construction.

AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.

The official PyTorch implementation of Google's Gemma models

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.

Build system, successor to Buck

ZP7: Zach's Peppy Parallel-Prefix-Popcountin' PEXT/PDEP Polyfill

S-LoRA: Serving Thousands of Concurrent LoRA Adapters

An annotated implementation of the Transformer paper.

Inference code for CodeLlama models

ChatLaw：A Powerful LLM Tailored for Chinese Legal. 中文法律大模型

SQuangLe is a C++ API for accessing MySQL servers

RE2 is a fast, safe, thread-friendly alternative to backtracking regular expression engines like those used in PCRE, Perl, and Python. It is a C++ library.

Header-only C++ library for low precision floating point type emulation.

Inference Llama 2 in one file of pure C

Towards a New File Format

Fastest Integer Compression

Benchmarks of approximate nearest neighbor libraries in Python

Warp speed Data Transfer (WDT) is an embeddedable library (and command line tool) aiming to transfer data between 2 systems as fast as possible over multiple TCP paths.

ai4db and db4ai work

Weaviate is an open-source vector database that stores both objects and vectors, allowing for the combination of vector search with structured filtering with the fault tolerance and scalability of …

Open-source vector similarity search for Postgres

Awesome-LLM: a curated list of Large Language Model

Cuckoo Index: A Lightweight Secondary Index Structure

A library for efficient similarity search and clustering of dense vectors.

magic-trace collects and displays high-resolution traces of what a process is doing