Distributed Triton for Parallel Systems

(Crafter + NetHack) in JAX. ICML 2024 Spotlight.

[NeurIPS 2023] Tree of Thoughts: Deliberate Problem Solving with Large Language Models

A generative world for general-purpose robotics & embodied AI learning.

[NeurIPS 2024 Best Paper][GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ult…

prime is a framework for efficient, globally distributed training of AI models over the internet.

DIAMOND (DIffusion As a Model Of eNvironment Dreams) is a reinforcement learning agent trained in a diffusion world model. NeurIPS 2024 Spotlight.

verl: Volcano Engine Reinforcement Learning for LLMs

A customisable 3D platform for agent-based AI research

800,000 step-level correctness labels on LLM solutions to MATH problems

SGLang is a fast serving framework for large language models and vision language models.

OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games.

Distributed Training Over-The-Internet

Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch

Tile primitives for speedy kernels

Code for Neurips24 paper: QuaRot, an end-to-end 4-bit inference of large language models.

[MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention

Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3.

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

STREAM, for lots of devices written in many programming models

A minimal GPU design in Verilog to learn how GPUs work from the ground up

A PyTorch Native LLM Training Framework

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.

Large World Model -- Modeling Text and Video with Millions Context

An educational resource to help anyone learn deep reinforcement learning.