Development repository for the Triton language and compiler

Making large AI models cheaper, faster and more accessible

Ongoing research training transformer models at scale

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Modeling, training, eval, and inference code for OLMo

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

OneFlow is a deep learning framework designed to be user-friendly, scalable and efficient.

Fast and memory-efficient exact attention

🚀🚀 「大模型」2小时完全从0训练26M的小参数GPT！🌏 Train a 26M-parameter GPT from scratch in just 2h!

NCCL Tests

A high-throughput and memory-efficient inference and serving engine for LLMs

RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.

SGLang is a fast serving framework for large language models and vision language models.

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

Optimized primitives for collective multi-GPU communication

OneDiff: An out-of-the-box acceleration library for diffusion models.

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.