sgl-project

All

11 repositories

ome
Public
OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)
k8s oracle-cloud model-serving model-as-a-service multi-node-kubernetes llm-inference
Go
•
MIT License
•11•144•6•12•Updated Jul 7, 2025Jul 7, 2025
sgl-project.github.io
Public
This is the documentation repository for SGLang. It is auto-generated from https://github.com/sgl-project/sglang/tree/main/docs.
HTML
•13•56•6•3•Updated Jul 7, 2025Jul 7, 2025
sglang
Public
SGLang is a fast serving framework for large language models and vision language models.
cuda inference pytorch transformer openai moe llama vlm blackwell llm
Python
•
Apache License 2.0
•2.3k•16k•499•407•Updated Jul 7, 2025Jul 7, 2025
whl
Public
Kernel Library Wheel for SGLang
cuda cutlass sglang flashinfer
HTML
•
MIT License
•2•9•0•0•Updated Jul 5, 2025Jul 5, 2025
genai-bench
Public
Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems.
Python
•
MIT License
•9•157•5•3•Updated Jul 3, 2025Jul 3, 2025
sgl-learning-materials
Public
Materials for learning SGLang
MIT License
•35•471•0•0•Updated Jun 22, 2025Jun 22, 2025
DeepGEMM
Public
DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling
Python
•
MIT License
•636•17•0•0•Updated Jun 12, 2025Jun 12, 2025
torchtitan
Public
A PyTorch native library for large-scale model training
Python
•
BSD 3-Clause "New" or "Revised" License
•419•0•0•0•Updated May 18, 2025May 18, 2025
sgl-attn
Public
Fast and memory-efficient exact attention
Python
•
BSD 3-Clause "New" or "Revised" License
•1.8k•10•0•1•Updated May 15, 2025May 15, 2025
sgl-test-files
Public
The test files for SGLang.
MIT License
•1•0•0•0•Updated Mar 12, 2025Mar 12, 2025
tensorrt-demo
Public
TensorRT LLM Benchmark Configuration
Python
•4•13•0•0•Updated Jul 26, 2024Jul 26, 2024