Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. All in a modern, AI-native editor.

A high-throughput and memory-efficient inference and serving engine for LLMs

A hyperparameter optimization framework, inspired by Optuna.