kannon92
Follow
Stars
3
results
for sponsorable starred repositories
written in Python
Clear filter
A high-throughput and memory-efficient inference and serving engine for LLMs
A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. All in a modern, AI-native editor.
Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM