Neural Magic
Neural Magic (Acquired by Red Hat) empowers developers to optimize & deploy LLMs at scale. Our model compression & acceleration enable top performance with vLLM
Pinned Loading
Repositories
Showing 10 of 81 repositories
- compressed-tensors Public
A safetensors extension to efficiently store sparse quantized tensors on disk
- vllm Public Forked from vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
- DeepEP-test Public Forked from smarterclayton/DeepEP
DeepEP: an efficient expert-parallel communication library
- arena-hard-auto Public Forked from lmarena/arena-hard-auto
Arena-Hard-Auto: An automatic LLM benchmark.
-
- pytorch Public Forked from pytorch/pytorch
Tensors and Dynamic neural networks in Python with strong GPU acceleration
- pydantic-regmix Public
Common mixins, registries, and utilities with native support for Pydantic used across popular repos such as GuideLLM and Speculators
- upstream-transformers Public Forked from huggingface/transformers
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
Top languages
Loading…