This repository was archived by the owner on Feb 6, 2026. It is now read-only.
v0.9.0
Refactored the repository to support AWQ and Exl2 pipelines as separate respective apps.
Heavily modified awq quantization to support new llama3.1 models and various rope_scaling configuration
Known bugs:
- exl2 pipeline is incomplete and untested
- awq pipeline no longer supports offloading tensors to the CPU
- awq pipeline no longer supports multi-gpu