Skip to content

quantfit v0.1.0

Choose a tag to compare

View all tags
@Sahil170595 Sahil170595 released this 27 Jun 01:06
v0.1.0
b1618f5

GPU-aware quantization CLI with a built-in safety-tax check.

  • verify-safety — does the quantized model still refuse what the fp16 baseline refused? A two-axis tax (refusal-robustness vs over-refusal), per-zone, local judge, curated public probes. Validated on qwen2.5-1.5b + llama-3.2-1b AWQ (0 harmful-compliance regressions on both).
  • Quantize — awq / gptq / smoothquant / fp8 / rtn (compressed-tensors) + GGUF, one frozen packed calibration.
  • GPU-aware 3-tier capacity: in-GPU / CPU-offload / refuse.
  • Commands: check / list / quantize / verify / verify-safety. Tested on Python 3.10–3.13.
Assets 2
Loading