Skip to content

Releases

v1.0.0 — Sub-Byte LLM Compression, Cholesky GPTQ/AWQ, SpinQuant & Native C++20 INT4 Engine

Choose a tag to compare

@nagarjunak-pixel nagarjunak-pixel released this 16 Nov 00:30

QuantForge v1.0.0 — Production & Hardware Research Release

Key Verified Capabilities & Measured Benchmarks

  • Native C++20 (clang++ -O3) Bit-Packed INT4 (2 weights/byte) Fused GEMM: Compresses dense float32 buffers 6.40x (524,288 B -> 81,920 B at G=64, 5.00 eff bits/weight incl. FP32 scales/zeros) with zero-materialization register accumulation (max |Δ| = 7.2e-7 vs explicit dequantization).
  • Second-Order Cholesky GPTQ (act_order) vs. AWQ (MLSys '24 Best Paper): Cuts 4-bit output reconstruction MSE under 22x outlier channels by 4.54x (AWQ, 27.79 dB) and 25.49x (GPTQ Cholesky $H^{-1}$, 35.28 dB, +14.06 dB SQNR gain) over Naive RTN.
  • Orthonormal Walsh-Hadamard Incoherence (SpinQuant / QuaRot) & Hybrid OCP FP8 (E4M3 / E5M2): Crushes activation crest factor 7.49x (24.56 -> 3.28), improves INT4 MSE 3.34x with exact bilinear invariance (6.87e-5), and proves why hybrid FP8 backward passes require E5M2 (26.26 dB, 0.00% overflow) over E4M3 (5.79 dB).
  • Hardware Roofline Autotuner & Live qwen3.5:4b (Q4_K_M) Audit: Models H100 SXM5, Apple M3 Max, and A100 Rooflines (3.55x INT4 Decode speedup vs 2.00x FP8 Prefill speedup) + live Ollama hardware telemetry (8.60 GB -> 3.16 GB [2.72x], 78.7 tok/s decode).
  • Quality Gates: 98.42% pytest + Hypothesis coverage, 0 mypy --strict errors, 0 ruff errors, and green Ubuntu C++20 + Python 3.13 + TypeScript/React CI.