Repository navigation
Releases
Releases · nagarjunak-pixel/quant-forge
v1.0.0 — Sub-Byte LLM Compression, Cholesky GPTQ/AWQ, SpinQuant & Native C++20 INT4 Engine
QuantForge v1.0.0 — Production & Hardware Research Release
Key Verified Capabilities & Measured Benchmarks
-
Native
C++20(clang++ -O3) Bit-PackedINT4(2 weights/byte) Fused GEMM: Compresses densefloat32buffers6.40x(524,288 B -> 81,920 BatG=64,5.00eff bits/weight incl. FP32 scales/zeros) with zero-materialization register accumulation (max |Δ| = 7.2e-7vs explicit dequantization). -
Second-Order Cholesky
GPTQ(act_order) vs.AWQ(MLSys '24 Best Paper): Cuts 4-bit output reconstruction MSE under22xoutlier channels by4.54x(AWQ,27.79 dB) and25.49x(GPTQCholesky$H^{-1}$ ,35.28 dB,+14.06 dBSQNR gain) over NaiveRTN. -
Orthonormal Walsh-Hadamard Incoherence (
SpinQuant/QuaRot) & Hybrid OCPFP8(E4M3/E5M2): Crushes activation crest factor7.49x(24.56 -> 3.28), improvesINT4MSE3.34xwith exact bilinear invariance (6.87e-5), and proves why hybridFP8backward passes requireE5M2(26.26 dB,0.00%overflow) overE4M3(5.79 dB). -
Hardware Roofline Autotuner & Live
qwen3.5:4b(Q4_K_M) Audit: ModelsH100 SXM5,Apple M3 Max, andA100Rooflines (3.55xINT4Decode speedup vs2.00xFP8Prefill speedup) + live Ollama hardware telemetry (8.60 GB -> 3.16 GB[2.72x],78.7 tok/sdecode). -
Quality Gates:
98.42%pytest+Hypothesiscoverage,0mypy --stricterrors,0rufferrors, and green Ubuntu C++20 + Python 3.13 + TypeScript/React CI.