Releases
v1.0.0
v1.0.0 — Sub-Byte LLM Compression, Cholesky GPTQ/AWQ, SpinQuant & Native C++20 INT4 Engine
Latest
Compare
Sorry, something went wrong.
No results found
QuantForge v1.0.0 — Production & Hardware Research Release
Key Verified Capabilities & Measured Benchmarks
Native C++20 (clang++ -O3) Bit-Packed INT4 (2 weights/byte) Fused GEMM : Compresses dense float32 buffers 6.40x (524,288 B -> 81,920 B at G=64, 5.00 eff bits/weight incl. FP32 scales/zeros) with zero-materialization register accumulation (max |Δ| = 7.2e-7 vs explicit dequantization).
Second-Order Cholesky GPTQ (act_order) vs. AWQ (MLSys '24 Best Paper) : Cuts 4-bit output reconstruction MSE under 22x outlier channels by 4.54x (AWQ, 27.79 dB) and 25.49x (GPTQ Cholesky $H^{-1}$ , 35.28 dB, +14.06 dB SQNR gain) over Naive RTN.
Orthonormal Walsh-Hadamard Incoherence (SpinQuant / QuaRot) & Hybrid OCP FP8 (E4M3 / E5M2) : Crushes activation crest factor 7.49x (24.56 -> 3.28) , improves INT4 MSE 3.34x with exact bilinear invariance (6.87e-5), and proves why hybrid FP8 backward passes require E5M2 (26.26 dB, 0.00% overflow) over E4M3 (5.79 dB).
Hardware Roofline Autotuner & Live qwen3.5:4b (Q4_K_M) Audit : Models H100 SXM5, Apple M3 Max, and A100 Rooflines (3.55x INT4 Decode speedup vs 2.00x FP8 Prefill speedup) + live Ollama hardware telemetry (8.60 GB -> 3.16 GB [2.72x], 78.7 tok/s decode).
Quality Gates : 98.42% pytest + Hypothesis coverage, 0 mypy --strict errors, 0 ruff errors, and green Ubuntu C++20 + Python 3.13 + TypeScript/React CI.
You can’t perform that action at this time.