v7.8.0

GPT-QModel 7.8.0 🚀

• GSQ: Added staged GSQ training with public method adapters, distributed staged training that reuses captured activations, and per-block bit allocation driven by an RCO search verified bit-identical against the IST-DASLab reference (measured results on Llama 3.2 1B). Added a paper-reproduction harness, best-epoch restore, HIPRTC-compiled fused scalar relaxation on ROCm, and segment-sum scale gradients.
• ROCm: Added RDNA4 (gfx1200/gfx1201/gfx1250) kernel and quantization support with per-architecture kernel tuning. MI350/gfx950 behavior is unchanged.
• GGUF: Added CPU coverage for every ggml tensor type (all K-quant and IQ variants plus explicit MXFP4/NVFP4) and a native ggml SIMD inference path with per-shape autotuning, measured 3–100x faster than the torch paths on AVX2. New gguf install extra.
• Multi-GPU: Added automatic GPU peer-access enablement on CUDA and ROCm with measured-link validation, so multi-GPU quantization copies stay on the interconnect instead of the driver's staging path.
• Experimental W4A: Revalidated the NVFP4 packed layout at load time, removed the retained E4M3 weight expansion, and relaxed the Triton build requirement on the Gluon FP8 GEMM.
• Performance: Package import now builds the kernel registry lazily and defers optional backend probes (previously ~9s and 6767 modules). Vectorized the GPTQ/QQQ MSE shrink-grid and AWQ clip-grid evaluation; grid workspace budgets moved from environment variables to a new MSE configuration option.
• Models: Added RecursiveSI Flint and Raon-1 mini support, Gemma 3 multimodal checkpoint compatibility, Qwen2.5-Omni thinker lm_head quantization, and the Transformers 5.22 Qwen-Omni layout.
• Compat: Kept the AVX512-BF16 GPTQ/AWQ CPU kernels trainable for PEFT LoRA; fixed OpenAI-server auth, stop handling, and ChatCompletion schema; surfaced compatibility-shim failures instead of failing silently; fixed the Marlin CPU-offload snapshot refresh; fixed the calibration minimum-length setting not reaching the dataset filter; re-checked for an emptied calibration set after min-length filtering; tolerated MoE early-stop in data-parallel forward; bounded remote image fetches; read accelerator memory telemetry from the active device; fixed config serialization mutating dynamic overrides; fixed GPTAQ loss-log capture and evaluation-metadata coercion for fast eval tasks.

Full list of changes

What's Changed

• docs: add quantization precision guide by @ZX-ModelCloud in #3433
• fix(qwen2.5-omni): quantize thinker lm_head by @ZX-ModelCloud in #3432
• feat(gguf): full CPU coverage for all ggml tensor types + optimized native inference by @Qubitium in #3434
• perf(quantizer): vectorize MSE shrink grid and AWQ clip grid evaluation by @Qubitium in #3435
• refactor(config): replace grid workspace env vars with MSEConfig by @Qubitium in #3436
• fix(looper): read accelerator memory telemetry from the active device by @li-lizhe in #3438
• Fix Gemma 3 multimodal checkpoint compatibility by @ZX-ModelCloud in #3429
• Add staged GSQ training and public method adapters by @Qubitium in #3110
• fix(config): avoid mutating dynamic overrides in to_dict by @anh-nguyenn in #3442
• fix(vram): return qualified non-persistent buffer names without mutating modules by @anh-nguyenn in #3441
• Fix calibration data minimum length propagation by @ZX-ModelCloud in #3443
• Fix module tree test alignment and verify Spark shared inputs by @ZX-ModelCloud in #3444
• fix(gsq): restore the best validation epoch instead of always exporting the last by @Qubitium in #3445
• fix(gsq): stop block captures outliving the blocks that need them by @Qubitium in #3447
• feat(gsq): add paper reproduction scripts and measured GPTQ vs GSQ results by @Qubitium in #3448
• feat(gsq): per-block bit allocation with RCO and measured results by @Qubitium in #3449
• test: fix qzeros shape expectation after GPU pack fallback by @ZX-ModelCloud in #3451
• test: skip NVFP4 selector on unsupported GPUs by @ZX-ModelCloud in #3450
• perf(gsq): compile fused scalar relaxation with HIPRTC on ROCm by @Qubitium in #3453
• perf(gsq): reduce scale gradient as a segment sum (1.5x, no VRAM cost) by @Qubitium in #3454
• fix(marlin): refresh CPU offload snapshot after post-init by @ZX-ModelCloud in #3452
• docs(readme): attribute IST-DASLab RCO bit-allocation port by @Qubitium in #3455
• GSQ: distribute staged training and reuse captured activations by @Qubitium in #3456
• [MODEL] support flint by @ZX-ModelCloud in #3458
• [MODEL] support raon1_mini by @ZX-ModelCloud in #3460
• ci(lint): make the ruff gate able to fail by @ZX-ModelCloud in #3461
• Rename remaining GPTQv2 method refs to GPTAQ by @Qubitium in #3462
• fix(w4a): tolerate a Torch-pinned Triton without the Gluon mma_v2 FP8 GEMM by @ZX-ModelCloud in #3463
• tests: register integration marker and add missing test package markers by @ZX-ModelCloud in #3464
• fix: make scripts/ a real package so repo helpers are importable by @ZX-ModelCloud in #3465
• Fix derive_fast_eval_tasks coercing evalution metadata with float() by @ZX-ModelCloud in #3466
• fix(exception-handling): surface compatibility-shim failures and stop swallowing control-flow exceptions by @ZX-ModelCloud in #3467
• fix(openai-server): auth, stop handling, and ChatCompletion schema by @ZX-ModelCloud in #3468
• fix(w4a): stop caching the E4M3 weight expansion by @ZX-ModelCloud in #3459
• feat(rocm/cuda): auto-enable GPU peer access so multi-GPU quantization stays on the interconnect by @ZX-ModelCloud in #3479
• perf(import): build the kernel registry lazily and defer optional backend probes by @ZX-ModelCloud in #3469
• fix(w4a): revalidate the packed layout in NVFP4 post_init by @Qubitium in #3481
• fix(cpu): new cpu kernels regressed trainable api calls from peft by @jiqing-feng in #3480
• feat(rocm): RDNA4 (gfx1200/gfx1201/gfx1250) kernel and quantization support by @ZX-ModelCloud in #3478
• fix(calibration): bound and diagnose remote image fetches by @ZX-ModelCloud in #3483
• tests: repair kernel suites and support the transformers 5.22 Qwen-Omni layout by @ZX-ModelCloud in #3485
• fix(looper): tolerate MoE early-stop in data-parallel forward by @ZX-ModelCloud in #3484
• fix(looper): re-check empty calibration after prepare_dataset_func by @Qubitium in #3487
• release: bump to 7.8.0 and add changelog by @Qubitium in #3488

New Contributors

• @li-lizhe made their first contribution in #3438
• @anh-nguyenn made their first contribution in #3442

Full Changelog: v7.7.0...v7.8.0