Repository navigation
GPT-QModel 7.11.0 🚀
- MACaQ: Native MXFP4 W4A4 quantization for low-precision weights and activations, with CUDA, ROCm, and Apple silicon inference. Includes packed checkpoints and a portable Torch fallback.
- NF12: Lossless 12-bit compression of BF16 weights reduces source-weight storage during calibration without changing the restored values. Packed inference now includes native Metal support alongside CUDA, HIP, and Torch paths. MLX · MoE capture contract
- Tenstorrent: Packed GPTQ inference through Torch/XLA and TTNN, multi-card execution, and block-float quantization. Long-context Llama serving adds paged caching and chunked prefill; compatible block-float checkpoints also run on CUDA, ROCm, and CPU. Guide · Serving · Device ownership
- Quantization: Resumable distributed EXL3 jobs (projection publication recovery), broader mixed-bit allocation across AWQ/EXL3/GGUF, and SmoothQuant preprocessing improve large-model workflows and quality control. Quantizer calibration flows document the shared Hessian-failure and zero-sample fallback contracts. MoE routing override and bypass defines when each mode is honoured and when it fails closed. Shared-input Hessian dedup covers QQQ/VPTQ/MACaQ/EXL3 and GPTAQ/FOEM.
- Inference: Opt-in attention and MLP projection fusion, faster quantized embedding lookups, and improved small-batch inference. Native backend selection and packed-weight storage are preserved. Fusion guide
- Configuration: Typed backend configurations unify backend selection and tuning, with explicit numerical and worker controls instead of environment switches.
- Calibration: Broader dataset input support, document-isolated AWQ replay, and opt-in attention-sink exclusion improve calibration coverage and control.
- Memory: Reduced packed-weight duplication, bounded EXL3 scratch, and shorter-lived inference workspaces lower memory use.
- Models: Added OLMo Hybrid, Granite SWA/MoE SWA, and Dots3 Note quantization, plus broader decoder-weight coverage and Transformers 5.22 compatibility. Model notes
- Compat: Major fixes across dense/MoE calibration, multimodal replay, checkpoint round trips, Transformers model loading, multi-GPU memory use, and ROCm execution. Unsupported combinations fail clearly rather than loading incomplete weights. Full notes
- Security: Runtime checks for vulnerable PyTorch versions, explicit checkpoint-code trust, stronger checkpoint validation, and bounded server/media requests. Runtime safety · Serving security
Full list of changes
What's Changed
- fix(calibration): declare IMAGE_TO_TEXT modality for 4 multimodal wrappers by @ZX-ModelCloud in #3558
- fix(awq): stage and pack every module at its own width by @Qubitium in #3563
- fix(gguf): honour a per-module ggml type in
dynamicby @Qubitium in #3562 - fix(rco): derive allocatable projections from model definition tags by @Qubitium in #3564
- feat(rco): allocate AWQ, EXL3 and GGUF formats under packed-weight budgets by @Qubitium in #3561
- fix(models): resolve module roles from explicit model trees by @Qubitium in #3565
- docs: explain module plans, model tags and DeepSeek V4.1 inventory by @Qubitium in #3566
- feat(models): opt-in module-plan projection fusion with validated model profiles by @Qubitium in #3567
- fix(tests): align AWQ widths and Ripple compatibility fixtures by @Qubitium in #3569
- fix(reshard): preserve backups and shard every detected layer stack by @Qubitium in #3568
- fix(loader): keep duplicate explicit GPU requests constrained by @Qubitium in #3570
- fix(tests): run checkpoint and MoE suites in fresh checkouts by @Qubitium in #3572
- fix(ricursive): preserve new token blocks with warm caches by @Qubitium in #3571
- fix(ricursive): preserve full prompts with capacity-padded masks by @Qubitium in #3574
- fix(tests): gate native fusion cases on extension availability by @Qubitium in #3575
- fix(cpu): compute BF16 int4pack with FP32 operands by @Qubitium in #3576
- fix(loader): preserve concrete device sequences by @Qubitium in #3577
- test(fusion): assert a ULP envelope instead of identical greedy IDs by @Qubitium in #3578
- fix(reshard): honor explicitly selected source file or index by @Qubitium in #3589
- fix(reshard): keep named indexed shards complete by @Qubitium in #3590
- fix(config): preserve per-layer-type RoPE dictionaries by @tpurtell in #3579
- fix(loader): restore direct nonpersistent rotary buffers by @tpurtell in #3580
- fix(writer): stream E8M0 safetensors scales by @tpurtell in #3581
- fix(exl3): match native half rounding in Torch codebooks by @tpurtell in #3582
- fix(loader): restore direct init buffers beside materialized parameters by @Qubitium in #3617
- fix(exl3): preserve logical outputs over padded storage by @tpurtell in #3583
- fix(writer): preserve existing shards and config extensions on export by @tpurtell in #3584
- feat(writer): reuse content-authenticated safetensors shards by @tpurtell in #3585
- feat(offload): reference authenticated safetensors payloads by @tpurtell in #3586
- fix(writer): honor umask without changing process state by @Qubitium in #3618
- fix(offload): preserve reference source and index permissions by @Qubitium in #3619
- fix(writer): protect referenced offload sources during export by @Qubitium in #3620
- fix(offload): resolve nested tensors in index scope by @Qubitium in #3621
- fix(reshard): reject ambiguous named shard indexes by @Qubitium in #3622
- fix(writer): preserve embedding shards on failed export by @Qubitium in #3623
- fix(writer): drop stale replacement tensors on repeated save by @Qubitium in #3624
- fix(exl3): handle inference-created bias in native version tracking by @Qubitium in #3625
- feat(nf12_ad): lossless 12-bit BF16 quantization (AlpinDale NF12) + RDNA4 HIP decode by @Qubitium in #3573
- fix(offload): allow absent optional buffers in reference offload by @Qubitium in #3626
- feat(nf12): public
nf12name andnf12_packedkey by @Qubitium in #3627 - fix(writer): serialize FNUZ float8 safetensors payloads by @Qubitium in #3628
- fix(nf12): refresh cached exponent when loader replaces buffer by @Qubitium in #3629
- fix(nf12): preserve source bias precision when packing by @Qubitium in #3630
- fix(nf12): make module repr safe before materialization by @Qubitium in #3631
- fix(offload): recognize meta buffers on repeated offload by @Qubitium in #3632
- perf(nf12): route m=16 to RDNA4 matrix cores, fix m=128 and split-K bias by @Qubitium in #3633
- fix(hf): size NF12 escape buffers before checkpoint assignment by @Qubitium in #3634
- fix(nf12): fail validation when explicit CUDA backend is unavailable by @Qubitium in #3636
- perf(nf12): pad the RDNA4 LDS stride and retune split-K for small weights by @Qubitium in #3637
- fix(nf12): verify installed runtime operators before reuse by @Qubitium in #3638
- fix(nf12): respect validator target device when selecting backend by @Qubitium in #3639
- fix(nf12): synchronize selected GPU in performance validator by @Qubitium in #3640
- perf(nf12): drop the RDNA4 store transpose and hoist loads over the mma by @Qubitium in #3641
- fix(nf12): benchmark HIP native encoder in validator by @Qubitium in #3642
- fix(nf12): use stable accuracy case seeds in validator by @Qubitium in #3643
- perf(nf12): cap the RDNA4 split-K reduce grid and retune the tail by @Qubitium in #3644
- fix(nf12-hip): bind WMMA scratch and dispatch to tensor device by @Qubitium in #3635
- perf(nf12): specialize the RDNA4 fused small-M kernel on the token count by @Qubitium in #3645
- fix(cuda): protect async activation copy storage across streams by @Qubitium in #3646
- fix(gsq): handle empty CUDA training geometries before launch by @Qubitium in #3647
- fix(cuda): add quantized linear bias before output rounding by @Qubitium in #3649
- perf(nf12): derive the RDNA4 buffer and split-K choices from the launch shape by @Qubitium in #3648
- docs(nf12): de-duplicate the credits block in the HIP file headers by @Qubitium in #3650
- fix(cuda): restore gradients for FP32-output quantized matmuls by @Qubitium in #3651
- docs(cuda): cite the merged gradient-flow PR by @Qubitium in #3652
- perf(nf12): split M at m=128 on narrow N as well as long K by @Qubitium in #3653
- fix(cuda): keep unbiased GPTQ streaming reductions in FP32 by @Qubitium in #3654
- fix(cuda): keep FP8 scaled matmul eligible after empty forward by @Qubitium in #3655
- fix(cuda): handle empty NVFP4 calibration replay operands by @Qubitium in #3656
- fix(cuda): round ParoQuant Triton bias once and preserve gradients by @Qubitium in #3657
- fix(cuda): keep AWQ Triton bias in wide epilogue and return gradient by @Qubitium in #3658
- feat(nf12): sync the RDNA4 HIP kernels with upstream nf12 by @Qubitium in #3659
- fix(cuda): keep AWQ GEMV Triton bias wide and restore input gradients by @Qubitium in #3660
- fix(cuda): add GPTQ TritonV2 bias before rounding and repair backward arity by @Qubitium in #3661
- feat(exl3): add bounded covariance accumulation with an FP64 outer sum by @tpurtell in #3587
- feat(looper): add authenticated disk-backed layer outputs by @tpurtell in #3588
- feat(exl3): preserve Hessian conditioning and projection diagnostics by @tpurtell in #3591
- fix(cuda): add portable QQQ bias before output rounding by @Qubitium in #3662
- fix(gguf): keep Triton bias epilogue wide and handle empty batches by @Qubitium in #3663
- fix(gguf): invalidate Triton metadata on packed weight updates by @Qubitium in #3664
- fix(bitsandbytes): decode odd-width Triton rows with global nibble parity by @Qubitium in #3665
- fix(gguf): support inference tensors in K-scale overflow cache by @Qubitium in #3666
- fix(gguf): return empty direct Triton launch outputs by @Qubitium in #3667
- fix(gguf): widen Triton K-quant metadata before weight reconstruction by @Qubitium in #3668
- fix(gguf): preserve K-quant CUDA range and BF16 precision by @Qubitium in #3670
- fix(gguf): preserve BF16 dense fallback precision across formats by @Qubitium in #3671
- fix(gguf): guard Q4_0/Q8_0 CUDA dots from FP16 overflow by @Qubitium in #3672
- fix(gguf): preserve finite FP16 CUDA dots across other formats by @Qubitium in #3673
- fix(nf12): split-K geometry and HIP review fixes by @Qubitium in #3676
- fix(cuda): align AWQ GEMV Triton tiles to irregular groups by @Qubitium in #3675
- fix(cuda): preserve AWQ GEMM eval gradients and adapter output by @Qubitium in #3677
- fix(lora): compose quantized CUDA outputs out of place by @Qubitium in #3678
- fix(gguf): preserve BF16/FP32 activation range in Triton matmuls by @Qubitium in #3679
- test(exl3): match native half rounding in independent oracle by @Qubitium in #3680
- fix(nf12): probe WMMA capability beyond cache capacity by @Qubitium in #3681
- docs(nf12): record validated ROCm 10.1 and HIP toolchain by @Qubitium in #3682
- fix(gguf): keep biased BF16/FP32 fused dots wide by @Qubitium in #3683
- fix(nf12): sync HIP routing bounds and exact encoder scheduling by @Qubitium in #3685
- fix(nf12): admit supported RDNA4 WMMA grid dimensions by @Qubitium in #3687
- fix(nf12): validate native HIP MoE bounds by @Qubitium in #3689
- fix(nf12): correct HIP split-K bias and routed API contracts by @Qubitium in #3691
- fix(nvfp4): pack strided activation views correctly by @Qubitium in #3684
- fix(bnb): handle strided Triton GEMM activations by @Qubitium in #3692
- fix(qqq): quantize strided Triton activation views correctly by @Qubitium in #3693
- fix(w4afp8): support strided prepacked GEMM operands by @Qubitium in #3694
- fix(nvfp4): decode strided packed operands correctly by @Qubitium in #3695
- fix(bnb): handle strided packed Triton operands by @Qubitium in #3696
- fix(w4afp8): honor packed and output strides in INT4 unpack by @Qubitium in #3697
- fix(paroquant): normalize strided Triton GEMM operands by @Qubitium in #3698
- fix(tritonv2): decode strided GPTQ operands correctly by @Qubitium in #3699
- perf(qlinear): derive ret_idx with one stable argsort by @ZX-ModelCloud in #3702
- fix(models): resolve real model_type spellings for Unlimited-OCR and RefinedWeb by @ZX-ModelCloud in #3690
- fix(rocm): build the RDNA4 HIP kernels for every family target by @ZX-ModelCloud in #3686
- feat(exl3): retain restart-idempotent projection and route ledgers by @tpurtell in #3592
- feat(exl3): checkpoint completed projections atomically by @tpurtell in #3593
- feat(exl3): persist authenticated capture frontiers by @tpurtell in #3594
- feat(exl3): resume capture at calibration-batch boundaries by @tpurtell in #3595
- feat(moe): expose ranked router candidates for route recovery by @tpurtell in #3596
- feat(exl3): persist deterministic adjacent mixed-tier plans by @tpurtell in #3597
- feat(exl3): dispatch authenticated projections to pipelined workers by @tpurtell in #3598
- feat(loader): preserve lazy native sources across replay and export by @tpurtell in #3599
- feat(looper): carry causal replay and durable recovery callbacks by @tpurtell in #3600
- feat(exl3): integrate resumable distributed and mixed-tier execution by @tpurtell in #3601
- feat(writer): compose native checkpoint passthrough and save overlays by @tpurtell in #3602
- feat(deepseek): carry bounded V4 target and MTP calibration replay by @tpurtell in #3603
- feat(glm): integrate Flash target and MTP recovery and export by @tpurtell in #3604
- feat(models): expose Dots3 Note routed experts individually by @tpurtell in #3605
- feat(kimi): add bounded K3 source and explicit AttnRes model contracts by @tpurtell in #3606
- feat(kimi): carry causal core calibration and padded EXL3 source storage by @tpurtell in #3607
- feat(kimi): carry K3.1 source and bounded calibration adapters by @tpurtell in #3608
- feat(glm): add bounded native core calibration and EXL3 sources by @tpurtell in #3609
- feat(glm): carry native full-MTP alignment and source loading by @tpurtell in #3610
- feat(glm): carry Flash core and MTP calibration adapters by @tpurtell in #3611
- feat(step5): carry bounded native core and MTP calibration by @tpurtell in #3612
- feat(deepseek): carry V4.1 native source and mapped PLE replay by @tpurtell in #3613
- feat(deepseek): carry V4.1 routed capture and adjacent recovery by @tpurtell in #3614
- feat(deepseek): persist authenticated typed V4.1 frontiers by @tpurtell in #3615
- feat(deepseek): carry V4.1 main-input and dSpark replay adapters by @tpurtell in #3616
- fix(ricursive): correct warm-cache input slicing and padded positions by @Qubitium in #3703
- perf(torch_fused): skip identity ret_idx gather by @ZX-ModelCloud in #3701
- fix(rocm): fuse bias before output rounding by @ZX-ModelCloud in #3700
- feat(marlin): probe the full device profile and derive arch checks from it by @Lumea-art in #3674
- fix(extension): report current load failure instead of stale JIT error by @Qubitium in #3704
- fix(marlin): validate raw profile ordinals before Torch wraps them by @Qubitium in #3705
- fix(v41): validate direct routed batches before Hessian capture by @Qubitium in #3706
- test(glm): provide a causal mask for direct DSA layer replay by @Qubitium in #3707
- test(v41): cover mapped PLE and authenticated replay frontiers by @Qubitium in #3708
- fix(ricursive): complete compatibility fixes and native-model regression coverage by @Qubitium in #3709
- fix(ricursive): repair training, tensor parallelism, masks, and concurrent forwards by @Qubitium in #3710
- test(rocm): handle non-ROCm LLC status by @Qubitium in #3711
- fix(writer): validate all selected checkpoint tensors by @Qubitium in #3712
- fix(loader): gate auto decoder bypass on native floatx by @Qubitium in #3713
- fix(exl3): make slot assignment atomic across processes by @Qubitium in #3714
- fix(exl3): make scoped frontier discard restart-safe by @Qubitium in #3715
- fix(nf12): run BF16T through split-K dispatcher by @Qubitium in #3716
- feat(multi-gpu): GIL-safe P2P copies across the quantization lifecycle by @Qubitium in #3489
- fix(kernels): enforce validation and cache lifecycle contracts by @Qubitium in #3717
- fix(p2p): validate peer copies before auto-enabling and guard ROCm by @Qubitium in #3718
- fix(mlx): round ParoQuant codes in FP32 by @Qubitium in #3734
- fix(exllamav2): reuse act-order handle by @Qubitium in #3733
- fix(exl3): route wide activations through Torch by @Qubitium in #3731
- fix(nf12): preserve BF16 weights in CPU fallback by @Qubitium in #3730
- fix(fp8): preserve E5M2 scales on reload by @Qubitium in #3725
- fix(qqq): configure quantizer with dynamic group size by @Qubitium in #3724
- fix(int8): add CPU bias before output rounding by @Qubitium in #3729
- fix(gguf): add CPU bias before output rounding by @Qubitium in #3728
- fix(fp8): preserve FP32 bias range by @Qubitium in #3726
- fix(nf12): preserve mixed dtype bias through fallback by @Qubitium in #3735
- fix(mlx): avoid FP16 overflow in ParoQuant zero offset by @Qubitium in #3736
- fix(gptq): accumulate Hessians across finalization by @Qubitium in #3721
- fix(exl3): invalidate native cache after inference metadata edits by @Qubitium in #3732
- fix(gptq): honor static groups after column reordering by @Qubitium in #3719
- fix(fp8): add bias before low-precision output rounding by @Qubitium in #3737
- fix(awq): keep affine ranges and packed decode finite by @Qubitium in #3722
- fix(gguf): preserve FP32 bias range by @Qubitium in #3727
- fix(gptq): use each group in mock quantization by @Qubitium in #3720
- fix(gguf): add C++ backend bias before output rounding by @Qubitium in #3739
- fix(exl3): keep Torch fallback out of checkpoint state by @Qubitium in #3738
- fix(awq): honor dynamic group size while staging by @Qubitium in #3723
- fix(fp8): preserve inverse scales on BF16 checkpoint reload by @Qubitium in #3741
- fix(gguf): preserve CPU cancellation under BF16 autocast by @Qubitium in #3743
- fix(paroquant): refresh rotation identity on cache clear by @Qubitium in #3744
- fix(exl3): accept noncontiguous BF16 and FP32 activations by @Qubitium in #3740
- test(gguf): skip C++ bias tests when optional CPU bridge is unavailable by @Qubitium in #3745
- fix(gptq): preserve asymmetric zero offsets in fused CPU int4 by @Qubitium in #3746
- fix(fp8): reject nonfinite source weights before scaling by @Qubitium in #3742
- fix(gptq): fuse CPU low-precision bias epilogue by @Qubitium in #3747
- fix(int8): preserve GPTQ checkpoint tensors after post-init by @Qubitium in #3748
- fix(fp8): retain packed weights across reload and module moves by @Qubitium in #3749
- test(ricursive): cover Transformers 4 default-cache rollback by @Qubitium in #3752
- test(contracts): align fused-bias oracles and GLM-5.3 MTP ownership by @ZX-ModelCloud in #3756
- fix(ernie4_5_vl_moe): release captured modules by attribute name by @ZX-ModelCloud in #3750
- feat(nf12): ship packed BF16 inference API in GPT-QModel by @Qubitium in #3755
- fix(nf12): keep BF16 weight range in the decoded fallback by @ZX-ModelCloud in #3751
- test(nf12): compare non-SM80 prefill with an FP64 oracle by @Qubitium in #3757
- fix(ernie4_5_vl_moe): expand routed text/vision expert indices by @ZX-ModelCloud in #3754
- fix(fp4,vptq): keep stored bias wide through the linear epilogue by @ZX-ModelCloud in #3753
- fix(cpu): keep fused AVX512 layers on concatenated int4pack operands by @Qubitium in #3762
- fix(awq): keep FP32 affine operands across CPU AVX512 dtype moves by @Qubitium in #3763
- test(fp4): pin the graph-safe bias gate on the FP4 wrapper by @ZX-ModelCloud in #3759
- test(qqq,exl3): realign stale oracles with the FP32 epilogue and Torch wide path by @Qubitium in #3764
- fix(cpu): keep checkpoint layout across inference transforms and fusion by @Qubitium in #3765
- fix(zamba2): tolerate missing optional mamba kernel symbols by @ZX-ModelCloud in #3768
- fix(granitemoehybrid): probe eager mamba forward instead of torch_forward by @ZX-ModelCloud in #3767
- fix(cpu): distinguish checkpoint and runtime fused state layouts by @ZX-ModelCloud in #3769
- bench: add an A/B harness that reports its own noise floor by @Lumea-art in #3766
- fix(cpu): preserve partial fused state loads by @Qubitium in #3770
- fix(exllamav2): preserve shuffled words across handle lifecycle by @Qubitium in #3771
- perf(nf12): staged B1, Ampere routers and live-shape embedding by @Qubitium in #3761
- perf(nf12): keep packed embedding lines in L1 during Lite prefill by @Qubitium in #3776
- test: fix subset replay doubles and share the ERNIE capture harness by @ZX-ModelCloud in #3774
- test: fix collection errors and gate CUDA-only kernels on ROCm by @ZX-ModelCloud in #3772
- fix(cpu): keep packed buffers when clearing fused caches by @ZX-ModelCloud in #3773
- perf(nf12): fuse four-copy embedding activation for Lite HC by @Qubitium in #3777
- proto(nf12): row-major packed embedding with fused HC4 stores by @Qubitium in #3779
- perf(nf12): stream SM80 one-ID embedding output stores by @Qubitium in #3778
- fix(nf12): support repeat4 with a preloaded legacy operator set by @Qubitium in #3782
- fix(nf12): invalidate prepared gate view on source changes by @Qubitium in #3783
- fix(exl3): remove temporary assignment link after publication by @Qubitium in #3784
- test(awq): accept group sizes in replay doubles by @Qubitium in #3785
- fix(torch): keep a stored wide bias through the GPTQ fallback epilogue by @ZX-ModelCloud in #3786
- fix(exllamav2 awq): repack replaced checkpoint words before rebind by @Qubitium in #3787
- fix(marlin): validate negative device indices before Torch parsing by @Qubitium in #3788
- fix(gguf): correct CUDA bias cancellation after native matmul by @Qubitium in #3789
- test(ricursive): cover empty accumulation and concurrent tuple outputs by @Qubitium in #3791
- fix(looper): bound disk-backed output staging and verify replay reads by @Qubitium in #3790
- perf(torch): decode quantized embedding lookups by token row by @ZX-ModelCloud in #3797
- test(moe): realign expert batching with the causal replay pass by @Lumea-art in #3795
- bench(qlinear): w4a16 decode benchmark over real model shapes by @Lumea-art in #3560
- fix(models): declare Gemma 2/3 and OLMo 2 decoder module plans by @ZX-ModelCloud in #3793
- fix(allspark): keep the checkpoint layout exportable after repack by @ZX-ModelCloud in #3792
- fix(allspark): add the linear bias in the FP32 epilogue by @ZX-ModelCloud in #3794
- fix(gemma2): reject unsupported backends with the documented error by @ZX-ModelCloud in #3798
- fix(torch): index embedding row decode by pack word width by @ZX-ModelCloud in #3799
- fix(awq): keep checkpoint scales and bias precision in Torch forward by @ZX-ModelCloud in #3800
- fix(threadx): join the device thread pool at exit by @Lumea-art in #3801
- fix(awq): pin Torch forward precision regressions and restore non-half dequantize by @ZX-ModelCloud in #3802
- fix(gptq): reject scales lost by FP16 checkpoint storage by @Lumea-art in #3803
- refactor(quantization): centralize lifecycle contracts by @Qubitium in #3796
- fix(torch): keep inference alive when a deferred torch.compile fails by @ZX-ModelCloud in #3809
- feat(tenstorrent): native TTNN, packed XLA and block-float inference by @Qubitium in #3760
- fix(tenstorrent): load block-float checkpoints at their own precision, not fp32 by @Qubitium in #3813
- deps: make NumPy optional in the quantization core by @Qubitium in #3807
- fix(tests): gate CUDA-only kernel tests by build and device ordinal by @ZX-ModelCloud in #3805
- fix(int8): keep checkpoint group scales in the CPU INT8 layout by @ZX-ModelCloud in #3804
- feat(tenstorrent): run bf8/bf4 checkpoints on ROCm with HIP kernels by @Qubitium in #3806
- feat(ext): add CPU kernels for Tenstorrent bf8_b/bf4_b/bf2_b block floats by @Qubitium in #3808
- feat(tenstorrent): run bf8/bf4 checkpoints on CUDA kernels by @Qubitium in #3810
- fix(rocm): resolve JIT toolkit paths from versioned core SDKs by @ZX-ModelCloud in #3816
- chore: require threadpoolctl 3.8.0 by @Qubitium in #3817
- fix: resolve post-merge GPU dispatch and checkpoint restore regressions by @Qubitium in #3818
- Rename NF12 APIs and remove implementation-specific labels by @Qubitium in #3834
- feat(models): add Granite SWA and Granite MoE SWA quantization support by @ZX-ModelCloud in #3833
- test(quant): pin dtype boundary behavior at min/max by @Lumea-art in #3814
- fix(p2p): delegate native VMM peer enablement to PyTorch by @ZX-ModelCloud in #3815
- perf(awq): tile and broadcast the AWQ GEMM dequantization by @ZX-ModelCloud in #3819
- chore: require transformers 5.22.2 on Python 3.11+ by @Qubitium in #3835
- fix(awq): bound large-group tiles and preserve scale gradients by @ZX-ModelCloud in #3839
- fix(models): explicitly exclude Granite SWA attention sinks by @ZX-ModelCloud in #3838
- chore: require torch 2.15.0 on Python 3.11+ by @Qubitium in #3837
- fix(threadx): balance warmup no-ops against the in-flight drain counter by @ZX-ModelCloud in #3836
- feat(models): add OLMo Hybrid quantization support by @ZX-ModelCloud in #3820
- fix(gemma3): restore sharded and language checkpoint keys by @ZX-ModelCloud in #3840
- perf(nf12): sync upstream Blackwell CUDA kernels by @Qubitium in #3841
- fix(looper): stage serial forward on the surviving filtered device by @ZX-ModelCloud in #3842
- fix(tenstorrent): preserve checkpoint architecture and bias precision by @ZX-ModelCloud in #3843
- fix(jamba): quantize routed expert projections by @ZX-ModelCloud in #3844
- fix(packing): restore GPTQ runtime zeros and preserve repack buffer dtypes by @ZX-ModelCloud in #3845
- fix(writer): accept safe nested per-layer shard paths in EXL3 passthrough by @ZX-ModelCloud in #3846
- fix(rotation): avoid FP16 overflow in portable Hadamard fallback (#3827) by @ZX-ModelCloud in #3847
- release: prepare 7.11.0 and refresh hardware documentation by @Qubitium in #3849
- fix(calibration): enforce remote image total_timeout across headers and redirects by @ZX-ModelCloud in #3848
- fix(tenstorrent): keep wider block-float embeddings off the CUDA kernels (#3830) by @ZX-ModelCloud in #3852
- fix(vptq): reject group layouts with empty trailing codebooks by @ZX-ModelCloud in #3851
- fix(config): restore P2P policy from saved quantization metadata by @ZX-ModelCloud in #3850
- fix(calibration): reap timed-out image fetch workers by @Qubitium in #3855
- docs: move proposed feature list out of README by @Qubitium in #3856
- feat(nf12): add automatic source dtype casting by @Qubitium in #3853
- refactor(nf12): slim AUTO_NF12 runtime implementation by @Qubitium in #3858
- fix(calibration): include the corpus tail in calibration windows (#3861) by @Qubitium in #3883
- fix(calibration): concatenate every row of a multi-row item (#3865) by @Qubitium in #3887
- fix(tenstorrent): keep fp32 bias precision through block-float forward (#3863) by @Qubitium in #3889
- fix(gptq): merge finalized and pending samples in hessian snapshot (#3862) by @Qubitium in #3890
- fix(nf12): fall back when native row-major kernels are unavailable (#3860) by @Qubitium in #3891
- fix(models): forward explicit load options on quantized-checkpoint redirect (#3864) by @Qubitium in #3892
- fix(paroquant): keep fp32 reduction and bias epilogue (#3866) by @Qubitium in #3893
- feat(bench): record the probed device limits with the results by @Lumea-art in #3894
- fix(causal-conv1d): write only unique circular cache slots (#3868) by @Qubitium in #3895
- fix(quantization): make VPTQ and QQQ Hessian materialization repeatable (#3869) by @Qubitium in #3896
- fix(quantization): keep asymmetric SmoothMSE zero points representable (#3867) by @Qubitium in #3897
- fix(causal-conv1d): leave empty varlen sequences zeroed (#3871) by @Qubitium in #3898
- fix(mmlupro): pass attention mask and keep prompt counting on CPU (#3875) by @Qubitium in #3899
- fix(vllm): reject attention masks a token prompt cannot represent (#3884) by @Qubitium in #3900
- fix(mmlupro): bound prompt-length retries at zero-shot (#3876) by @Qubitium in #3901
- fix(mmlupro): score each prediction set once (#3877) by @Qubitium in #3902
- fix(data): enforce sample and block limits in make_data_block (#3882) by @Qubitium in #3905
- fix(nf12): dequantize row-major embeddings on the packed device (#3885) by @Qubitium in #3907
- fix(qqq): implement ungrouped Conv1d and reject grouped convolutions (#3870) by @Qubitium in #3906
- fix(data): honor default preprocessor and reject streaming datasets (#3880) by @Qubitium in #3904
- feat(quantize): print a one-line summary when a run finishes by @Lumea-art in #3909
- fix(adapter): match rank_pattern suffixes and regex patterns (#3873) by @Qubitium in #3908
- fix(adapter): preserve case-sensitive module names in PEFT load/export (#3874) by @Qubitium in #3911
- fix(nf12): reuse a preloaded partial operator namespace (#3886) by @Qubitium in #3910
- test(p2p): make enablement policy tests host-topology independent (#3881) by @Qubitium in #3912
- fix(sglang): alias rocm to cuda only on a HIP build (#3888) by @Qubitium in #3903
- chore(paroquant): sort the new numerics import (ruff I001 on main) by @Qubitium in #3913
- fix(loader): resolve nested checkpoints under the requested subfolder (#3879) by @Qubitium in #3914
- fix(threadx): acquire each distinct lock once per lock scope (#3878) by @Qubitium in #3915
- feat(security): refuse to start on a vulnerable PyTorch runtime (CVE-2025-3000) by @Qubitium in #3920
- fix(adapter): match rank patterns without rewriting their case (#3918) by @Qubitium in #3923
- fix(quantization): honour a changed target device when reusing a Hessian (#3916) by @Qubitium in #3924
- test(module_tree): give looper fixtures the AUTO_NF12 caster contract (#3872) by @Qubitium in #3926
- fix(paroquant): gate native AWQ GEMM on its real shape and device contract (#3919) by @Qubitium in #3934
- fix(nf12): mask out-of-range IDs in the embedding fallback (#3922) by @Qubitium in #3940
- fix(vllm): keep each row's original prefix width in generate output (#3921) by @Qubitium in #3941
- fix(security): align the Torch gate with every affected range and pinned environment (#3933, #3929, #3930) by @Qubitium in #3942
- fix(models): report peak VRAM for the quantization devices (#3931) by @Qubitium in #3945
- fix(quantization): reproduce convolution padding during capture (#3917) by @Qubitium in #3949
- fix(quantization): consume the Hessian on the layer's device (#3916) by @Qubitium in #3953
- fix(bench): reject --iters below two in the Ada kernel benchmark (#3937) by @Qubitium in #3952
- fix(bench): profile and time the --device target, not the current GPU (#3925) by @Qubitium in #3950
- fix(gptq): give Hessian snapshots their own storage (#3862) by @Qubitium in #3954
- fix(sglang): default the public generate path to greedy decoding (#3935) by @Qubitium in #3956
- fix(threadx): drain accepted work and reject submissions after shutdown (#3927) by @Qubitium in #3958
- fix(models): forward the checkpoint subfolder and revision to dispatch (#3879) by @Qubitium in #3957
- fix(tt-blockfloat-hip): reject exponents from a different device (#3943) by @Qubitium in #3959
- fix(adapter): keep cached adapter weights intact across loads (#3936) by @Qubitium in #3960
- fix(nf12): probe the row-major operators and survive a failed pack (#3860) by @Qubitium in #3946
- fix(models): emit the quantize summary after finalization on every path (#3932) by @Qubitium in #3955
- fix(tt-blockfloat): gate native CUDA tests on real extension availability (#3928) by @Qubitium in #3962
- fix(requirements): pin defuser 0.0.30 for the Tenstorrent manifest (#3938) by @Qubitium in #3963
- fix(p2p): bind cached peer verdicts to the policy generation (#3947) by @Qubitium in #3964
- fix(exllamav3): synchronize the EXL3 native dispatch caches (#3951) by @Qubitium in #3966
- ci(gil0): add a dedicated PYTHON_GIL=0 lane and GIL assertion (#3948) by @Qubitium in #3965
- fix(tt-blockfloat-hip): synchronize the WMMA device cache (#3944) by @Qubitium in #3961
- fix(paroquant): constrain output widths to the packer's INT4 alignment (#3939) by @Qubitium in #3967
- fix(security): contain checkpoint-derived paths in reshard and save (#3969) by @Qubitium in #3979
- fix(security): gate checkpoint code imports on trust_remote_code (#3968) by @Qubitium in #3981
- fix(security): bound OpenAI server and media fetches (#3970) by @Qubitium in #3980
- fix(gil0): propagate thread-pool errors and isolate Triton autotuner state (#3972) by @Qubitium in #3986
- fix(audio): probe audioread without an unused import (unblocks ruff-check on main) by @Qubitium in #3987
- fix(security): validate checkpoint index tensors before native indexing (#3971) by @Qubitium in #3988
- fix(awq): fold scales into the correct norm and resolve offset gains (#3973) by @Qubitium in #3989
- fix(adapter): honor LoRA and EoRA scaling (#3975) by @Qubitium in #3990
- feat(tt-blockfloat): solve GPTQ with exponent blocks along out_features (TT-native layout) by @Qubitium in #3992
- fix(awq): decode 8-bit packed readers in natural order (#3974) by @Qubitium in #3993
- feat(tt-blockfloat): checkpoint/module format for out-axis exponent blocks by @Qubitium in #3994
- fix(quantization): native calibration batch split, 3-bit pack contract, fallback group offset (#3978) by @Qubitium in #3995
- fix(dtype): preserve checkpoint and packed tensor precision (#3976) by @Qubitium in #3997
- perf(qlinear): avoid full-size FP8 scales and lower AWQ GEMM threshold (#3984) by @Qubitium in #3996
- feat(tenstorrent): device-resident Llama decode for out-axis block-float checkpoints by @Qubitium in #3998
- perf(dequant): vectorise and device-preserve qzeros pack/unpack (#3985) by @Qubitium in #3991
- perf(triton): fuse TritonV2 dispatch and pack-load small-M GEMM (#3982) by @Qubitium in #3999
- perf(kernels): cache per-call dispatch decisions (#3983) by @Qubitium in #4000
- fix(qlinear): keep FP16 reductions wide and stop BF16-through-FP16 quantization (#3977) by @Qubitium in #4001
- feat(tt-blockfloat): sequential block-by-block GPTQ calibration by @Qubitium in #4009
- feat(tenstorrent): paged KV, chunked prefill and quantized-only storage for TT Llama (8k-64k) by @Qubitium in #4010
- fix(tenstorrent): size the TT Llama KV pool to at least one page-table row by @Qubitium in #4011
- fix(nf12): keep excluded-layer replay dense and fail closed on leaked AUTO_NF12 buffers (#4007) by @Qubitium in #4012
- fix(looper): replay each layer with its own layer-type mask and RoPE (#4004) by @Qubitium in #4019
- docs(test): correct the AUTO_NF12 excluded-layer page and harden its ROCm bound (#4007) by @Qubitium in #4018
- fix(rotation,calibration,qqq,w4a): device-aware Hadamard, truncation side, QQQ export scale, exact FP8 divide (#4008) by @Qubitium in #4016
- Update requirements.txt by @Qubitium in #4022
- test(nf12): widen the unaffected-exclusion logit bound for ROCm (#4007) by @Qubitium in #4021
- Update requirements.txt by @Qubitium in #4027
- fix(awq,adapter): honor norm data flow and PEFT pattern rules (#4002) by @Qubitium in #4023
- fix(looper): build each layer-type replay mask with the transformers builder table (#4004) by @Qubitium in #4024
- fix(exl3): refresh deferred runtime weights per layer forward and align generic geometry (#4006) by @Qubitium in #4026
- fix(quantization): unify FP8 scale semantics, resave bias dtype, sym fallbacks, 3-bit validate (#4003) by @Qubitium in #4025
- fix(openai,mmlupro,vllm): handle multi-EOS lists, SDK request fields, and tokenizer outputs (#4014) by @Prajwal7-2002 in #4029
- fix(awq): build scaling groups for norm-less trees and skip non-sources (#4005) by @Qubitium in #4030
- fix(awq): dispatch dequantization on the saved AWQ layout and per-module bits (#4002) by @Qubitium in #4031
- fix(qlinear): refresh fused dispatch and bias epilogues (#4028) by @Qubitium in #4032
- fix(server): isolate the seeded RNG state per request (#4014) by @Qubitium in #4033
- fix(checkpoint): accept 5.4.x gptaq booleans and quantize untied lm_head (#4013) by @Qubitium in #4034
- fix(moe): scale AWQ norm consumers, split MLA fold groups, guard ERNIE patch, drop padding rows (#4017) by @Qubitium in #4035
- docs: refresh 7.11.0 README changelog by @Qubitium in #4036
- perf(quantization): release replay activations and dedup AWQ captures (#4020) by @Qubitium in #4043
- fix(calibration): replay base-type masks, honor super() overrides, type AWQ rotary (#4038) by @Qubitium in #4044
- fix(awq): resolve typed rotary layer type through nested self_attn (#4038) by @Qubitium in #4045
- fix(fp8,foem): preserve packed save payloads and fix capture/block-boundary paths (#4037) by @Qubitium in #4046
- fix(kernels): pack strided inputs, bind devices, widen offsets, handle M=0 (#4042) by @Qubitium in #4047
- fix(module-tree): order consumers after producers and declare missing linears (#4041) by @Qubitium in #4048
- fix(models): transformers 5.22 loader, mixer guards, tree root and META buffers (#4040) by @Qubitium in #4049
- fix(models): replay multimodal outer-loop state during calibration (#4015) by @Qubitium in #4050
- fix(models): support Transformers 5.22 trees and calibration (#4040) by @Qubitium in #4055
- fix(config): normalize EXL3 serialization and reject silent kwarg drops (#4052) by @Qubitium in #4057
- fix(awq): cover every split-K tile and bound gemmv2 k-tiles (#4054) by @Qubitium in #4058
- fix(models): honor third-party checkpoint interop in save, Marlin, and export (#4053) by @Qubitium in #4059
- fix(moe,awq,kernels,server): keep padding out of expert Hessians and restore live paths (#4051) by @Qubitium in #4060
- deps(defuser): require 0.0.31 for GLM-4.x router tuple by @Qubitium in #4056
- fix(models): honor third-party checkpoint filters and layouts (#4053) by @Qubitium in #4061
- fix(dispatch): CPU group-size gate, CPU-offload kernel choice, max_memory, compile paths (#4054) by @Qubitium in #4062
- fix(awq): correct scaling groups across dense and MoE families (#4039) by @Qubitium in #4065
- test(kernels): resolve the generic torch alias to GGUF_TORCH (#4059 regression) by @Qubitium in #4064
- fix(looper,gptq): restore EXL3 AUTO_NF12 and the in-memory v2 qzero layout (#4063) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4069
- fix(awq): aggregate variable-length calibration for every module (#4063) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4071
- fix(hf): run native MoE bridge, remap legacy VL keys, retune AWQ groups (#4073) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4084
- fix(rocm): install deterministic ROCm env defaults (#4072, #4075) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4079
- fix(gptaq,foem,gguf): unblock MoE experts and warn on K-quant aliases (#4067) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4080
- fix(awq,qqq,paroquant): gauge offset-gain folds and fix QQQ/ParoQuant load paths (#4074) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4081
- fix(dispatch): repair CPU optimize/compile, max_memory and HIP/autograd gates (#4068) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4082
- fix(mlx): accept generate() inputs and honor dynamic on export (#4076) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4083
- fix(checkpoint): repair v1 state_dict loads, HF quant methods, pickling (#4070) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4085
- fix(features): honor GPTQ dynamic exclusions and repair AUTO_NF12/EoRA/rotation/GSQ (#4077) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4086
- fix(multi-gpu): release worker task refs, StopForward tracebacks, and ExLlamaV2 device (#4078) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4088
- feat(macaq): add native MACaQ MXFP4 W4A4 quantization by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4066
- fix(awq): size scale-search replay by sequence rows and inherit fallback threshold (#4087) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4089
- feat(mlx): add Metal kernels for MACaQ MXFP4 and NF12 by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4097
- fix(macaq): int64 row offsets, device-bound Triton launches, sm_120 K tiles (#4090) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4099
- feat(protocol): resolve rule actions via registry and implement smoothquant (#3781) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4106
- fix(awq): validate group size against the packing kernel (#4095) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4104
- fix(concurrency): refcount global guards and serialize Triton JIT (#4094) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4102
- fix(models): restore native remote-code RoPE and repair load/generate paths (#4093) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4108
- perf(mlx): fuse the NF12 small-token tile and the MACaQ rotation by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4098
- fix(interop): HF quant refusers, optimize() pickling, CPU-only max_memory, MLX masks/export (#4091) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4101
- fix(models): correct remote-code AWQ trees, coverage, position replay, and looped calibration (#4093) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4109
- fix(packing): repair hf_repack_layer and hf_pack_layer GPTQ v1 qzero format alignment (#4095) by @Prajwal7-2002 in https://github.com/ModelCloud/GPTQModel/pull/4096
- fix(awq): replay packed calibration rows with their own mask and positions (#4091) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4110
- fix(checkpoint): make state_dict() a real checkpoint for every kernel (#4091) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4103
- fix(looper): resolve concurrent module-tree branches (#1998) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4111
- test(mlx): skip Metal-only tests when no Metal backend is present (#4098) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4113
- fix(rocm): scope the deterministic BLAS pin and stage HIP sources (#4092) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4105
- fix(kernels): gate GGUF AUTO selection by qtype and resolve AWQ GEMV on ROCm (#4107) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4114
- docs: refresh 7.11 highlights and archive older releases by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4115
- docs: make README a concise project overview by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4118
- docs: restore README model coverage and citations by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4119
- docs: welcome contributors and project sponsors by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4122
- perf(nf12): retune the MLX Metal kernels on the RDNA4 short-M shapes by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4120
- fix(protocol): resolve smoothquant by tensor identity and verify each fold (#4117) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4124
- fix(looper): capture calibration inputs per parallel branch (#4116) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4126
- fix(cpu): refcount the BF16x3 matmul guard and drop the EXL3 duplicate (#4116) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4127
- perf(nf12): keep the MLX Metal decode in the dot product through 15 rows by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4128
- perf(nf12): share activation loads across column blocks in the Metal dot product by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4133
- fix(awq): preserve the captured mask form and fallback precedence (#4121, #4125) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4131
- fix(models): run the pre-load hook on native loads and share one legacy generate adapter (#4121) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4132
- fix(checkpoint): refuse runtime-layout HF saves, emit AWQ snapshots, carry layout in a tensor (#4125) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4134
- fix(awq): resolve multimodal rotary and stop folding across residual/nonlinearity (#4121) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4135
- feat(nf12): sign the CUDA/ROCm kernel ABI and add the MLX embedding variant by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4136
- fix(qlinear): restore bare-module state_dict() and classify MPS kernels in the audit (#4130) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4138
- docs: add Fujitsu and IBM Research usage references by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4139
- fix(models): serialize model loads and reject corrupted state (#4129) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4140
- fix(rocm): pin every Hessian builder and serialize mixed BLAS scopes (#4123) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4141
- feat(nf12): pin the upstream ABI and consume it verbatim by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4143
- fix(vptq): scope deterministic algorithms for gpu weighted kmeans (#4137) by @Prajwal7-2002 in https://github.com/ModelCloud/GPTQModel/pull/4142
- fix(awq): preserve scale-search routing and reject incomplete replay groups by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4144
- docs: link README methods and accuracy terms to guides by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4146
- fix(bloom): preserve causal AWQ replay and reject unsafe norm folds by @Prajwal7-2002 in https://github.com/ModelCloud/GPTQModel/pull/4145
- docs(agents): require AWQ safety qualification for model onboarding by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4147
- docs: fix README links outside GitHub by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4148
- fix(quantize): replay each layer with its own calibration context (#4150) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4153
- fix(awq): replay MoE groups under the capture routing override (#4151) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4154
- fix(checkpoint): stop HF save_pretrained from writing qzeros_format markers (#4152) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4155
- fix(vptq): reduce k-means segments locally instead of toggling global determinism (#4149) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4156
- build(deps): bump actions/upload-artifact from 4 to 7 in the github-actions group by @dependabot[bot] in https://github.com/ModelCloud/GPTQModel/pull/4157
- fix(gsq): recover AWQ codes with the packer's arithmetic by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4159
- fix(paroquant): fit affine grids that contain one-sided groups by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4160
- fix(quantizer): fit symmetric ranges around zero for non-negative groups by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4161
- fix(gsq): score learned AWQ scales in their returned dtype by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4162
- test(quantizer): correct symmetric range oracles and packed coverage by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4164
- fix(paroquant): stabilize affine fitting and bounded exports by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4165
- fix(w4a): keep FP8 token scales positive when the division underflows by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4185
- MACaQ: rotation-free W4A16 mode, identity fast paths, stock MLX export by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4163
- fix(w4a): keep NVFP4 candidate ranking finite for large activations by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4186
- fix(awq): reject unrepresentable scale storage before packing by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4188
- fix(checkpoint): hash source files through HF snapshot symlinks by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4189
- fix(awq): normalize additive attention masks as non-negative (#4187) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4190
- fix(models): gate model rotation on explicit per-model-type opt-in (#4169) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4208
- fix(models): keep declared fp32 modules exempt from the load dtype guard (#4170) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4209
- test(gil0): probe the VPTQ k-means segment reduction by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4212
- fix(gptq): permute W once when the Hessian permutation OOMs (#4168) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4230
- fix(checkpoint): one qzeros layout invariant across load, save and pack (#4166, #4167, #4171) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4235
- perf(threads): cap torch intra-op thread pools for tests and CLI tools by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4236
- fix(looper): key shared KV cache per batch and refresh on replay (#4182) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4242
- fix(awq): replay with the layer's attention type, real rotary positions and explicit fallbacks (#4176) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4243
- fix(fp8): keep TorchFP8Linear weight-only unless W8A8 is explicitly requested (#4177) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4244
- fix(models): declare every decoder leaf for issue #4174 (#4174) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4245
- test(looper): drop wrapper lambdas flagged by code quality (#4182) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4246
- fix(awq): keep checkpoint scales and bias out of the activation dtype (#4178) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4247
- fix(gptq): share RTN fallback and preserve act-order retries (#4179) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4248
- fix(awq): preserve output for gated/lerped scale folds (#4175) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4249
- fix(marlin): seed g_idx with the canonical contiguous index (#4239) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4250
- test: fix test-side failures from ROCm, GGUF qtype and Hessian order (#4234) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4251
- fix: stage pre_quantize layers on the filtered quantization device (#4233) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4252
- fix: pin staged GSQ GPTQ to the deterministic BLAS backend (#4232) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4253
- fix: detect ROCm Triton distributions in nogil autotuner patch (#4231) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4254
- fix: reject original LLaVA checkpoints with a clear error (#4229) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4255
- fix: keep the base text path for pre-tokenized VLM calibration rows (#4228) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4256
- fix: keep the final assistant turn in InternVL-chat calibration (#4227) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4257
- fix: replay ParoQuant with the owning branch's masks/positions (#4226) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4258
- fix: serialise MACaQ per-layer rotation under its own key (#4225) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4259
- fix: make VPTQ segment sum deterministic without a float cumsum (#4224) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4260
- fix: replay all experts during bypass-mode AWQ scale search (#4223) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4261
- fix: use logical CUDA index for BitBLAS target detection (#4222) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4262
- fix: derive MacaqLinear torch compute dtype from the input (#4217) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4263
- fix(looper): stop copying a layer onto the device it already lives on (#4183) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4270
- fix: size the QQQ workspace from the kernel's 64-column lock tiles (#4215) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4269
- fix: drop group_size=32 from AwqMacheteLinear, which native Machete rejects (#4218) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4267
- fix: share one GGML CUDA weight copy across every batch-size plan (#4216) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4268
- fix: make the GGUF_CPP CPU matmul path capability-based instead of timed (#4219) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4266
- fix: propagate NaN row scales in the W4AFP8 and QQQ Triton quantizers (#4220) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4265
- fix(calibration): strip padding before the position budget by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4271
- fix(fusion): drop pre-fusion ExLlamaV2 operand snapshots on rebind (#4240) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4272
- fix: round ParoQuant bf16 once, use FP32-reduce MACaQ matmuls, keep MLX QQQ bf16 wide (#4221) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4264
- docs(gil0): add missing ledger rows for threaded linalg, streams, and threadpool APIs by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4273
- fix(gguf): bound the GGUF_CPP activation-quantization loss and make the native/exact path rule static (#4219) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4280
- perf(allspark): keep a representable bias on the half-precision path by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4281
- fix: refcount process-wide threadpool limits (#4206) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4279
- fix: serialize save-side shell build, dequant and TT from_config under the model load lock (#4204) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4278
- perf(auto): stop ranking ExLlamaV2 first for bf16/fp32 GPTQ (#4213) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4277
- fix: reject DoRA and other unsupported PEFT adapter features (#4211) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4276
- fix(calibration): normalize causal masks and validate rows per token by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4274
- fix(adapter): export EoRA rank_pattern from trained ranks (#4199) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4287
- fix: cache and release LazyTurtle .bin conversions (#4205) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4285
- fix(gptaq): accumulate per-device Hessians and pair native inputs by batch (#4237) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4284
- fix(adapter): honour PEFT target modules and apply dense targets (#4210) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4283
- fix: keep
-:dynamic exclusions first on every mutation (#4198) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4289 - fix: compile protocol -: selectors with search semantics (#4197) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4288
- fix: fold AWQ modules_to_not_convert into
-:dynamic rules (#4195) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4290 - fix(looper): accept calibration inputs without len() (#4192) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4291
- fix(calibration): stop duplicate BOS in chat-template rows and concat separators by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4292
- fix(calibration): unpack columnar mappings instead of calibrating on keys by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4293
- fix: translate producer exclusions at module boundaries (#4196) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4294
- fix: restore offloaded base modules after quantization (#4203) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4295
- fix: serialize object-valued dynamic overrides and lift meta on factory reload (#4201) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4296
- test: realign stale CUDA suite expectations with current contracts (#4241) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4298
- fix: validate effective per-module dynamic settings (#4200) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4299
- fix: isolate offload bundles per quantize run (#4202) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4297
- fix(cpu): align int4pack kernel contracts with the packed op by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4326
- fix(checkpoint): tighten metadata, tied-head, fallback and scale contracts by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4327
- fix: fold AWQ exclusions set after construction and drop stale claims (#4309) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4328
- fix: materialise one-shot calibration input for native and EoRA passes (#4307) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4329
- fix: convert v1 qzeros on assign=True state dict loads (#4304) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4330
- fix: unpack VLM calibration Mapping columns instead of chat key names (#4308) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4332
- fix: filter empty-text calibration rows instead of aborting (#4306) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4331
- fix: resolve the export shell for AWQ GEMV/GEMV_FAST re-save (#4305) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4333
- fix: keep packed and scale buffer dtypes on checkpoint re-save (#4303) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4334
- fix: reject PEFT trainable_token_indices adapters instead of dropping deltas (#4310) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4335
- style: fix ruff failures on main (unused import, dict literal) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4337
- fix: serialize GGUF CUDA forward per device (#4314) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4336
- fix: rehome GPTAQ/FOEM dXXT with H on multi-GPU (#4312) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4338
- fix: derive the MACaQ Triton compute dtype from the input (#4315) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4339
- fix(loader): put LazyTurtle pickle conversion under offload_to_disk_path by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4340
- fix: release the AWQ per-layer ROCm allocator cache (#4313) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4342
- fix: make exact GGUF_CPP CPU matmul selectable and correct accuracy docs (#4316) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4341
- fix(bailing_moe): group shared-expert down_proj after gate/up (#4325) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4343
- fix(macaq,nvfp4): propagate NaN/Inf activation blocks instead of a finite code by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4344
- test: align FOEM oracle with token-count normalisation (#4322) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4346
- fix(threadx): skip warmup for devices excluded from quantization by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4347
- fix(w4a): declare the source qzero layout before a native re-save (#4323) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4348
- fix: accept datasets columns and close calibration input gaps (#4311) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4352
- fix: publish processor quant log for every method, not an allow-list (#4321) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4353
- fix(looper): publish the per-module quant log for every quantizer (#4321) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4354
- test(gptaq): encode FOEM's token-normalized Hessian in the routed-expert oracle (#4322) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4345
- fix(gptq): classify solver allocation failures for CPU fallback (#4350) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4359
- test: realign ROCm suite expectations, thread-cap probe and qzeros marker by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4360
- fix: preserve bias dtype across checkpoint restore (#4324) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4362
- fix(exllamav2): serialize the cold native q-matrix rebuild; #4184 review dispositions by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4361
- fix: restore dense source for pruned MoE experts before save (#4349) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4363
- refactor(kernel): select kernels, numerics and threads through KernelConfig, not env vars (#4180) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4364
- fix: round ParoQuant bf16 rotation once in the fused kernel (#4317) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4365
- Unify backend selection and tuning with typed BackendConfig by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4375
- fix(gsq): keep the fused relaxation exponent and probability in FP32 (#4357) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4376
- fix(threadx): give every device-pool worker its own intra-op team (#4356) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4390
- fix(kernel): import resolve_kernel_config in ParoQuantTritonLinear and restore the Ruff gate by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4366
- fix(models): share the Nemotron-H layer declaration with Nemotron Omni (#4355) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4392
- fix: repair the BaseQModel attr recursion and the transformers-5.22 cache shim, and realign the stale tests from #4358 by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4397
- fix(loader): omit forced offload_state_dict to prevent TMPDIR copies (#4385) by @Prajwal7-2002 in https://github.com/ModelCloud/GPTQModel/pull/4399
- fix(vlm): reject multimodal calibration rows instead of dropping tensors (#4371) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4402
- fix(qlinear): dequantize to model compute dtype, not scales dtype (#4370) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4403
- fix(config): serialize true_sequential so it round-trips (#4382) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4407
- fix(looper): publish one quant-log row per module by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4406
- fix(generation): preserve checkpoint do_sample instead of forcing sampling (#4383) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4408
- fix(offload): remove per-run offload directory after save, teardown, or failure (#4368) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4409
- fix(looper): detect tied lm_head from storage identity, not nested config (#4369) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4410
- fix(loader): treat bare integer device as accelerator ordinal (#4384) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4411
- test(awq): gate the CPU fold invariants on an eager Torch Mamba-2 path by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4413
- fix(offload): keep per-run bundles after save so in-session forward works by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4416
- test(awq): accept the per-layer syms kwarg in stale best-scale mocks by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4415
- fix(qlinear): convert v1 qzeros copied into fresh v2 kernels (#4377) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4414
- fix(checkpoint): preserve checkpoint buffer dtypes on re-save (#4379) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4417
- fix(definitions): declare untagged decoder linears (#4374) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4422
- fix(export): fail closed on unknown formats and keep dense exports loadable (#4386) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4418
- fix: offload the live qzero layout so restored layers decode as v2 (#4380) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4425
- fix(module-tree): quantize consumers after producers in six definitions (#4372) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4427
- fix(awq): resolve fp16-only GEMV dtype before calibration (#4394) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4424
- fix: honour cpu_workers and live kernel policy; warn on retired env vars (#4396) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4423
- fix(gguf): serialize ggml-cuda entry process-wide (#4378) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4421
- fix: restore base modules and re-pick CPU kernels after weight-only quantize (#4381) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4426
- fix: stop forcing max_split_size_mb on CUDA allocator config (#4389) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4419
- test(cpu): pin the #4181 int4pack dispatch and dequantize contracts by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4428
- fix(models): honor Ling-2.6-flash remote-code layout and declare MLA projections (#4373) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4429
- fix(macaq): release dead rotation buffers before flip sweep (#4401) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4431
- fix(exl3): release per-stream tile scratch at end of run (#4391) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4432
- test(models): gate flash-attn eval args and pin QQQ test to fp16 (#4398) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4433
- test(awq): widen CPU dense logit-RMSE bound to measured cross-CPU range by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4435
- fix(awq,bitsandbytes): correct triton schedule-mode plan and reload fp32 quant state (#4388) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4434
- fix(gptq): release shared-input Hessian sources after adoption (#4400) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4430
- fix(kernels): share QQQ reduce scratch and GGUF CUDA plan arena, cut QQQTorch temporaries (#4387) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4436
- fix(vptq): bound distance and Hessian temporaries (#4391) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4438
- fix(awq,qlinear): keep clip scratch splittable and bound pack_gpu transient (#4401) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4437
- fix(gptq): add opt-in attention-sink masking for the calibration Hessian (#4393) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4440
- fix(cpu): stop host-memory duplication in GGUF CPU, offload, and pinned staging (#4395) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4439
- fix(awq): honor the per-module Triton schedule mode after #4434 (#4388) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4441
- fix(loader,gptq,qqq,exl3): complete the #4400 memory fallbacks by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4442
- fix(looper): share bitwise-identical native sibling inputs (#4391) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4443
- test(llama3_2): exclude attention sinks and refresh slow baselines (#4393) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4444
- test(models): fix silently-skipping tests, auto eval batch OOM, refresh Qwen2.5 slow baselines (#4398) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4445
- fix: preserve generation config when resaving quantized checkpoints by @mikemikimike in https://github.com/ModelCloud/GPTQModel/pull/4489
- fix(qqq): initialize cache after kernel rehome by @mikemikimike in https://github.com/ModelCloud/GPTQModel/pull/4490
- fix: native class for natively supported remote-code checkpoints; keep checkpoint dtype/device on dequant/export; accept VLM calibration metadata (#4453, #4455) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4491
- fix(gptq,qqq): exact OOM block retry, static_groups CPU fallback, release QQQ Hessian (#4464, #4463) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4492
- fix(load): load quantized-module buffers at their checkpoint dtype (#4458, #4452) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4495
- fix: bound offload source-reference RSS and stop cached kernel-validation failures pinning models (#4463, #4465) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4496
- fix(backend): keep resolved policy out of QuantizeConfig.kernel; apply every pool setting (#4446, #4448, #4449) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4497
- fix(offload): leave a normal in-session model after disk-offloaded quantize (#4450) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4498
- fix(paroquant): race-free, plan-invariant and correct fused rotation gradient; round-13 test triage (#4461, #4460, #4462) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4499
- fix(moe): GPTAQ/FOEM flattened-row native split, share-cache release, AWQ fallback=None expert prune (#4457, #4463, #4456) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4500
- fix(reshard): recover root from nested shard by @mikemikimike in https://github.com/ModelCloud/GPTQModel/pull/4493
- fix(writer): skip removed checkpoint modules by @mikemikimike in https://github.com/ModelCloud/GPTQModel/pull/4494
- fix(scalar_type): compare NanRepr members instead of their integer values by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4501
- fix(threadx): accept one-shot device iterables in DeviceThreadPool by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4502
- fix(int8): crop padded input rows in the cold GPTQ source decode by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4503
- fix(conv): keep per-axis padding order when capturing Conv2d by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4504
- feat(kv_cache): estimate what context costs next to the quantized weights by @Lumea-art in https://github.com/ModelCloud/GPTQModel/pull/4158
- fix(paroquant): deterministic ROCm GEMMs and config-only autotune knobs (#4460) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4505
- fix(qqq): RTN fallback for uncaptured modules and missing inverse (#4468) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4506
- fix(export): preserve dense Parameter dtypes and repack checkpoint snapshots (#4466) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4507
- feat(kv_cache): expose the estimate on the model and log it at load by @Lumea-art in https://github.com/ModelCloud/GPTQModel/pull/4510
- fix(int8): pad group codes to the kernel's 8-wide contraction alignment by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4512
- fix(awq): search a partial output tile instead of rejecting the width by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4513
- fix(loader): keep packed FP8 weights out of the accelerate cast (#4387) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4509
- fix(looper): release subset capture hooks on cancel and failure by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4515
- fix(loader): report KV-cache quantization as KV-cache quantization by @Lumea-art in https://github.com/ModelCloud/GPTQModel/pull/4508
- fix(kv_cache): count chunked attention, skip non-attention blocks, allow fp8 by @Lumea-art in https://github.com/ModelCloud/GPTQModel/pull/4516
- fix(looper): dedup shared-input Hessians for QQQ/VPTQ/MACaQ/EXL3 and GPTAQ/FOEM (#4401, #4391) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4514
- fix(cpu): serialize BF16x3 capability probing with the precision owner (#4486) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4517
- fix(ttbfp): publish the CPU capability cache through a thread-safe static (#4485) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4518
- fix(jit): stage HIP sources under the build lock (#4487) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4519
- fix(bitsandbytes): preserve batched activation dims in INT8 forward (#4479) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4520
- fix(tt_blockfloat): define the CUDA execution policy for tied heads (#4482) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4521
- fix(mlx): cover NF12 short-row output tails in dot-product dispatch (#4478) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4523
- fix(qqq): own reduce scratch per CUDA stream (#4488) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4524
- fix(config): reject exclude_attention_sinks on paths that ignore it (#4447) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4525
- fix(gptq): mask attention-sink position only for a real BOS token (#4459) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4526
- fix(triton): resolve autotuner future on pruning exceptions (#4477) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4527
- test(awq): cover the partial output tile and explicit non-2-D error (#4474) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4528
- fix(macaq): finite balance pre-scale for single-output weights (#4472) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4529
- fix(exl3): reclaim interrupted projection publication temporaries (#4473) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4530
- fix(ttnn): key device reuse and grid caches by requested placement (#4484) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4522
- fix(glm4v): defuse fused MLP before strict plan resolution (#4481) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4531
- fix(cpu-int4): rebuild checkpoint qweight instead of keeping a clone (#4395) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4532
- fix(numerics): keep portable Torch reductions and bias epilogues wide (#4471) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4533
- fix(mlx): keep QQQ scaled reduction and bias wide until one cast (#4534) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4539
- fix(int8): keep the asymmetric FP32 reduction out of outer autocast by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4535
- fix(paroquant): wire the benchmark fused-rotation flag through config (#4541) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4556
- fix(config): content-key dynamic resolution when typed leaves can mutate (#4475) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4553
- fix(awq): size clip scratch per real tile row count (#4540) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4558
- fix(mlx): keep FP32 QQQ activations wide before dynamic quantization (#4537) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4569
- fix(blockfloat): preserve dense weight and bias dtypes on save (#4549) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4564
- fix(kv_cache): honour per-layer attention kind and KV sharing (#4542, #4543, #4544) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4573
- perf(qlinear): pack a whole CPU block per shift/reduce by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4574
- fix(model_test): bound the eval OOM retry when the batch size cannot shrink (#4536) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4576
- fix(nvfp4): widen accumulator bounds mask to int64 (#4551) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4578
- fix(looper): restore the in-flight layer when quantize() is cancelled or fails (#4546) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4580
- fix(qqq): keep act-order permutation on the CPU OOM retry device (#4538) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4581
- fix(tt-blockfloat): allow the shipped auto_map loader past the HF refuser (#4557) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4583
- fix(kernels): read the module kernel snapshot at forward dispatch (#4555) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4585
- fix(exl3): skip zero-row hadamard launches (#4550) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4587
- fix(foem): apply block-boundary beta correction on MLX path (#4548) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4588
- perf(fp8): keep the native GEMM for range-safe bias-free weights (#4545) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4594
- test(blockfloat): cover bias storage range in the native writer (#4549) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4593
- fix(nf12): keep MoE forwards graph-capturable and bound staged escapes (#4547, #4552) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4598
- fix(protocol): fail closed on dropped targets/unknown weight args and mask each calibration pass (#4560, #4559) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4596
- fix(awq): portable CPU GEMV reader so quantize() completes in session (#4554) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4599
- fix(bitsandbytes): pack row-strided INT8 activations (#4584) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4603
- fix(gsq): staged AWQ inference-mode weights and FP8 Gram symmetry tolerance (#4561, #4579) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4602
- fix(cpu): reject partial-group int4pack and serialize first-forward repack (#4571, #4565) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4600
- fix(gguf): keep one resident copy of the packed weight on CUDA and CPU (#4387, #4395) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4577
- fix(tt-blockfloat): make the dense fallback in a native checkpoint loadable by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4570
- fix(marlin): materialize column-strided activations at the GEMM boundary (#4563) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4605
- fix(backend): inherit explicit enum load backend into quantize (#4582) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4606
- fix(models): defuse fused-expert MoE shell on re-save (#4589) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4608
- fix(qqq): apply Fallback.threshold before GPTQ (#4566) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4609
- fix(gptq): retain BOS sink decisions after the calibration dataset is released (#4568) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4610
- fix(offload): finish the in-session materialisation contract (#4572, #4562, #4597) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4611
- fix(threadx): scope worker intra-op budget to each task (#4601) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4612
- fix(fp8): keep the in-session bias FP32 so it matches a reload (#4575) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4613
- docs(agents): require transition-based regression tests and structural contract escalation by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4616
- fix(mlx): translate transformers-5 rope_parameters into the keys mlx-lm reads (#4607) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4617
- fix(gptaq,foem): CPU fallback around the accelerator Hessian inverse (#4604) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4618
- fix(vptq): let AUTO select the fused CUDA kernel (#4591) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4622
- fix(fp4): keep the effective weight in range through the decode paths by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4623
- fix(awq,qqq): validate the packing kernel per module and fail early for QQQ (#4567) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4624
- perf(tt_llama): shrink the decode width as a batch drains by @Lumea-art in https://github.com/ModelCloud/GPTQModel/pull/4625
- fix(qqq): allocate the reduce scratch per call instead of caching it (#4621) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4626
- fix(awq): decode Torch AWQ weights inside the checkpoint scale range by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4629
- fix(looper): fence submitted work before tearing down a failed run (#4595) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4631
- fix(gsq): score the exported codes per row so per-linear GSQ changes GPTQ (#4592) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4628
- fix(gguf): select the fused Torch forward from static gates, not a timing race (#4590) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4627
- fix(loader): validate kernels against the max_memory devices, not every GPU (#4632) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4637
- fix(calibration): use provided input_ids over a co-existing text column (#4635) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4638
- fix(tt-blockfloat): derive the shared exponent exactly so FLT_MAX cannot overflow to NaN by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4639
- test(policy): compose the requested-vs-resolved snapshot contract (#4615) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4641
- fix(tt-blockfloat): score exponent candidates on the normalised block by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4640
- fix(exllamav3): scope TORCH_CUDA_ARCH_LIST to the EXL3 JIT build (#4634) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4642
- fix(rocm): pin EoRA's input Gram to the deterministic BLAS backend (#4614) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4644
- fix(exl3): own kernel scratch per call instead of per device (#4619) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4645
- fix(moe): honour PhiMoE routing override and fail closed without hooks (#4636) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4646
- test(rocm): assert EoRA's Gram runs under the deterministic pin (#4614 follow-up) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4647
- fix(models): route every VLM prepare_dataset through the shared calibration helper (#4633) by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4648
- fix(awq): keep the non-quantized bias out of the scale storage dtype by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4649
- refactor(tt-blockfloat): drop the guards the normalised search made redundant by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4650
- fix(exl3): share one quantization stream and tile scratch per device (#4586) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4654
- fix(gguf): select the checkpoint loader and dequant device from config by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4655
- fix(calibration): unpack DataFrame and NumPy object columns into rows (#4652) by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4656
- fix(marlin): keep the checkpoint bias out of the tile layout and the output dtype by @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/4657
- docs: extend 7.11.0 README highlights by @Qubitium in https://github.com/ModelCloud/GPTQModel/pull/4658
New Contributors
- @tpurtell made their first contribution in #3579
- @Prajwal7-2002 made their first contribution in #4029
Full Changelog: v7.10.0...v7.11.0