GPT-QModel v7.6.0

What's Changed

• fix(sglang): harden runtime correctness and test coverage by @ZX-ModelCloud in #3085
• Fix Qwen3.5-MoE multimodal calibration routing by @ZX-ModelCloud in #3086
• auto-decode native FP8/NVFP4 sources for W4A16 by @Qubitium in #3084
• test: expand native floatx W4A16 GSM8K coverage by @Qubitium in #3087
• docs: sharpen GPT-QModel quantization platform overview by @Qubitium in #3088
• [MODEL] support agnes quantization by @ZX-ModelCloud in #3089
• perf: optimize AWQ Triton scheduling by @ZX-ModelCloud in #3091
• [MODEL] Add zdtaichu5_0 quantization support by @ZX-ModelCloud in #3090
• feat: add Xing4.0-29B-A4B quantization support by @ZX-ModelCloud in #3093
• Add caller-owned Marlin scratch buffers and bounded eager reuse by @ZX-ModelCloud in #3092
• [MODEL] add qwen4_exp_text quantization support by @ZX-ModelCloud in #3094
• fix: materialize fused DeepSeek-V4 expert checkpoints by @ZX-ModelCloud in #3097
• [MODEL] support olmoe by @ZX-ModelCloud in #3098
• [MODEL] support qwen3_vl_moe by @ZX-ModelCloud in #3099
• [MODEL] support exaone4_5 by @ZX-ModelCloud in #3100
• Speed up eager GPTQ block updates by @Qubitium in #3102
• [MODEL] support intervl by @ZX-ModelCloud in #3103
• Fuse GPTQ block updates with native CUDA across group sizes by @Qubitium in #3104
• Enable CUDA GPTQ block updates for 2 through 8 bits by @Qubitium in #3105
• Reduce GPTQ Hessian inversion peak allocation by @Qubitium in #3106
• Release GPTQ Hessian before block updates to lower peak memory by @Qubitium in #3107
• Fix module tree isolation and dynamic config caching by @ZX-ModelCloud in #3101
• Include sample counts in quantization CSV rows by @peter-positron in #3116
• Preserve dynamic rule precedence in saved configs by @peter-positron in #3111
• Realign hooks after materialized module relocation by @peter-positron in #3114
• Restore offloaded tensors outside inference mode by @peter-positron in #3112
• Add native MLX GPTQ and AWQ quantization for Apple silicon by @Qubitium in #3117
• Preserve requested device and dtype after offload hook detach by @Qubitium in #3119
• Preserve absent rotary config for GLM NoPE models by @peter-positron in #3115
• Preserve complete positional inputs in Paro replay hooks by @peter-positron in #3113
• Add automatic OCP MXFP4/MXFP8 weight decoding by @Qubitium in #3109
• Accelerate GPTQ and AWQ inference with MLX on Apple Silicon by @Qubitium in #3118
• Support GPTQ for MiMo V2.6 Flash-RL and Pro-RL checkpoints by @peter-positron in #3120
• Add native MLX GGUF Q4_0 packing kernel by @Qubitium in #3121
• Add native MLX GGUF Q8_0 packing kernel by @Qubitium in #3122
• Add native MLX GGUF Q1_0 packing kernel by @Qubitium in #3123
• Add native MLX GGUF Q2_0 packing kernel by @Qubitium in #3124
• Add native MLX GGUF Q4_K packing kernel by @Qubitium in #3125
• Add native MLX GGUF Q5_K packing kernel by @Qubitium in #3126
• Add native MLX GGUF Q6_K packing kernel by @Qubitium in #3127
• Add native MLX GGUF TQ1_0 packing kernel by @Qubitium in #3128
• Add native MLX GGUF TQ2_0 packing kernel by @Qubitium in #3130
• Support every GPTQ/AWQ source group in packed MLX inference by @Qubitium in #3129
• Add native MLX GGUF MXFP4 packing kernel by @Qubitium in #3131
• Fix native MLX GPTQ affine rounding at BF16 ties by @Qubitium in #3133
• Add native MLX ParoQuant export packer by @Qubitium in #3132
• Add MLX inference for ParoQuant and supported quantization formats by @Qubitium in #3134
• Add native MLX FOEM quantization group kernel by @Qubitium in #3135
• Optimize QQQ MLX inference with fused activation quantization and packed 4-bit weights by @Qubitium in #3136
• Add native MLX FP8 weight quantization kernel by @Qubitium in #3137
• Fuse ParoQuant MLX rotation stages in Metal by @Qubitium in #3138
• Add native MLX bitsandbytes weight quantization kernels by @Qubitium in #3139
• Add native MLX RTN weight quantization kernel by @Qubitium in #3141
• Add native MLX GPTAQ correction and quantization kernel by @Qubitium in #3143
• Use native MLX MXFP8 for exact FP8 inference by @Qubitium in #3140
• Preserve ParoQuant MLX activation output dtype by @Qubitium in #3144
• Preserve FP16/BF16 outputs in MLX group-16 kernel by @Qubitium in #3145
• Add native MLX QQQ quantization kernel by @Qubitium in #3146
• Require FP16 and BF16 accuracy checks for MLX kernels by @Qubitium in #3147
• Add native MLX ParoQuant pseudo-quantization kernel by @Qubitium in #3148
• Preserve FP16/BF16 outputs in MLX GPTQ inference by @Qubitium in #3149
• Add native MLX ParoQuant rotation kernel by @Qubitium in #3152
• Add native MLX EXL3 trellis packing kernel by @Qubitium in #3154
• Preserve FP16/BF16 outputs in MLX AWQ inference by @Qubitium in #3150
• Preserve FP16/BF16 outputs in MLX GGUF inference by @Qubitium in #3151
• Preserve FP16/BF16 outputs in MLX BitsAndBytes inference by @Qubitium in #3153
• Add native MLX EXL3 codebook reconstruction kernel by @Qubitium in #3155
• Preserve FP16/BF16 outputs in MLX EXL3 inference by @Qubitium in #3156
• Add native MLX EXL3 Viterbi quantization kernel by @Qubitium in #3157
• Add native MLX EXL3 Hadamard kernel by @Qubitium in #3158
• Add native MLX EXL3 tile-layout kernel by @Qubitium in #3159
• Add packed MLX BitsAndBytes inference kernels by @Qubitium in #3160
• Add native MLX EXL3 block RMS kernel by @Qubitium in #3162
• Fuse MLX ParoQuant runtime rotations by @Qubitium in #3161
• Add fused MLX EXL3 regularization kernel by @Qubitium in #3163
• Add native MLX GGUF Q6_K decode kernel by @Qubitium in #3164
• Add native MLX EXL3 scale-search sampler by @Qubitium in #3165
• Add native MLX EXL3 global scale search by @Qubitium in #3166
• Add native MLX GPTQ group-16 decode kernel by @Qubitium in #3167
• Add native MLX EXL3 fallback quantization by @Qubitium in #3168
• Add native MLX AWQ group-16 decode kernel by @Qubitium in #3169
• Remove extraneous MLX API attribution headers by @Qubitium in #3170
• Parallelize native MLX RTN quantization by @Qubitium in #3171
• Add native MLX EXL3 LDLQ quantization by @Qubitium in #3172
• Add native MLX EXL3 block-LDL factorization by @Qubitium in #3174
• Parallelize MLX GGUF Q6_K packing by @Qubitium in #3173
• Parallelize MLX GGUF Q5_K packing by @Qubitium in #3176
• Parallelize MLX GGUF Q4_K packing by @Qubitium in #3178
• Add native MLX EXL3 Hessian finalization by @Qubitium in #3177
• Sync GGUF Q8_1 block size with llama.cpp by @Qubitium in #3179
• Speed up MLX GGUF TQ2_0 packing by @Qubitium in #3180
• Compose native MLX EXL3 regularization transforms by @Qubitium in #3181
• Speed up MLX GGUF TQ1_0 packing by @Qubitium in #3182
• Speed up MLX GGUF Q8_0 packing by @Qubitium in #3183
• Optimize MLX GGUF Q4_0 packing by @Qubitium in #3184
• Avoid FP32 temporaries in MLX GGUF MXFP4 packing by @Qubitium in #3185
• Avoid FP32 temporaries in MLX GGUF Q1_0 packing by @Qubitium in #3186
• Avoid FP32 temporaries in MLX GGUF Q2_0 packing by @Qubitium in #3187
• Avoid FP32 temporaries in MLX GGUF Q4_K packing by @Qubitium in #3188
• Avoid FP32 temporaries in MLX GGUF Q5_K packing by @Qubitium in #3189
• Avoid FP32 temporaries in MLX GGUF Q6_K packing by @Qubitium in #3190
• Avoid FP32 temporaries in MLX GGUF TQ1_0 packing by @Qubitium in #3191
• Avoid FP32 temporaries in MLX GGUF TQ2_0 packing by @Qubitium in #3192
• Fuse MLX FOEM parameter selection by @Qubitium in #3193
• Avoid FP32 temporaries in MLX RTN quantization by @Qubitium in #3194
• Avoid FP32 temporaries in MLX ParoQuant pseudo-quantization by @Qubitium in #3195
• Vectorize MLX ParoQuant rotation metadata by @Qubitium in #3196
• Update PR review and kernel validation checklist by @Qubitium in #3197
• Avoid FP32 temporaries in MLX BitsAndBytes quantization by @Qubitium in #3198
• Fuse MLX GPTAQ parameter selection by @Qubitium in #3199
• Avoid FP32 temporaries in MLX FP8 quantization by @Qubitium in #3200
• Accelerate MLX BitsAndBytes subnormal quantization by @Qubitium in #3202
• Fuse MLX QQQ dynamic parameter selection by @Qubitium in #3201
• Speed up MLX RTN target-dtype output by @Qubitium in #3203
• Speed up MLX ParoQuant target-dtype output by @Qubitium in #3204
• Avoid uint32 code copies in MLX ParoQuant packing by @Qubitium in #3205
• Encode MLX FP8 weights directly from float bits by @Qubitium in #3207
• Fuse MLX GPTQ affine parameter selection by @Qubitium in #3206
• Reduce MLX MXFP4 codebook search by @Qubitium in #3208
• Optimize MLX AWQ group-16 output dtype by @Qubitium in #3209
• Accelerate MLX EXL3 regularization transform by @Qubitium in #3211
• Tune MLX BitsAndBytes decode threads by @Qubitium in #3212
• Store MLX GPTQ group-16 outputs directly by @Qubitium in #3213
• Accelerate MLX EXL3 Hadamard stages by @Qubitium in #3214
• Grow dynamic override cache for large MoE models by @Qubitium in #3210
• Decode four BitsAndBytes 4-bit values per MLX lane by @Qubitium in #3216
• Store MLX GGUF Q6_K outputs directly by @Qubitium in #3215
• Accelerate MLX GGUF Q6_K small BF16 prefill by @Qubitium in #3217
• Accelerate MLX AWQ group-16 small BF16 prefill by @Qubitium in #3218
• Accelerate MLX GPTQ group-16 small BF16 prefill by @Qubitium in #3219
• Store small MLX BitsAndBytes outputs directly by @Qubitium in #3220
• Optimize MLX BitsAndBytes mid-prefill tiling by @Qubitium in #3222
• Tune wide MLX QQQ activation quantization by @Qubitium in #3223
• Add native MLX GGUF NVFP4 quantization by @Qubitium in #3224
• Add native packed EXL3 inference kernels for MLX by @Qubitium in #3225
• Speed up BitsAndBytes 4-bit processor packing with MLX by @Qubitium in #3226
• Add packed FP8 inference kernels for MLX by @Qubitium in #3227
• Add full-bit AWQ inference for MLX by @Qubitium in #3230
• Add MLX inference for GPTQ Marlin and BitBLAS formats by @Qubitium in #3229
• Fuse wide EXL3 decode output on MLX by @Qubitium in #3232
• Speed up QQQ processor quantization with MLX by @Qubitium in #3231
• Fuse QQQ decode output finalization on MLX by @Qubitium in #3233
• Add native MLX EXL3 quantization pipeline by @Qubitium in #3234
• Speed up ParoQuant optimization with MLX by @Qubitium in #3235
• Speed up BitsAndBytes INT8 packing with MLX by @Qubitium in #3237
• Speed up GGUF processor quantization with MLX by @Qubitium in #3238
• Add MLX inference for AWQ Marlin and BitBLAS formats by @Qubitium in #3236
• Speed up FP8 processor quantization with MLX by @Qubitium in #3239
• Support 2–8-bit AWQ BitBLAS inference on MLX by @Qubitium in #3240
• Speed up RTN processor quantization with MLX by @Qubitium in #3241
• Speed up FOEM processor quantization with MLX by @Qubitium in #3242
• Widen packed FP8 prefill tiles on MLX by @Qubitium in #3243
• fix(gemma4): stop unused full-length KV store from OOMing calibration replay by @okdshin in #3245
• Optimize GPTQ Hessian collection and same-device factor reuse by @ZX-ModelCloud in #3244
• [MODEL] support granitemoe by @ZX-ModelCloud in #3246
• [MODEL] support smolvlm by @ZX-ModelCloud in #3247
• Speed up GPTQ Cholesky damping recovery with an exact grid search by @ZX-ModelCloud in #3248
• Add GB10 GPTQ W4A activation stream: FP8 (A8) and NVFP4 (A4) over native INT4 weights by @Qubitium in #3228
• Fix GPTQ damp recovery abort when the inverse step fails by @ZX-ModelCloud in #3249
• [MODEL] support jamba by @ZX-ModelCloud in #3250
• Expose stable hf_ kernel/packing/repacking APIs (#726) by @ZX-ModelCloud in #3251
• Declare requests as a runtime dependency by @li12345678990 in #3252
• Bump version from 7.5.0 to 7.6.0 by @Qubitium in #3256
• docs: consolidate 7.6.0 release notes and extract Bonsai guide by @Qubitium in #3257

New Contributors

• @li12345678990 made their first contribution in #3252

Full Changelog: v7.5.0...v7.6.0