Skip to content

GPT-QModel v7.11.0

Latest

Choose a tag to compare

@Qubitium Qubitium released this 17 Nov 09:00
· 133 commits to main since this release
d0c751b

GPT-QModel 7.11.0 🚀

  • MACaQ: Native MXFP4 W4A4 quantization for low-precision weights and activations, with CUDA, ROCm, and Apple silicon inference. Includes packed checkpoints and a portable Torch fallback.
  • NF12: Lossless 12-bit compression of BF16 weights reduces source-weight storage during calibration without changing the restored values. Packed inference now includes native Metal support alongside CUDA, HIP, and Torch paths. MLX · MoE capture contract
  • Tenstorrent: Packed GPTQ inference through Torch/XLA and TTNN, multi-card execution, and block-float quantization. Long-context Llama serving adds paged caching and chunked prefill; compatible block-float checkpoints also run on CUDA, ROCm, and CPU. Guide · Serving · Device ownership
  • Quantization: Resumable distributed EXL3 jobs (projection publication recovery), broader mixed-bit allocation across AWQ/EXL3/GGUF, and SmoothQuant preprocessing improve large-model workflows and quality control. Quantizer calibration flows document the shared Hessian-failure and zero-sample fallback contracts. MoE routing override and bypass defines when each mode is honoured and when it fails closed. Shared-input Hessian dedup covers QQQ/VPTQ/MACaQ/EXL3 and GPTAQ/FOEM.
  • Inference: Opt-in attention and MLP projection fusion, faster quantized embedding lookups, and improved small-batch inference. Native backend selection and packed-weight storage are preserved. Fusion guide
  • Configuration: Typed backend configurations unify backend selection and tuning, with explicit numerical and worker controls instead of environment switches.
  • Calibration: Broader dataset input support, document-isolated AWQ replay, and opt-in attention-sink exclusion improve calibration coverage and control.
  • Memory: Reduced packed-weight duplication, bounded EXL3 scratch, and shorter-lived inference workspaces lower memory use.
  • Models: Added OLMo Hybrid, Granite SWA/MoE SWA, and Dots3 Note quantization, plus broader decoder-weight coverage and Transformers 5.22 compatibility. Model notes
  • Compat: Major fixes across dense/MoE calibration, multimodal replay, checkpoint round trips, Transformers model loading, multi-GPU memory use, and ROCm execution. Unsupported combinations fail clearly rather than loading incomplete weights. Full notes
  • Security: Runtime checks for vulnerable PyTorch versions, explicit checkpoint-code trust, stronger checkpoint validation, and bounded server/media requests. Runtime safety · Serving security
Full list of changes

What's Changed

New Contributors

Full Changelog: v7.10.0...v7.11.0