Skip to content

GPT-QModel v7.11.4

Latest

Choose a tag to compare

@Qubitium Qubitium released this 18 Nov 15:02
· 7 commits to main since this release
c1807f8

GPT-QModel 7.11.4

  • Quantization: SmoothMSE keeps the unclipped range when clipping would increase error, and QQQ's GSQ calibration now uses the same FP32 activation scales as inference. AWQ samples routed-expert activations uniformly across the calibration set instead of favoring later batches. Paro retains mixed-shape batches, and SenseNova-Loop calibrates on every pass through its repeated layers.
  • Quantization memory: AWQ and Paro limit how many activation rows they retain. AWQ reuses its expert sampling buffers and scales experts in place across GPUs, while EXL3 spreads expert Hessians across the configured GPU pool. ROCm allocator defaults reduce peak reserved VRAM for wide MACaQ layers.
  • Inference: Marlin speeds up large-batch prefill and fixes GPTQ/AWQ output errors under GPU contention and MXFP8/MXFP4 execution. Portable QQQ and VPTQ Torch kernels now support compiled generation. Fixed crashes when loading another model during compiled inference and improved EXL3 BF16 execution and Paro dispatch for long prefills.
  • Compat: Fixed Transformers device_map="auto" when GPTQ, RTN, or AWQ layers spill to CPU or disk. Checkpoint loads preserve required FP32 tensors, and re-saves preserve quantized embeddings and output heads. Marlin now requires NVIDIA Ampere or newer.
Full list of changes

What's Changed

Full Changelog: v7.11.2...v7.11.4