QQQ

QQQ

Documentation home · Quickstart · Lifecycle

On this page

• Overview
• Workflow at a glance
• Concepts
• Usage and configuration
• Rotation and QQQ
• Implementation internals
• GSQ refinement
• Pitfalls and edge cases
• Source files
• See also

Overview

QQQ is a calibration-based, 4-bit weight quantization method with dynamically quantized 8-bit activations at inference. The project cites QQQ: Quality Quattuor-Bit Quantization for Large Language Models (paper, reference implementation). GPT-QModel supplies its own quantizer, looper processor, packed checkpoint format, native GPU runtime, portable Torch runtime, and Apple-silicon paths. Native device declarations and actual extension requirements are distinguished in the backend reference below.

QQQ's quality optimization uses activation statistics and a quadratic, Hessian-weighted error model. It is not simply a different name for ordinary GPTQ INT4 storage: the packed weights, scale hierarchy, and INT8 activation arithmetic have their own contract.

Rotation is not automatically provided by this implementation. Although rotation-based preprocessing is relevant to low-bit quantization, the QQQ quantizer and QQQ checkpoint do not implement a QQQ-specific rotation transform. The shared rotation configuration is inherited, but the loader currently accepts rotated checkpoints only for GPTQ/GPTQ_V2 and selected GPTQ backends. Leave rotation=None for QQQ. See Rotation and QQQ before applying a rotation recipe.

Identifier	Value	Meaning
METHOD.QQQ	"qqq"	Calibration/quantization algorithm
FORMAT.QQQ	"qqq"	QQQ packed weight and scale checkpoint layout
BACKEND.QQQ	"qqq"	Native INT8-by-INT4 runtime
BACKEND.QQQ_TORCH	"qqq_torch"	Portable runtime consuming the same layout
BACKEND.MLX	"mlx"	Packed-weight transfer and MLX inference, subject to MLX compatibility checks

For the surrounding model lifecycle, see Quantization Lifecycle. For ordinary GPTQ checkpoints, see GPTQ.

Workflow at a glance

Reading the diagram: Independent preparation and execution concerns; neither uses model rotation. QQQ_TORCH and MLX have separate execution paths.

Text version of the workflow

• Weight preparation: Collect Gram statistics → Damp and factor → Quantize 4-bit weights → Pack QQQ state.
• Native QQQ forward: Use FP16 working input → Quantize rows to INT8 → Run packed W4A8 GEMM → Restore output contract.

Open full-size SVG

Concepts

Quadratic error compensation

For a linear layer, calibration forwards collect input activations. QQQ.add_batch() accumulates an input Gram matrix on each input device and counts flattened input tokens. materialize_hessian() merges those partials in FP32 and scales their sum by 2 / total_samples.

The quantization pass:

1. Copies the weight to an FP32 working matrix, transposing Transformers Conv1D weights and flattening Conv2d weights when necessary.
2. Neutralizes dead input columns in the Hessian/weight pair.
3. Adds damp_percent * mean(diag(H)) to the diagonal.
4. Computes an upper Cholesky factor of the inverse damped Hessian.
5. Quantizes successive columns and propagates their scaled errors into the remaining columns, processing 128-column blocks by default.

The quadratic objective accounts for which weight errors matter to the layer's observed inputs. Calibration coverage therefore matters; it is not interchangeable with weight-only round-to-nearest quantization. Some compatible CUDA cases use the shared fused GPTQ block-update kernel instead of issuing a small eager operation for each column. That optimization does not change the QQQ checkpoint format.

Whole-row and grouped weight quantization

group_size refers to contiguous input columns within an output channel, not the calibration batch size.

Setting	Quantization representation	Packed runtime interpretation
group_size=-1	One symmetric quantizer per output row; signed 4-bit codes	Signed nibbles are represented in the runtime's INT8 domain with a factor of 16; s_channel supplies the output-channel scale and s_group is empty
group_size=128	Contiguous groups of 128 input columns; grouped codes use the centered unsigned 4-bit representation	s_group supplies the group-to-channel scale ratio; grouped values are rounded/clamped into the signed INT8 domain and s_channel supplies the channel scale

The scalar quantization helper distinguishes symmetric whole-row quantization from grouped quantization. sym=True remains the supported runtime contract in both cases; grouped unsigned storage does not mean sym=False checkpoints are supported.

Grouped quantization also computes an additional whole-row 8-bit scale (scale_extra in the quantizer result, s_extra at packing). This is necessary for the QQQ W4A8 runtime. Do not drop it when implementing a packer or calling pack() directly. Internally, group_size=-1 becomes K in the linear module, and the module treats any effective group_size == K as whole-row storage (empty s_group). In particular, K=128, group_size=128 takes that storage branch; it is not a distinct one-group grouped layout. The quantizer likewise omits the extra scale when its resolved group size equals K.

Dynamic activation quantization

For each flattened input row/token, the runtime computes an absolute maximum, divides by 127 to obtain an activation scale, rounds scaled activations, clamps to [-128, 127], and converts to torch.int8. Matrix multiplication is followed by activation and output-channel rescaling and optional bias/adapter application.

This is runtime activation quantization. There is no static activation-scale calibration artifact to supply separately. Also, it does not turn FORMAT.QQQ into a generic GPTQ checkpoint: the QQQ packed representation must still be interpreted correctly.

In QQQLinear.dynamic_quant(), the maximum and division by 127 happen in the input dtype before the scale is promoted to FP32. Native inference first casts activations to FP16; the Torch backend does not make that cast. The helper has no explicit zero-scale guard, so an all-zero row or a scale that underflows is not protected by an epsilon in this Python implementation. Do not assume identical BF16 arithmetic or zero-row numerics across all runtimes.

Usage and configuration

Quantize and save a model

The example follows the repository's QQQ recipe. The model must be accessible to your Hugging Face account; substitute an accessible supported model if necessary. Calibration strings are tokenized through the model's tokenizer.

import torch
from datasets import load_dataset
from gptqmodel import GPTQModel, QQQConfig

model_id = "meta-llama/Llama-3.2-1B-Instruct"
output_path = "Llama-3.2-1B-Instruct-QQQ-4bit"

calibration = load_dataset(
    "allenai/c4",
    data_files="en/c4-train.00001-of-01024.json.gz",
    split="train",
).select(range(256))["text"]

qcfg = QQQConfig(
    bits=4,
    group_size=128,
    desc_act=False,
    mse=2.4,
)
model = GPTQModel.load(model_id, qcfg, dtype=torch.float16)
model.quantize(calibration, batch_size=1, calibration_concat_size=2048)
model.save(output_path)

mse=2.4 is the documented quality recipe, not the constructor default. For whole-row quantization, change group_size to -1. Both layouts require symmetric weights and disabled activation ordering.

Load for inference

Select a QQQ-aware backend, not a GPTQ backend:

import torch
from gptqmodel import GPTQModel
from gptqmodel.utils.backend import BACKEND

# Linux, supported CUDA GPU and native QQQ extension:
model = GPTQModel.load(
    "Llama-3.2-1B-Instruct-QQQ-4bit",
    device="cuda:0",
    backend=BACKEND.QQQ,
    dtype=torch.float16,
)

from gptqmodel import GPTQModel
from gptqmodel.utils.backend import BACKEND

# Portable implementation, including CPU inference:
model = GPTQModel.load(
    "Llama-3.2-1B-Instruct-QQQ-4bit",
    device="cpu",
    backend=BACKEND.QQQ_TORCH,
)

When deriving the preferred backend, the model lifecycle chooses QQQ_TORCH for CPU/NPU and QQQ otherwise. The processor's packing selector additionally chooses QQQTorchLinear unconditionally on Darwin. Backend selection still validates devices, shapes, dtypes, and extension availability. Explicit selection is useful when diagnosing a deployment; a portable backend does not waive QQQ's packed-shape restrictions.

See Loading and Inference for generation, tokenizer, and placement APIs.

QQQConfig reference

QQQConfig is a dataclass derived from GPTQConfig. Its own field declarations and method defaults are:

@dataclass
class QQQConfig(GPTQConfig):
    method: METHOD = field(default=METHOD.QQQ)
    format: FORMAT = field(default=FORMAT.QQQ)

The following table distinguishes normalized defaults from runtime restrictions. Inherited constructor fields are not evidence that every GPTQ feature is implemented for QQQ.

Field	Default after normalization	QQQ behavior/constraints
method	METHOD.QQQ	Only QQQ is allowed by this config class
format	FORMAT.QQQ	QQQ's supported export format
bits	4	Both QQQ kernels support only 4-bit weights
group_size	128	Runtime supports only 128 and -1
sym	True	Both QQQ kernels require True
desc_act	False	True raises in QQQConfig, including dynamic overrides; the quantizer and runtime also reject it
act_group_aware	False	Inherited GPTQ-family option; the QQQ quantizer does not implement GPTQ's group-aware ordering
damp_percent	0.005	Applied relative to the mean Hessian diagonal; validated as strictly between 0 and 1
damp_auto_increment	0.001	Inherited and validated as nonnegative, but QQQ.quantize() does not implement an automatic damping-increment retry loop
static_groups	False	Precomputes group parameters for positive group sizes; excludes the MLX quantization fast path. Do not combine with group_size=-1: the implementation iterates range(0, columns, groupsize) without resolving -1 first
mse	MSEConfig(norm=0.0, grid_budget=None, clip_grid_budget=None)	A positive norm enables scale shrink-grid search; a number such as 2.4 is normalized to that norm
dynamic	None	Processor handles layer exclusion and inherited override lookup; desc_act=True is prohibited. Keep overridden widths/group sizes within the QQQ runtime contract
pack_dtype	torch.int32	Only supported packed dtype
pack_impl	"auto"	Shared packing hint; QQQ uses its own packer/layout rather than generic GPTQ qweight/qzeros
true_sequential	True	Shared sequential quantization lifecycle
lm_head	False	Output head is not quantized unless requested and supported by the model definition
rotation	None	Keep unset for supported QQQ loading; see below
fallback	Fallback(strategy=FallbackStrategy.RTN, threshold="0.5%", smooth=None)	Low-coverage fallback policy; QQQ consults the threshold when inverse-Hessian factorization fails
device	None	Resolved by the model/device-placement lifecycle
offload_to_disk	True	Shared quantization offload behavior

MSEConfig.grid_budget controls candidate-chunk workspace in the shared shrink-grid search, not total quantization memory. It accepts bytes or a size string such as "256M"; None chooses the device-derived budget and 0 requests single-candidate chunking. clip_grid_budget is for AWQ clipping, not a QQQ inference option. For example:

from gptqmodel import QQQConfig
from gptqmodel.quantization import MSEConfig

qcfg = QQQConfig(
    group_size=128,
    mse=MSEConfig(norm=2.4, grid_budget="256M"),
)

The remaining shared placement, offload, adapter, telemetry, and Hessian configuration fields are documented in QuantizeConfig Reference. Their presence on the base config must not be read as a promise that QQQ consumes GPTQ-specific options such as GPTAQ or FOEM.

Rotation and QQQ

Rotation, activation ordering, and packed-layout permutations are three different operations:

Operation	Implementation	QQQ status
Orthogonal model rotation	Shared fuse_layer_norms() / rotate_model() in the model quantization lifecycle; rotation choices "hadamard" and "random"	Not a supported QQQ checkpoint-loading path: loader requires GPTQ/GPTQ_V2 and selected GPTQ backends
GPTQ activation ordering	Reordering input columns by Hessian diagonal, normally represented with group-index metadata	Explicitly rejected: QQQ's runtime has contiguous group scales and no stored g_idx mapping
QQQ packing permutation	Nibble interleave and scale permutations in QQQLinear.pack()	Required representation transform, not an orthogonal/model rotation

gptqmodel/quantization/qqq.py does not call rotate_model() or store rotation matrices. The shared model quantization code can enter rotation setup when the inherited field is set, but that is not an end-to-end QQQ compatibility guarantee: quantization then restricts the backend to GPTQ_TORCH, GPTQ_TRITON, GPTQ_W4AFP8, or GPTQ_W4A_NVFP4, and loading also requires FORMAT.GPTQ or FORMAT.GPTQ_V2. These restrictions exclude the normal QQQ backends and format.

Consequently, do not advertise QQQConfig(rotation="hadamard") as a supported round-trip recipe, and do not confuse the packer's permutations with automatic outlier-removing rotation. If experimenting with externally rotated models, rotation-aware model/runtime reconstruction is your responsibility; saving QQQ weights alone does not establish that compatibility. See Rotation, Smoothing, and Norm Tuning.

Implementation internals

Quantizer API and results

The principal source signatures are:

# gptqmodel/quantization/qqq.py — QQQ methods
def __init__(self, module: nn.Module, qcfg: Optional[QuantizeConfig] = None):
def add_batch(self, inp, out):
def materialize_hessian(self, target_device: Optional[torch.device] = None) -> torch.Tensor:
def quantize(
        self,
        blocksize=128,
):

These are method signatures, not a standalone script. Quantizer in the same file implements parameter search, MSE search, and pseudo-quantization; QQQ owns calibration statistics and the sequential error-compensation pass.

quantize() returns the tuple consumed by QQQProcessor.process():

(wq, q_scales, q_zeros, q_g_idx, duration, avg_loss,
 damp, q_scales_extra, nsamples)

wq is a floating pseudo-quantized weight for replay and packing. The remaining tensors are quantization metadata, not an already packed inference layer. Although the quantizer returns zeros/group indices through the common interface, the final QQQ runtime stores B, s_channel, and s_group, not a generic GPTQ g_idx buffer.

The seventh result is the actual diagonal damping value (damp_percent * mean(diag(H))), although the processor names its local variable damp_percent. The dedicated fallback routine returns 0.0 in that slot. Normal-path numeric loss is the summed Hessian-scaled column error divided by the captured token count; fallback paths may return a descriptive string instead.

When factorization fails, the quantizer checks the configured coverage threshold. If fallback is permitted, non-RTN strategies return through _fallback_quantize(); if the threshold does not permit fallback, the factorization error is re-raised. The RTN branch is incomplete in this revision: it sets Hinv=None, but the subsequent eager block loop still indexes Hinv. It must not be presented as a working Hessian-free fallback. Likewise, the NaN-loss branch sets mock_quantization=True and recurses, but this implementation does not provide a separate mock-quantization algorithm. These paths are not substitutes for representative calibration. The inherited damping-increment field causes no automatic retries. Inspect the per-module logs, especially for sparsely routed MoE experts.

QQQProcessor lifecycle

Stage	Work performed
Preprocess	Selects the QQQ linear class, clones config for the module, handles dynamic exclusions/override lookup, creates a QQQ task, resolves fallback/expected calibration tokens, configures the scalar quantizer
Calibration hook	Passes captured layer inputs/output to add_batch(); per-device partials avoid repeatedly moving all activations to one device
Process	Calls quantize(), frees task statistics, moves scales/zeros/group indices and extra scales to CPU, records logs/results, and installs the pseudo-quantized weight for subsequent calibration replay
Submodule finalization	Creates the actual QQQ quantized module, packs replay weights and scales, replaces the original layer and releases its full-precision weight
Finalization	Marks the model quantized and sets METHOD.QQQ; FORMAT.QQQ comes from the config. Then performs shared cleanup

The processor configures its Quantizer using the processor-wide bits, symmetry, MSE norm, and group size even though a task config is cloned for inherited dynamic handling. Do not assume arbitrary mixed per-layer quantization recipes are equivalent to a globally configured QQQ run. In particular, unsupported bit widths or group sizes are not made valid by placing them inside dynamic.

See Looper Processors for processor ordering, replay, and adapter interactions.

FORMAT.QQQ storage and packing

For a layer with input width K and output width N:

Tensor	Shape	Dtype	Meaning
B	(K // 16, N * 2)	torch.int32	Permuted 4-bit weights; eight nibbles per word
s_channel	(1, N)	torch.float32	Per-output-channel scale
s_group	(K // group_size, N) when the effective group size differs from K; (0,) otherwise	torch.float16	Group-scale ratios in QQQ order
bias	(N,) when present	Initialized as torch.float16	Linear bias
workspace	N // 128 * 16 elements	torch.int32	Non-persistent native-kernel scratch
reduce_buffer	(max_par * 64, N), max_par=16	torch.int	Non-persistent accumulation scratch

post_init() normalizes channel scales to FP32 and group scales to FP16. pack() applies the QQQ weight/nibble and scale permutations and packs bounded chunks rather than retaining an expanded inference weight matrix. Its signature is:

def pack(self, linear: torch.nn.Module, scales: torch.Tensor, s_extra=None):

Grouped packing requires s_extra; whole-row packing does not. Both runtimes consume the same saved buffers. QQQ configuration serialization retains bits and also writes wbits, which the vLLM QQQ config loader expects. That compatibility field is not a promise that every serving engine or model architecture supports every QQQ checkpoint. See Packing Formats and Compatibility and Serving.

Kernel/backend reference

Contract	QQQLinear / BACKEND.QQQ	QQQTorchLinear / BACKEND.QQQ_TORCH
Method/format	QQQ / QQQ	QQQ / QQQ
Bits/group sizes	4; -1, 128	Same
Symmetry/ordering	sym=True, desc_act=False	Same
Declared devices	CUDA, ROCm	All Torch device types, subject to available operations
Declared platforms	Linux	All
Activations/output	FP16 or BF16 input/output; native computation casts input to FP16 and converts output back	FP16 or BF16; final output follows input dtype
Native dependency	QQQ torch.ops extension	Portable operations; optional packed CPU extension
Padding	No automatic padding	No automatic padding
Packing	torch.int32	torch.int32
Training	Not supported	Not supported
Adapter declaration	Lora	Lora

Both classes declare input/output widths divisible by 64 and use the inherited QQQ pairwise tile validation: at least one of (64, 256), (128, 128), (128, 64), (64, 128) must divide (K, N). A width pair that is merely divisible by 64 may therefore still be rejected. Grouped packing must also fit contiguous 128-column groups.

For NVIDIA devices, QQQLinear.validate_device() requires compute capability at least 8.0. QQQTorchLinear inherits that method too; portable CUDA selection does not bypass the capability check. ROCm bypasses this NVIDIA check, but passing device validation is not proof that the native QQQ kernel can build or execute there. Its source contains CUDA-specific headers and inline PTX, including cp.async. Treat native ROCm support as unverified rather than inferring it from SUPPORTS_DEVICES; QQQ_TORCH avoids the QQQ GPU extension, subject to its own operation and shape checks.

The Torch forward path tries these routes in order:

1. A packed INT4 CPU kernel when CPU activations/weights, AVX512-BF16 detection, extension availability, and input-width limits permit it. The width guard excludes K > 131071.
2. Temporary INT8 weight reconstruction and CPU torch._int_mm() when that operation supports the device and shape. Failed shapes are remembered, not a blanket permanent disable for every shape.
3. FP32 torch.matmul() over temporarily reconstructed quantized values.

Only a packed INT4 CPU representation and unpermuted scale metadata are cached. Expanded INT8 or dense floating weights are not retained between forward calls. A portable path is not necessarily as fast as the native W4A8 kernel.

Native extension: utils/qqq.py

TorchOpsJitExtension registers extension gptqmodel_qqq_ops, namespace gptqmodel_qqq, required operation qqq_gemm, and the sources gptqmodel_ext/qqq/qqq.cpp and qqq_gemm.cu. Its CUDA include resolver explicitly checks for cusparse.h. requires_cuda=True checks Torch's CUDA-facing availability API, not the GPU vendor; the shared loader's ROCm handling does not by itself establish portability of these native kernels.

def clear_qqq_extension_cache() -> None:
def qqq_runtime_available() -> bool:
def qqq_runtime_error() -> str:
def prewarm_qqq_extension() -> bool:
def qqq_gemm(
    A,
    B,
    C,
    D,
    s1,
    s2,
    s3,
    workspace,
    thread_k=-1,
    thread_n=-1,
    sms=-1,
    max_par=16,
):

Control	Meaning
GPTQMODEL_QQQ_BUILD_ROOT	Override native extension build directory
GPTQMODEL_QQQ_FORCE_REBUILD	Extension force-rebuild control
GPTQMODEL_EXT_VERBOSE	Verbose extension build output

Availability checks and prewarming go through the central extension registry. prewarm_qqq_extension() returns whether loading succeeded; qqq_runtime_error() is empty when available and otherwise reports the registry error or a fallback diagnostic. The native wrapper raises if the required operation is unavailable. A supported GPU plus a runtime-only PyTorch installation is not sufficient to guarantee JIT compilation: a working compiler/toolkit, headers, Ninja, and compatible device build targets are required.

The source dependency baseline is Python >=3.10 and PyTorch >=2.14.1; consult the checked-out pyproject.toml and requirements.txt rather than using an older QQQ package's installation requirements.

Apple silicon: two distinct mlx_qqq components

Calibration-time acceleration lives in gptqmodel/quantization/mlx_qqq.py. The internal selector requires all of the following:

• Darwin on arm64 with MLX importable, and debugging disabled.
• CPU or MPS working weights, with the inverse-Hessian factor on the same device.
• At least 4_194_304 weight elements and blocksize=128.
• Group size -1 or 128; grouped input width divisible by 128.
• Symmetric quantization, no MSE scale search, no static groups, and matching scalar-quantizer group/code-range settings.

Thus the recommended mse=2.4 recipe deliberately does not use this MLX fast path.

The public helper signatures are:

def qqq_quantize_weight_mlx(weight, inverse_hessian, *, group_size=128):
def qqq_quantize_weight_mlx_with_loss(
    weight, inverse_hessian, *, group_size=128
):

They accept nonempty rank-2 MLX weights in FP16/BF16/FP32 and an FP32 inverse-Hessian factor matching the input width. They return pseudo-quantized weights, scales, zeros, extra scales, and optionally aggregate loss. Inputs are already ordered; neither activation ordering nor MSE search is performed inside the helper. The result is not a packed QQQ inference module.

Inference transfer uses QQQMlxQuantLinear in nn_modules/qlinear/mlx.py and MlxQQQLinear in nn_modules/qlinear/mlx_qqq.py. The holder declares Darwin/MPS, 4-bit QQQ source weights, group sizes -1/128, FP16 source compatibility, no adapters/training, and 64-divisible dimensions, then checks the exact source buffer layout.

Whole-row signed nibbles transfer to MLX affine 4-bit storage without requantizing their values. Grouped QQQ transfers its rounded effective INT8 values into affine 8-bit storage; that transfer preserves QQQ arithmetic but is not a 4-bit-memory claim for the MLX runtime. MlxQQQLinear dynamically quantizes activations per token, applies input/channel scales and FP16 rounding, adds bias, and restores the activation dtype; its finalize kernel handles FP16/BF16/FP32 outputs. Transfer fails rather than silently requantizing incompatible packed QQQ state.

See MLX on Apple Silicon for installation, model conversion, and runtime entry points.

GSQ refinement

QQQConfig.gsq refines QQQ's 4-bit packing against captured calibration moments. It requires FORMAT.QQQ export and fixed scales — learn_scales is rejected — and skips non-nn.Linear or tensor-parallel-padded modules. Staged gsq_training is unsupported for QQQ. See GSQ.

Pitfalls and edge cases

Symptom or misconception	Explanation/action
desc_act=True fails	Expected. QQQ cannot encode activation-order group mappings; use False, including dynamic rules
Arbitrary GPTQ bits/group sizes fail	QQQ runtime is 4-bit, symmetric, groups -1 or 128, with strict tile checks
Rotation fails during quantization or loading	QQQ backends are excluded by the shared rotation backend check, and the loader also excludes FORMAT.QQQ; leave the inherited field unset
Cholesky error or RTN fallback indexes None	Check module calibration coverage and damp_percent; fallback is threshold-dependent and the RTN failure branch is incomplete in this revision
Poor MoE expert quality	Review routed-token counts and fallback logs rather than treating a calibration dataset's total token count as every expert's sample count
Native kernel unavailable	Inspect qqq_runtime_error(), toolchain/headers, Linux/device capability, and build diagnostics; use QQQ_TORCH where appropriate
BF16 model produces FP16-like arithmetic/overflow behavior	Native QQQ computes via FP16 before returning the original dtype; BF16 output storage does not provide BF16 internal range
Unexpected CPU speed or memory use	The portable fallback may reconstruct INT8/FP32 values temporarily; only the packed AVX512 path avoids that whole fallback route
Apple-silicon run does not enter the MLX quantizer	Check the full selector, especially MSE, matrix size, static groups, and inverse-Hessian device
Generic GPTQ code expects qweight/qzeros/g_idx	FORMAT.QQQ has different buffer names, scales, and permutations; do not relabel it as GPTQ

Source files

The behavior above follows the implementation at c1976918cfe3. Relevant paths in the repository:

Path	Responsibility
gptqmodel/quantization/config.py	METHOD, FORMAT, QQQConfig, inherited config normalization and serialization
gptqmodel/quantization/qqq.py	Scalar Quantizer, QQQ, calibration, error compensation, fallback, acceleration dispatch
gptqmodel/quantization/gptq.py	Related GPTQ implementation; not the QQQ quantizer entry point
gptqmodel/quantization/mlx_qqq.py	MLX pseudo-quantization Metal helpers
gptqmodel/looper/qqq_processor.py	Calibration/replay/packing lifecycle
gptqmodel/looper/stage_subset.py	Calibration coverage policy and uncovered-module handling
gptqmodel/nn_modules/qlinear/qqq.py	QQQLinear, QQQTorchLinear, packing, dynamic activation quantization
gptqmodel/utils/qqq.py	Native extension configuration and operation wrappers
gptqmodel_ext/qqq/qqq.cpp, qqq_gemm.cu, qqq_gemm.h	Native QQQ operation and kernel
gptqmodel/extension.py	Central extension registration
gptqmodel/nn_modules/qlinear/mlx.py	QQQ packed-source compatibility and MLX transfer
gptqmodel/nn_modules/qlinear/mlx_qqq.py	MLX QQQ inference wrapper
gptqmodel/utils/mlx.py	Model-level packed-weight conversion
gptqmodel/models/base.py, gptqmodel/models/loader.py	Quantization dispatch, loading, shared rotation restrictions
gptqmodel/quantization/rotation/rotation.py	Shared model rotation implementation, not QQQ-local rotation
gptqmodel/utils/backend.py, gptqmodel/utils/importer.py	Backend names and kernel selection
README.md, pyproject.toml, requirements.txt	QQQ example, paper citation, installation baseline

See also

• What Is Quantization
• GPTQ
• VPTQ
• GSQ
• QuantizeConfig Reference
• Quantization Lifecycle
• Looper Processors
• Kernel Backends Reference
• Packing Formats and Compatibility
• Rotation, Smoothing, and Norm Tuning
• MLX on Apple Silicon
• Troubleshooting and FAQ