GPTQ

GPTQ, GPTAQ, and RTN

Documentation home · Quickstart · Lifecycle

On this page

• Overview
• Workflow at a glance
• Concepts
• Configuration reference
• Usage
• Implementation internals
• Looper integration
• Quantization protocol: algorithm versus representation
• Accuracy and low-precision activations
• GSQ refinement
• Pitfalls and edge cases
• Source files
• See also

Overview

GPT-QModel provides three related weight-quantization paths:

Path	Configuration	Calibration	What it optimizes
GPTQ	GPTQConfig(...)	Required	Layer-output reconstruction using input second-order statistics and compensation of rounding errors in the remaining weights
GPTAQ	GPTQConfig(gptaq=GPTAQConfig(...))	Required, including a native-input reference pass	GPTQ-style reconstruction plus a correction for the difference between native and replayed inputs
RTN	RTNConfig(...)	Not required	Direct round-to-nearest weight quantization, optionally with weight-range smoothing/search; no Hessian

These are quantization algorithms, not inference kernels. Quantization produces rounded weights and scale/zero/group metadata; packing then encodes them for a checkpoint and runtime backend. RTN can produce a GPTQ-format checkpoint without running the GPTQ algorithm. Similarly, GPTAQ still belongs to the METHOD.GPTQ family. RTNConfig also uses METHOD.GPTQ; there is no METHOD.RTN. Its uses_weight_only_lifecycle() selects the calibration-free path, while the export format determines the checkpoint's method family.

The name GPTAQ needs particular care:

• The activation-aware algorithm is implemented by GPTAQ in gptqmodel/quantization/gptaq.py. Select it with gptaq=GPTAQConfig(...).
• FORMAT.GPTQ_V2, whose string value is "gptq_v2", is a checkpoint zero-point convention. It is not a switch for the GPTAQ algorithm.
• There is no v2 field in the current GPTQConfig. Do not copy v2=True from a different API or an older example into a current configuration.

This page describes the implementation at source revision c1976918. The method-specific configuration API, protocol compiler, and W4A policies below should be read in that source context. See Installation and Quickstart for installation and QuantizeConfig Reference for the complete shared configuration surface.

Workflow at a glance

Reading the diagram: Alternative paths, not consecutive stages. GPTQ/GPTAQ repeat per layer and subset; GPTQ_V2 selects zero-point storage, not GPTAQ.

Text version of the workflow

• GPTQ: Capture calibration → Build inverse factor → Round and compensate → Pack weights.
• GPTAQ: Capture native inputs → Compare replayed inputs → Apply GPTAQ correction → Pack weights.
• RTN: no calibration: Read module weights → Fit group ranges → Round directly → Pack compatible layout.

Open full-size SVG

Concepts

From round-to-nearest to second-order reconstruction

For a linear layer, let the weight matrix have shape [out_features, in_features], and let calibration inputs be rows of a matrix X with shape [tokens, in_features]. The reconstruction objective compares the original and rounded layer outputs:

$$ \min_{\widehat W};\lVert XW^T-X\widehat W^T\rVert_F^2. $$

For an individual output row, the quadratic error is controlled by the input Gram matrix. GPT-QModel accumulates unnormalized X.T @ X contributions and materializes

$$ H=\frac{2}{N}\sum_b X_b^T X_b, $$

where N is the accumulated activation-row/token count after the module-specific input reshape. This is the Hessian of the layerwise quadratic reconstruction objective, not a full-model loss Hessian obtained by backpropagation.

The optimal-brain-quantization (OBQ) idea is to round one weight coordinate and adjust the remaining coordinates using inverse-Hessian information. GPTQ applies that idea across output rows together, in a shared column order, and batches the expensive updates into blocks. RTN instead rounds each group without compensating the other columns for the introduced error.

Practical consequences:

• Calibration inputs affect GPTQ's result even though only weights are stored at low precision.
• An unobserved input direction has no calibration evidence; the implementation must explicitly handle zero diagonal entries.
• A small layer reconstruction loss is not a downstream benchmark score.
• GPTQ alone is not activation quantization. Low-precision activation policies are separate, explicit options.

Cholesky inverse and the meaning of Hinv

GPTQ.hessian_inverse(H) first regularizes the diagonal and attempts Cholesky factorization. On the normal CPU/CUDA path, it computes a Cholesky factor of the regularized Hessian, applies torch.cholesky_inverse, and takes an upper Cholesky factor of that inverse.

Consequently, the variable named Hinv in the quantization loop is an upper-triangular inverse factor, not simply a dense inverse matrix. If the regularized Hessian is Hλ, the factor U satisfies:

$$ U^T U=H_\lambda^{-1}. $$

For a working column w, rounded column q, and factor diagonal d = U[i, i], the eager loop computes:

$$ e=(w-q)/d, \qquad W_{:,i:}\leftarrow W_{:,i:}-e,U_{i,i:}. $$

The within-block loss contribution is (w - q).square() / d.square(), divided by two after the block. The accumulated error matrix updates later blocks through a matrix multiplication. This is why replacing the inverse factor with an ordinary inverse would not be equivalent.

Damping: damp, damp_percent, and damp_auto_increment

Damping adds a diagonal term before factorization:

$$ H_\lambda=H+\lambda,\operatorname{mean}(\operatorname{diag}(H))I. $$

Name	Current meaning	GPTQ default / behavior
damp_percent	Dimensionless fraction λ multiplying the mean Hessian diagonal	Dataclass field starts as None; normalized GPTQ default is 0.05
damp_auto_increment	Additive step for the compatible retry grid	Field starts as None; normalized GPTQ default is 0.01
damp in returned results/logging	The damping fraction actually selected by the inverse routine	May differ from the requested starting fraction
percdamp	Argument name in the low-level Hugging Face compatibility wrapper	Wrapper default is 0.01, not the GPTQConfig default

0.05 means five percent of the mean diagonal, not a literal diagonal addition of 0.05. It is not an inference-time activation scale.

Configuration validates 0 < damp_percent < 1. Negative damp_auto_increment is rejected; zero is accepted and leaves only the starting damping candidate. With a positive increment, candidates reproduce repeated floating-point addition from the configured start while the next candidate remains below 1.0. The inverse routine searches for a usable factor, restores the diagonal between probes, and, after failures, can retry with diagonal floors over up to six floor attempts. Normal CPU/CUDA probes use torch.linalg.cholesky_ex(..., check_errors=False); NPU uses npu_inverse_cholesky_factor.

An unrecoverable factorization returns (None, 1.0). The ordinary quantization loop can then proceed without Hessian error updates. This is distinct from the separate insufficient-calibration fallback policy, which may select RTN before an inverse is attempted. Inspect the selected damping, fallback warnings, samples, and loss together rather than assuming that a completed run used the original settings for every module.

Activation ordering: desc_act and act_group_aware

Classic activation ordering (desc_act=True, also called act-order) sorts input columns by descending Hessian diagonal. GPTQ permutes both weights and the Hessian, quantizes in that order, and restores the original column order before returning the weight tensor. It also restores the group-index mapping needed by packing/inference.

The current API additionally provides group-aware ordering:

Setting	Effective GPTQ default	Behavior
desc_act	False	Global descending Hessian-diagonal order when enabled
act_group_aware	True after normalization	Uses local group permutations and a global group permutation, composes them, and extends the permutation over a partial tail
Both False	Explicit opt-out	Natural input-column order
Both explicitly True	Invalid	Raises a configuration error

Both fields are declared as optional and initially default to None; the table lists the resolved GPTQ behavior. Setting desc_act=True without specifying act_group_aware automatically disables group-aware ordering with a warning. Set act_group_aware=False explicitly to make classic act-order intent unambiguous.

Do not assume that desc_act=False alone means no ordering: the current GPTQ default still enables act_group_aware. Ordering and group-index support must also be compatible with the chosen inference kernel; see Kernel Backends Reference.

Groups and static_groups

group_size is the number of input columns sharing a scale/zero point for an output row. It is not the block-update width. The default group size is 128; -1 selects a full-row group. GPTQ.quantize(blocksize=128) controls the computational block width independently.

Full-row GPTQ requires an explicit ordering change: task construction rejects act_group_aware=True with group_size <= 0. Because group-aware ordering resolves to True by default, use GPTQConfig(group_size=-1, act_group_aware=False, static_groups=False) for natural-order full-row quantization. RTNConfig(group_size=-1) has no group-aware ordering field and does not have this restriction.

static_groups	Parameter discovery	Interaction with classic act-order
False (default)	Refresh quantizer parameters at group boundaries using the current working weight matrix	Groups follow the processing order and require the corresponding g_idx mapping
True	Deep-copy and fit one quantizer per original group before the column loop	The column's original index selects the precomputed group after the desc_act permutation

The static-group implementation iterates with range(0, columns, group_size). Use a positive group size when requesting static groups; group_size=-1 is the full-row mode, not a positive iteration step for that precomputation. Static groups do not disable the Hessian or the sequential error correction.

The ordering and static-group descriptions above refer to ordinary GPTQ's dense-Hessian solve. GPTAQ and embeddings have separate loops; their differences are described below.

true_sequential is a model-replay setting

true_sequential=True is the default. Model definitions organize a layer into subsets of modules. The looper quantizes a subset, replays its quantized activations, then uses those updated activations to calibrate later subsets in the layer.

With true_sequential=False, ModuleLooper combines the subsets into a single subset. Later modules therefore do not receive the same within-layer sequential replay. This is separate from GPTQ's per-column loop and its blocksize.

The source explicitly warns that disabling sequential replay can produce lower but misleading reconstruction losses and worse post-quantization behavior. Leave it enabled unless an experiment deliberately studies that change, and evaluate the resulting model rather than comparing only logged layer losses.

Configuration reference

Shared fields used by GPTQ and RTN

GPTQConfig and RTNConfig inherit PreProcessorConfig, which inherits BaseQuantizeConfig. These are the central fields for this page; device scheduling, adapters, telemetry, and other shared controls are documented in QuantizeConfig Reference.

Field	Declared default	Meaning / constraints
bits	4	Integer bit widths listed by the base config are 2, 3, 4, 5, 6, 7, 8; available layouts and kernels impose additional restrictions
group_size	128	Positive input-column group size, or -1 for a full-row group
sym	True	Symmetric range selection; False enables affine asymmetric scale/zero selection
desc_act	None	Resolves to False for both concrete configs
true_sequential	True	Within-layer sequential calibration replay; meaningful for the calibration-driven lifecycle
method	METHOD.GPTQ	Family identifier, including RTN's GPTQ-compatible weight-only configuration
format	FORMAT.GPTQ	Exported checkpoint representation, not algorithm selection
mse	MSEConfig()	Shrink-grid exponent and scratch budgets; a numeric value remains accepted as the legacy exponent form
dynamic	None	Per-module overrides and negative skip rules
fallback	Fallback()	Insufficient-calibration policy, defaulting to RTN below a "0.5%" threshold
lm_head	False	Whether the output head is selected for quantization
activation	None	Explicit opt-in input-activation policy; omitted for ordinary weight-only GPTQ
pack_dtype	torch.int32	Integer storage word type; not the model activation dtype
pack_impl	"auto"	Packing hint; source also defines "original", "gpu", and "cpu"
offload_to_disk	True	Offload completed module storage during quantization
offload_to_disk_path	None	Optional offload directory
device	None	Device selection, normalized by loading
rotation	None	Optional rotation; choices are "hadamard" and "random"
preprocessors	[]	Preprocessor configurations
smoother	None	Preferred smoother configuration
smooth	None	Backward-compatible smoothing alias; new code should use smoother

GPTQ-specific fields

Field	Declared default	Resolved default / meaning
damp_percent	None	0.05 for GPTQ
damp_auto_increment	None	0.01 for GPTQ
act_group_aware	None	True for GPTQ, unless classic act-order disables it
static_groups	False	Precompute group parameters before the column loop
gptaq	None	A GPTAQConfig enables GPTAQ
foem	None	Separate FOEM configuration; not GPTAQ and not enabled by default
mock_quantization	False	Simplified computation for loading/lifecycle validation; not a quality configuration
hessian	HessianConfig()	Gram accumulation, staging, chunking, and shared-input deduplication controls

GPTQConfig.allowed_quant_methods(self) returns (METHOD.GPTQ,). QuantizeConfig(...) is a backward-compatible factory that dispatches to concrete config classes; explicit GPTQConfig and RTNConfig make the desired lifecycle clearer.

Insufficient-calibration fallback

Ordinary GPTQ checks coverage before materializing/inverting the Hessian. This is a per-module policy, not whole-model RTN selection and not the same as recovery from a failed Cholesky factorization.

Fallback field	Dataclass default	Meaning
strategy	FallbackStrategy.RTN	Weight-only rounding strategy when coverage is insufficient
threshold	"0.5%"	Trigger when observed activation rows/tokens are strictly below this fraction of the processor's total calibration-token count
smooth	None	Optional smoothing/range search for the fallback solve

Threshold configuration	Behavior
fallback=None	Disable the configured low-sample fallback and its NaN-to-mock retry policy
Fallback(threshold=True)	Trigger only for zero observed samples; resolves to 1e-9 and uses strict <
Fallback(threshold=100)	Trigger for fewer than 100 observed samples
Fallback(threshold="0.5%")	Compare against 0.005 * expected_total_samples; if the expected total is unavailable or zero, the resolver uses 1.0 as its base
Fallback(threshold=0) or Fallback(threshold=False)	Disable the threshold
fallback={"strategy": "rtn"}	Base-config dictionary normalization supplies "1.0%" when threshold is omitted, unlike the Fallback() dataclass's "0.5%"; set the threshold explicitly

FallbackStrategy defines the string values "rtn", "midpoint", "mean", "median", and "stdclip". The non-RTN strategies use alternative weight-range/center rules; stdclip uses a three-standard-deviation interval. They do not perform second-order compensation. A SmoothMSE fallback takes the MSE-search branch before strategy-specific range selection.

from gptqmodel.quantization import GPTQConfig
from gptqmodel.quantization.config import Fallback, FallbackStrategy

qcfg = GPTQConfig(
    fallback=Fallback(strategy=FallbackStrategy.RTN, threshold="0.5%"),
)

The early fallback returns a loss string of the form fallback(rtn): ... for RTN, not a GPTQ reconstruction loss. An exhausted inverse solve can instead continue without Hessian updates and report a fallback label or sentinel loss. Neither case establishes downstream accuracy. GPTAQ's overridden solve does not run this early coverage check or the NaN-to-mock retry.

GPTAQConfig

GPTAQConfig is a nested algorithm configuration, not a replacement for GPTQConfig:

@dataclass
class GPTAQConfig:
    alpha: float = field(default=0.25)
    device: Union[str, torch.device] = field(default="auto")

Field	Default	Meaning
alpha	0.25	Multiplier on the activation-difference correction; validated as numeric
device	"auto"	Storage placement for captured native inputs; a non-empty string or torch.device

NativeProcessor resolves "auto" to the project's DEVICE_1, and "cpu" to CPU. This is native-reference storage placement, not a new checkpoint format. Extra native inputs and cross-term state increase memory requirements compared with ordinary GPTQ.

RTNConfig

RTNConfig adds no Hessian, GPTAQ, or damping fields. It inherits the shared and preprocessor fields, declares method=METHOD.GPTQ and format=FORMAT.GPTQ, resolves desc_act to False, and returns True from uses_weight_only_lifecycle().

Do not select RTN with a nonexistent METHOD.RTN enum member. Use RTNConfig(...). Its GPTQ family identifier permits GPTQ-style export; its concrete class selects the weight-only path.

Relevant export enums

Enum	String value	Interpretation
METHOD.GPTQ	"gptq"	GPTQ family identifier
FORMAT.GPTQ	"gptq"	Legacy/continuous GPTQ checkpoint layout
FORMAT.GPTQ_V2	"gptq_v2"	Continuous layout with corrected zero semantics
FORMAT.GPTQ_P	"gptq_p"	Planar GPTQ layout, using v2 zero semantics on disk
FORMAT.MARLIN	"marlin"	Legacy format enum; not the recommended format for new quantization
FORMAT.BITBLAS	"bitblas"	BitBLAS export representation
FORMAT.GEMM	"gemm"	AWQ GEMM representation, also listed for RTN export
FORMAT.GEMV	"gemv"	AWQ GEMV representation, also listed for RTN export
FORMAT.GEMV_FAST	"gemv_fast"	AWQ GEMV-fast representation, also listed for RTN export
FORMAT.LLM_AWQ	"llm-awq"	LLM-AWQ representation, also listed for RTN export

GPTQ_EXPORT_FORMATS lists GPTQ, GPTQ_V2, GPTQ_P, MARLIN, and BITBLAS. RTN_EXPORT_FORMATS lists GPTQ, GPTQ_V2, GPTQ_P, GEMM, GEMV, GEMV_FAST, and LLM_AWQ. A listed format does not guarantee that every bit width, group size, device, or backend supports it. In particular, BaseQModel.quantize() explicitly rejects new quantization into FORMAT.MARLIN and recommends FORMAT.GPTQ, which can use a Marlin inference kernel.

For the GPTQ family, 5-, 6-, and 7-bit layers require the planar layout. Configuration automatically changes a requested FORMAT.GPTQ or FORMAT.GPTQ_V2 to FORMAT.GPTQ_P if the base bit width or a non-negative dynamic override uses one of those widths. Do not infer that the continuous layouts support every integer listed in the base bits field.

Usage

Quantize a model with GPTQ

Use representative calibration text from the deployment domain. The following is the API pattern; calibration_texts should be a real list of calibration strings, not a few synthetic prompts used as a production recipe.

from gptqmodel import GPTQModel
from gptqmodel.quantization import GPTQConfig

qcfg = GPTQConfig(
    bits=4,
    group_size=128,
    sym=True,
    true_sequential=True,
)

model = GPTQModel.load("/models/dense-model", quantize_config=qcfg)
model.quantize(calibration_texts, batch_size=1)
model.save("/models/quantized-gptq")

This keeps the current default group-aware ordering. For classic global act-order, specify:

qcfg = GPTQConfig(
    bits=4,
    group_size=128,
    desc_act=True,
    act_group_aware=False,
    static_groups=True,
    true_sequential=True,
    damp_percent=0.05,
    damp_auto_increment=0.01,
)

For an explicitly natural-order experiment, set both desc_act=False and act_group_aware=False. Change one factor at a time when comparing accuracy.

Enable GPTAQ

from gptqmodel import GPTQModel
from gptqmodel.quantization import GPTAQConfig, GPTQConfig

qcfg = GPTQConfig(
    bits=4,
    group_size=128,
    act_group_aware=False,
    gptaq=GPTAQConfig(alpha=0.25, device="cpu"),
)
model = GPTQModel.load("/models/dense-model", quantize_config=qcfg)
model.quantize(calibration_texts)
model.save("/models/quantized-gptaq")

The model lifecycle inserts native input capture; do not bypass it by constructing an unprepared GPTAQ task and assuming its reference inputs will exist. Ordinary GPTQ and GPTAQ both reject a missing calibration dataset at the model API level. For GPTAQ, explicitly set act_group_aware=False: its own loop does not implement ordinary GPTQ's group-aware permutation and supports classic desc_act or natural order instead.

Quantize without calibration using RTN

from gptqmodel import GPTQModel
from gptqmodel.quantization import FORMAT, RTNConfig

qcfg = RTNConfig(bits=4, group_size=128, sym=True, format=FORMAT.GPTQ)
model = GPTQModel.load("/models/dense-model", quantize_config=qcfg)
model.quantize()
model.save("/models/quantized-rtn")

This is a native weight-only lifecycle. It does not manufacture an identity Hessian or feed dummy calibration through GPTQ. It is useful as a direct rounding baseline, but its error statistic is not GPTQ's Hessian reconstruction loss.

Asymmetric weights and v2 checkpoint zeros

from gptqmodel.quantization import FORMAT, GPTQConfig

qcfg = GPTQConfig(
    bits=4,
    group_size=128,
    sym=False,
    format=FORMAT.GPTQ_V2,
)

This selects affine asymmetric weight quantization and v2 checkpoint zero semantics. It does not enable GPTAQ. Conversely, gptaq=GPTAQConfig(...) does not require sym=False.

Legacy gptq (v1) serialization shifts packed zeros by one; loading for a v2-requiring kernel applies the corresponding correction. This matters particularly for asymmetric zero points. MIN_VERSION_WITH_V2 is "0.9.0": the loader guards asymmetric format="gptq" checkpoints using quantizer producer/version metadata. Its helper methods have historical names such as is_quantized_by_gptaq(), but the actual check is for a gptqmodel producer version at or above that minimum; the name is not proof that GPTAQ ran.

Do not fix compatibility by editing checkpoint_format, sym, or packed qzeros manually. Format conversion is bit-layout-aware; for example, INT3 zero fields can span packed words. Choose a compatible exporter/loader/backend and preserve the recorded producer metadata. See Packing Formats and Compatibility.

Bound Hessian staging and enable MSE shrink search

import torch
from gptqmodel.quantization import GPTQConfig, HessianConfig, MSEConfig

qcfg = GPTQConfig(
    bits=4,
    group_size=128,
    hessian=HessianConfig(
        chunk_size=1024,
        staging_dtype=torch.float32,
        dedup_shared_inputs=True,
    ),
    mse=MSEConfig(norm=2.4, grid_budget="128M"),
)

The example values 1024, 2.4, and "128M" are explicit choices, not project defaults. Hessian chunking bounds input-materialization work; it does not remove the dense [in_features, in_features] Gram matrix or its factorization. MSE workspace budgets are separate from Hessian memory.

HessianConfig field	Default	Meaning
chunk_size	None	Optional row-count limit for Hessian accumulation chunks
chunk_bytes	None	Optional chunk byte budget
staging_dtype	torch.float32	Staging dtype; allowed choices are float32, float16, and bfloat16
dedup_shared_inputs	True	Enable compatible shared-input Hessian reuse

The Gram products and accumulated Hessian are float32. A lower staging dtype changes staging/storage behavior; it does not make Cholesky a low-bit operation. chunk_size takes precedence over chunk_bytes when both are set. A byte budget smaller than one row still stages at least one row. Lower-precision staging is used only for FP16/BF16 inputs, and BF16 additionally requires device support; otherwise staging falls back to FP32. These controls apply to ordinary GPTQ's accumulation, not GPTAQ's overridden recurrence.

For embeddings, GPTQ uses a dedicated diagonal-statistics path: it counts calibration token IDs instead of constructing a dense one-hot input matrix. Internally, the embedding weight is transposed so these statistics index vocabulary/token columns. Token IDs outside [0, num_embeddings) are rejected. The embedding solve quantizes weight groups with the applicable ordering, restores the weight/group layout, and reports a diagonal-weighted reconstruction loss. static_groups=True bypasses embedding ordering; this path does not run the dense inverse-factor/error-update loop or the ordinary early low-sample fallback. Do not interpret embedding token-frequency statistics as the dense Linear Hessian described above.

MSEConfig field	Default	Meaning
norm	0.0	Exponent of the reconstruction-error objective; search runs when positive
grid_budget	None	GPTQ/quantizer vectorized shrink-grid scratch target
clip_grid_budget	None	AWQ clipping-grid scratch target; not an additional GPTQ loss

The implicit budget is 16 MiB on CPU and 256 MiB on non-CPU devices. Byte budgets accept non-negative integers or strings with binary K, M, or G suffixes, optionally followed by B; for example, "128M" means 128 * 2**20 bytes. 0 forces single-candidate chunks (and one-token product tiles for AWQ), not zero allocation. They are calibration-time scratch targets, not checkpoint fields that change stored weight precision, and are not a total-process memory cap. mse=2.4 is the legacy equivalent of setting the exponent through MSEConfig(norm=2.4).

Implementation internals

Entry points and compatibility signatures

The model-level method is declared in gptqmodel/models/base.py as follows (comments and body omitted):

def quantize(
    self,
    calibration: Optional[Union[List[Dict[str, Union[List[int], torch.LongTensor]]], List[str], List[int]]] = None,
    calibration_concat_size: Optional[int] = None,
    calibration_sort: Optional[str] = "desc",
    batch_size: int = 1,
    tokenizer: Optional[PreTrainedTokenizerBase] = None,
    backend: Optional[BACKEND] = BACKEND.AUTO,
    adapter: Adapter = None,
    adapter_calibration_dataset: Union[List[Dict[str, Union[List[int], torch.LongTensor]]], List[str], List[int]] = None,
    calibration_data_min_length: int = 10,
    calibration_concat_separator: Optional[str] = None,
    embed_quant_config: Optional[Union[QuantizeEmbedConfig, QuantizeEmbed]] = None,
    embed_quant_mode: Optional[QuantizeEmbed] = None,
    checkpoint: Optional["CheckpointConfig"] = None,
) -> Dict[str, List[Dict[str, str]]]:
    ...

The accepted calibration annotation includes raw strings and tokenized inputs. calibration_sort supports "asc", "desc", and "shuffle"; its default is "desc". A model definition that disables batch quantization can override the requested batch size. calibration=None is allowed by the signature so native weight-only configurations such as RTNConfig can use the same method; GPTQ and GPTAQ still require calibration at runtime.

The low-level task class in gptqmodel/quantization/gptq.py has these signatures (method declarations shown without bodies):

def __init__(self, module: nn.Module, qcfg: Optional[QuantizeConfig] = None, region_timer=None):
    ...

def quantize(
        self,
        blocksize=128,
):
    ...

def hf_quantize(
        self,
        blocksize=128,
        percdamp=0.01,
        damp_auto_increment=0.0015,
        group_size=-1,
        actorder=False,
        static_groups=False,
        act_group_aware: Optional[bool] = None,
):
    ...

hf_quantize() copies its arguments onto the task configuration, resolves ordering, calls quantize(), assigns Q to the module weights, and returns (scale, zero, g_idx, duration, avg_loss, damp_percent). fasterquant() delegates to this compatibility path, without exposing its final act_group_aware parameter. Do not confuse the wrapper's 0.01/0.0015 damping defaults with the normalized GPTQConfig defaults 0.05/0.01. Its default group_size=-1 does not itself disable a task's existing group-aware ordering: explicitly disable that ordering for full-row use.

The direct task quantize() result is:

(Q, scale, zero, g_idx, duration, avg_loss, damp, nsamples)

Q is a dequantized representation of the rounded weights for module assignment and packing; it is not already the packed checkpoint qweight. GPTAQ and RTN return the corresponding eight-part payload so downstream packing can share the same interface.

1. Module shapes and activation capture

GPTQ handles nn.Linear, Transformers Conv1D, nn.Conv1d, nn.Conv2d, and a dedicated nn.Embedding path. Linear-like inputs become two-dimensional rows. Convolution inputs are unfolded into receptive-field rows; Transformers Conv1D weights use the corresponding transposed weight convention. Tensor-parallel padding is handled before quantization where required.

GPTQ.add_batch(inp, out, batch_index=None) updates per-device Hessian partials and sample accounting. compute_hessian_xtx(matrix) computes float32 matrix.T @ matrix; with chunking, it accumulates chunk products into a float32 square matrix and releases temporary materializations. materialize_global_hessian() merges device partials in stable device order and applies the 2 / total_samples normalization.

The processor's forward hook is responsible for supplying usable inputs. Masked inputs preserve per-sample boundaries where required, and embeddings have their own handling rather than being treated as ordinary Linear input activations.

2. Finalization, dead columns, and ordering

Before the main loop, GPTQ:

1. Checks whether observed calibration is below the configured fallback threshold.
2. If a fallback is selected, releases unused Hessian partials and enters the fallback quantizer.
3. Otherwise finalizes the Hessian, clones weights to its compute device, and finds initial quantizer parameters.
4. Neutralizes calibration-dead columns: zero Hessian diagonal entries become 1.0, and the matching weight columns become zero.
5. Precomputes static-group quantizers when requested.
6. Applies classic or group-aware ordering.
7. Computes or reuses the inverse factor and initializes output/loss buffers.

The implementation has targeted CUDA-out-of-memory recovery around Hessian permutation and inversion that moves the affected work to CPU and logs a CPU fallback. This does not make CPU and GPU memory interchangeable: a dense Hessian still requires storage and factorization workspace on whichever device performs that stage.

3. Quantizer parameter selection and the shrink grid

The standard quantizer in gptqmodel/quantization/quantizer.py exposes:

def __init__(self, qcfg: BaseQuantizeConfig, shape=1, name: str=None):
    ...

def configure(
        self,
        perchannel=False,
        grid=100,
        maxshrink=0.8,
        trits=False,
        bits: int | str | None = None,
        sym: bool | None = None,
):
    ...

def find_params(self, x, weight=False):
    ...

The GPTQ processor calls configure(perchannel=True). Thus weight ranges are fitted per output channel, with group boundaries selecting the input-column subset. This differs from constructing a raw Quantizer and leaving its perchannel=False default unchanged.

For the standard affine path, with maxq = 2**bits - 1, quantization is:

q = torch.clamp(torch.round(x / scale) + zero, 0, maxq)
x_hat = scale * (q - zero)

find_params() includes zero in the observed range, adjusts ranges for sym=True, uses midpoint zeros for symmetric affine quantization, and rounds -xmin / scale for asymmetric quantization. All-zero ranges receive finite sentinel ranges. _replace_zero_scale() prevents division by zero and repairs undefined zero points in affected lanes.

When mse.norm > 0, the shrink grid evaluates smaller ranges. With the raw quantizer defaults, candidates are based on p = 1 - i / grid for i in range(int(maxshrink * grid)). For each candidate, scale/zero values are recomputed and ranked by the sum of abs(x_hat - x)**norm. Despite the name, an exponent such as 2.4 is supported; it need not be mathematical mean squared error with exponent exactly two.

The vectorized implementation preserves sequential candidate order and first-minimum tie-breaking, treats NaN objectives as non-winning, and preserves reduction order for transposed views. Scratch budgeting chunks the candidate evaluation; it is not a different codebook or an additional inference step. The asymmetric 2-bit implementation additionally searches an adjacent affine zero-point orientation.

4. Block and column loop

In the ordinary non-mock path, GPTQ iterates blocks, normally 128 input columns at a time:

1. Clone the current block into W1 and take output/loss slices.
2. At each column, select a static-group quantizer or refresh parameters at a dynamic group boundary. Dynamic parameters are fitted from the current outer working matrix W, not refitted from the locally corrected W1 after every column.
3. Quantize the column with self.quantizer.quantize(w.unsqueeze(1)).flatten().
4. Save the rounded column and its factor-normalized error.
5. Correct the remaining columns inside W1 using the inverse-factor row.
6. Divide block loss by two and update columns after the block using Err1.matmul(Hinv[i1:i2, i2:]).
7. Release block temporaries.

Finally, GPTQ restores ordering and module shape, builds g_idx, concatenates scales/zeros, and returns the payload. With a valid inverse and nonzero samples, average loss is sum(Losses) / nsamples. A no-inverse path omits Hessian corrections; its work chunks can align to positive group boundaries.

This separation between columnwise quantization and blockwise matrix updates is the core GPTQ acceleration structure. group_size, blocksize, calibration batch size, and Hessian accumulation chunk size control different parts of the algorithm.

5. Optional native CUDA block update

gptqmodel/quantization/gptq_cuda.py wraps a JIT-loaded Torch operator. The callable interface is:

def block_update_available() -> bool:
    ...

def gptq_block_update(
    work: torch.Tensor,
    hinv: torch.Tensor,
    scale: torch.Tensor,
    zero: torch.Tensor,
    column_group: torch.Tensor,
    quantized: torch.Tensor,
    errors: torch.Tensor,
    losses: torch.Tensor,
    maxq: int,
) -> None:
    ...

The wrapper uses gptqmodel_ext/gptq_block/gptq_block.cpp and gptq_block.cu. block_update_available() builds/loads on first use and returns False if CUDA compilation fails. The ordinary PyTorch loop remains the alternative.

Dispatch condition	Requirement
Hessian factor	Must exist
Device	Working weights on a CUDA device
Block size	Requested blocksize == 128; only full 128-column blocks use the native update
Bit width	Integer 2–8, with the expected (1 << bits) - 1 quantizer bound
Quantizer	Exactly the standard Quantizer type
Mock mode	Disabled
Environment	env_flag("GPTQMODEL_DISABLE_GPTQ_CUDA") is false
Extension	block_update_available() succeeds

The operator checks float32 work/factor/scale/zero/output tensors on the same CUDA device, a 128-by-128 factor block, 128 contiguous work columns, and CUDA int32 column-group indices. Python prepares group parameters before launch, including blocks that cross groups. Short tail blocks stay in the eager loop. The update to later blocks remains outside the fused within-block operator.

Environment variable	Purpose
GPTQMODEL_DISABLE_GPTQ_CUDA	Disable the optional native GPTQ update for diagnosis
GPTQMODEL_GPTQ_BLOCK_BUILD_ROOT	Override this extension's build directory
GPTQMODEL_GPTQ_BLOCK_FORCE_REBUILD	Request rebuilding through the JIT extension helper
GPTQMODEL_EXT_VERBOSE	Extension build verbosity

This is a quantization-time accelerator, not Marlin, ExLlama, or another inference backend. It requires a working PyTorch extension toolchain and CUDA compilation support. The .cu implementation is not a blanket guarantee of support on every ROCm build: ROCm uses PyTorch's CUDA-facing device API too, but the extension must actually build/load. Failure to load leaves the eager path available.

For diagnosis, set GPTQMODEL_DISABLE_GPTQ_CUDA=1 before quantizing. The shared env_flag helper recognizes 1, true, yes, on, and y, case-insensitively; setting the variable to 0 does not disable the extension.

GPTAQ: native inputs and the activation-difference term

GPTAQ subclasses GPTQ, but it keeps its own input-statistics recurrence and block loop. Its constructor takes module: NamedModule, a configuration, and the optional region timer. It consumes native inputs from module state under NATIVE_INPUTS_STATE_KEY = "native_inp".

The calibration lifecycle supplies two input streams:

Stream	Role
Native reference	Detached inputs captured by NativeProcessor before the downstream quantized replay
Current/replayed input	Inputs seen during quantization after upstream changes and any active activation replay

GPTAQ converts aligned input matrices to float32 and maintains both H and dXXT. In its channel-by-token representation, the updates include inp @ inp.T and (native_inp - inp) @ inp.T. Its recurrence is normalized by calibration requests, while nsamples tracks activation columns/tokens.

After neutralization, ordering, and inverse-factor computation, it forms the correction:

P = self.qcfg.gptaq.alpha * ((self.dXXT @ Hinv.T).triu(diagonal=1)) @ Hinv

The GPTAQ column loop uses both Hessian compensation and this native-versus-current activation correction. _token_normalized_avg_loss() converts its batch-normalized loss sum into GPTQ-compatible token-normalized reporting. A prepared task must have non-None qcfg.gptaq; the loop checks this after computing the inverse and raises ValueError if it is absent. Constructing a task without its required native cache can fail earlier.

GPTAQ does not automatically inherit every optimization or recovery policy from GPTQ.quantize():

Feature	GPTAQ behavior at this revision
Ordering	Its loop implements desc_act and natural order, not act_group_aware; explicitly set act_group_aware=False to avoid relying on the ordinary GPTQ default
Input statistics	Its own float32 recurrence; does not use HessianConfig chunked Gram accumulation
Shared-input reuse	Excluded from the processor's plain-GPTQ Hessian deduplication/factor-cache cohort
Fused CUDA block update	Not used by the GPTAQ loop
Low-sample and NaN recovery	No ordinary GPTQ early fallback or NaN-to-mock retry; a NaN average loss raises ValueError
Inverse failure	Reuses hessian_inverse(), but then requires a valid factor for P and the loop; no ordinary GPTQ no-inverse continuation
Input reshape	Explicit handling for Linear/Transformers Conv1D and nn.Conv2d; do not infer Conv1d or embedding support from the superclass module validator alone

Both native capture and task selection must be enabled consistently. BaseQModel inserts NativeProcessor from the top-level gptaq setting, while per-module task selection also reads dynamic overrides. Adding GPTAQ only through a dynamic rule does not itself insert the required native processor.

The correction is not synonymous with enabling four-bit activations, and alpha is not a damping coefficient. GPTAQ's extra native-input cache and cross-term matrix must be included in memory planning. The documented GB10 W4A4 campaign explicitly dropped GPTAQ because of its memory contract; a completed checkpoint from that experiment was not an accepted accuracy result.

RTN: direct weight-only quantization

gptqmodel/quantization/rtn.py defines:

def __init__(self, module: nn.Module, qcfg: RTNConfig):
    ...

def quantize(self):
    ...

quantize() runs under torch.inference_mode(). The task supports nn.Linear, nn.Conv1d, nn.Conv2d, and Transformers Conv1D. It clones/reshapes weights, applies tensor-parallel padding if required, determines the effective group size, fits parameters per group, and directly rounds/dequantizes the weights. It never enters GPTQ's activation/Hessian lifecycle.

Optional smoothing can change range fitting. A supplied SmoothMSE() defaults to steps=32 and maxshrink=0.8, but this is not the raw quantizer's 100-point shrink grid: mse_optimal_quant() converts steps to a logarithmic ternary-search iteration count, searches retained range multipliers between clamp(maxshrink, 1e-3, 1.0) and 1.0, and refines three nearby candidates. Thus the default 0.8 is the lower retained-range multiplier, not an 80% reduction. That search is not second-order compensation. The result's loss field is a string rtn: {mean_abs_err:.7f}, damping is 0.0, and sample count remains 0 because no calibration inputs were captured.

RTN has a separate MLX quantization fast path:

Eligibility	Required value
OS / architecture	Darwin / arm64
Dependency	mlx.core imports successfully
Weight device	CPU or MPS
Weight shape / size	Two-dimensional, at least 256 * 1024 elements
Smoothing	None
Bits	Integer 2–8
Group size	-1 or positive

The MLX path returns quantized weights, scales, and zeros through DLPack. Otherwise RTN uses its Torch implementation. This is an Apple Silicon quantization optimization, not an MLX requirement for all RTN runs and not a promise that every resulting checkpoint has an MLX inference kernel. See MLX Apple Silicon.

Looper integration

The normal model API uses GPTQProcessor, not direct user-managed Hessians.

Phase	Implementation behavior
Per-module configuration	clone_gptq_config_for_module() deep-copies the config and applies dynamic overrides for bits, symmetry, MSE, group size, ordering, damp_percent, static groups, fallback, Hessian settings, GPTAQ, and FOEM; it does not apply a dynamic damp_auto_increment override
Task selection	preprocess() selects GPTAQ when gptaq is present, otherwise FOEM when configured, otherwise GPTQ
Quantizer setup	Configures each task quantizer with perchannel=True
Capture	pre_process_fwd_hook() routes activations to add_batch(), with masking/per-sample and embedding handling
Shared-input reuse	Compatible explicit shared-input groups can elect a Hessian leader; followers adopt copied Hessian data and can reuse compatible inverse factors
Solve	process() calls g.quantize(), records timing/loss/sample/damping information, moves scale/zero/group payloads to CPU, and assigns rounded weights to the module
Pack	submodule_finalize() creates the runtime quantized module and calls pack_module(...)
Finish	Marks the model quantized and records METHOD.GPTQ

Shared-input deduplication is enabled by HessianConfig.dedup_shared_inputs=True, but is compatibility- and group-aware and limited to tasks whose exact type is GPTQ; it is not arbitrary sharing across modules with unrelated inputs and does not include GPTAQ/FOEM tasks. The processor's name is "gptq", "gptaq", or "foem" according to active configuration. Dataset verification rejects absent calibration for this processor.

For GPTAQ, BaseQModel inserts a NativeProcessor before quantization processing. It captures detached inputs on the configured native-reference device and transfers the cache into module state for the GPTAQ task. RTN instead selects the model's weight-only lifecycle through RTNConfig.uses_weight_only_lifecycle(); it does not invoke the GPTQ calibration processor with an empty dataset.

See Looper Processors, Quantization Lifecycle, and Model Definitions and Module Tree.

Quantization protocol: algorithm versus representation

docs/quantization_protocol.md describes a version-2 protocol organized as version and ordered stages, with rules matching model objects and configuring weight, input, output, or kv_cache targets. Its key distinction for GPTQ is:

Section	Question answered	Example intent
prepare	What transforms happen before quantization?	Weight smoothing
quantize	How are quantized values produced?	GPTQ versus RTN
quantize.fallback	What if calibration evidence is insufficient?	RTN fallback for sparse expert traffic
export	How is the result represented?	GPTQ-packed weights versus AWQ GEMM layout

The design document contains forward-looking DSL examples, multi-rule patch semantics, and activation-aware activation_mode="ignore", "fake", and a reserved "real" mode. Those examples are not all implemented constructor arguments. In particular, they are not a v2 flag and should not be passed indiscriminately to GPTQConfig.

The actual gptqmodel/quantization/protocol.py compiler is narrower. compile_protocol() parses only protocol version 2. compile_plan_to_quantize_config() currently accepts exactly one stage and one rule, only the weight target, and rejects aliases, actions, stop, non-default target replacement mode, and input/output/KV-cache compilation. Method compilation supports "gguf", "gptq", and "awq"; the document's method: rtn translation is not currently accepted by that compiler. Use RTNConfig directly for current RTN execution.

This single-rule GPTQ example uses the implemented mapping API:

from gptqmodel.quantization.protocol import compile_protocol_to_quantize_config

qcfg = compile_protocol_to_quantize_config({
    "version": 2,
    "stages": [{
        "name": "ptq",
        "rules": [{
            "match": "*",
            "weight": {
                "quantize": {
                    "method": "gptq",
                    "bits": 4,
                    "group_size": 128,
                    "sym": True,
                    "desc_act": False,
                    "act_group_aware": True,
                },
                "export": {"format": "gptq"},
            },
        }],
    }],
})

The GPTQ compiler maps bits, group size, symmetry, ordering, export format, and supported negative match exclusions into a QuantizeConfig; it is not a general transport for every field in the design document. It rejects weight.prepare for GPTQ/AWQ and does not forward arbitrary quantize arguments such as damping, fallback, or nested GPTAQ settings. Unmapped arguments can remain in the parsed plan without affecting the constructed config; inspect the result rather than assuming parsing means they took effect. YAML parsing uses yaml.safe_load and requires PyYAML. A parsed execution plan is not by itself evidence that every planned feature can compile into the current model lifecycle.

Accuracy and low-precision activations

Ordinary GPTQ is not W4A4

With activation=None, ordinary GPTQ keeps its weight-only calibration/inference behavior. The opt-in W4A policies change selected GEMM operands while retaining ordinary INT32-packed GPTQ qweight, qzeros, scales, and g_idx in the checkpoint. They do not establish that GPTAQ is enabled or that all model operations run at four or eight bits.

The GB10 guide distinguishes:

Policy	Activation operand	Weight arithmetic
w4afp8	E4M3 with one dynamic scale per token	Exact centered INT4 codes represented in E4M3; original GPTQ group scales applied to partial products
w4a_nvfp4	E2M1 codes with per-16 E4M3 scales and a global scale	Two exact E2M1 integer planes reconstruct centered INT4 values; original GPTQ group scales remain authoritative

Residual arithmetic, RMSNorm, attention internals, RoPE, SiLU, and the current KV cache are not thereby converted into fully low-precision arithmetic. The current transport keeps a wider reference for residual/norm operations; selected GEMMs consume encoded operands and scales. Q/K/V share an encoded normalized input, as do gate/up. The checkpoint weight codebook remains GPTQ's codebook.

For fresh quantization, activation replay rounds selected Linear inputs before Hessian capture and propagates their effects downstream. The replay keeps decoded GEMM operands in FP32 until the projection and returns outputs to the model dtype, matching the intended inference arithmetic. Merely attaching a policy to already packed weights does not redo the GPTQ solve.

Requirements and scope

Component	Verified requirements / scope
Package at this source baseline	Python >=3.10; requirements include Torch >=2.14.1, Transformers >=5.20.2, and Accelerate >=1.15.0; consult the installation page for the complete environment
Ordinary GPTQ / GPTAQ	PyTorch computation plus a compatible selected model/device/backend; the optional CUDA block extension is not required for the eager algorithm
Native GPTQ block update	Working Torch JIT-extension/CUDA build toolchain; full 128-column blocks and the dispatch conditions above
RTN MLX fast path	Optional mlx.core, Darwin arm64, and the eligibility conditions above
W4A FP8 runtime	Linux CUDA, Triton FP8 support, FP16/BF16 model dtype; backend minimum compute capability (8, 9)
W4A NVFP4 runtime	Linux CUDA, FP16/BF16, group size 128, symmetric 4-bit weights, desc_act=False, int32 packing; backend minimum compute capability (12, 1)
GB10 recorded accuracy	Llama 3.2 1B Instruct and explicitly selected decoder coverage; not a guarantee for other models, hardware, or recipes

The W4A backends have zero AUTO-selection priority. An ordinary GPTQ run does not opt into them by merely importing their module definitions. See FP8 and NVFP4 for full activation-policy and kernel requirements.

Both current W4A backends require 4-bit symmetric group-128 gptq/gptq_v2 checkpoints, desc_act=False, int32 packing, and lm_head=False. They declare input features divisible by 128 and output features divisible by 32, without automatic padding, training, or adapter support. The activation config rejects dynamic weight-layout overrides containing bits, sym, group_size, or desc_act. NVFP4 additionally checks for Triton activation packing, torch.float4_e2m1fn_x2, and torch.nn.functional.scaled_mm, ScalingType, and SwizzleType support. These restrictions also apply to the FP8 policy where applicable; the package dependency floor does not replace the native FP4 API checks. Do not assume that a CUDA device with FP8 support can execute NVFP4.

For an eligible model and device, the current explicit opt-in configuration can be written as:

qcfg = GPTQConfig(
    bits=4,
    group_size=128,
    sym=True,
    desc_act=False,
    activation={
        "mode": "w4a_nvfp4",
        "recipe": "least_squares_grid",
        "attention": {"mode": "w4afp8"},
    },
)

This selects a mixed operand policy, not a general accuracy guarantee. Current ActivationConfig accepts mode, recipe, attention, mlp, and global_scales; it does not accept the historical activation.version field shown in older experiment excerpts. Those policy versions are unrelated to the gptq_v2 checkpoint format. Use activation="w4afp8" for the separate per-token FP8 policy, or omit activation for ordinary GPTQ.

What the recorded results establish

docs/gptq-w4a-gb10.md is the usage/validation guide, while docs/gptq-w4a-accuracy-notes.md is an experimental log that includes failed and superseded runs. Its introductory description of an “accepted” balanced policy is not equivalent to a statistically passed noninferiority gate.

The appendix's latest full-model validation section, dated 2026-10-07, reports saved-checkpoint inference at runtime commit 7b6814c2. All 16 decoder layers and 112 projections were selected. Each policy was compared against its own same-weight W4A16 control over all 1,209 GSM8K Platinum rows:

Policy	W4A16 correct / 1,209	Activation-policy correct / 1,209	Change	Paired 95% CI	Budget verdict
Balanced A4: FP8 attention, NVFP4 MLP, least_squares_grid	502 (41.52%)	474 (39.21%)	−2.316 percentage points	[−4.729, +0.097] pp	inconclusive
Per-token FP8, w4afp8	432 (35.73%)	438 (36.23%)	+0.496 percentage points	[−1.618, +2.611] pp	within_budget

The two rows use different GPTQ weight checkpoints. Their absolute activation scores are not an A4-versus-A8 comparison. This particular validation covered saved-checkpoint inference, not fresh quantization.

Other useful lessons from the campaign:

• On one fixed A4-aware plain-GPTQ checkpoint, uniform dynamic NVFP4 scored 412/1,209 against 502/1,209 at W4A16: a −7.44 pp confirmed regression. Keeping attention at FP8, preserving the wider residual reference, and using least_squares_grid recovered the balanced policy's 474/1,209 without changing saved weight bytes.
• The same checkpoint with all 16 MLPs promoted to FP8 scored 506/1,209 (+0.33 pp, within_budget). Promoting only layers 10, 11, 12, and 15 scored 484/1,209 (−1.49 pp), but remained inconclusive; a favorable point estimate is not a passed confidence-interval gate.
• GPTAQ was removed from the GB10 W4A4 campaign because its extra state violated the required 2 GiB memory-headroom contract. The archived GPTAQ checkpoint received no A4 evaluation and is not an accuracy acceptance artifact. This is a campaign constraint, not removal of the general GPTAQ implementation.
• Kernel-oracle, lifecycle, and encoded-boundary tests validate numerical/transport contracts. They do not establish downstream quality by themselves.

Evaluation and calibration discipline

The guide's provisional acceptance test requires the paired 95% interval to rule out a loss greater than 2 percentage points on the full GSM8K Platinum dataset. An inconclusive verdict is not a pass. ARC-Challenge is a sensitivity diagnostic, not the controlling acceptance task.

For meaningful comparisons:

1. Use identical weights for the W4A16 control and its activation-policy candidate; preserve the saved per-row baseline.
2. Match rendered prompts, tokenizer artifacts, targets, row order, and generation settings. Freeze date-dependent chat-template content.
3. Separate calibration/fit data, selection data, and benchmark evaluation data. The W4A calibration loader verifies artifacts and rejects normalized 13-word question overlaps against its evaluation exclusion registry.
4. Verify actual decoder coverage. Historical two-layer fast checkpoints cannot stand in for a full 16-layer run just because the evaluation used all rows.
5. Report point estimates, confidence intervals, and paired correctness flips. Recovered answers and changed answers are different statistics.
6. Keep fresh calibration and loaded-checkpoint inference claims separate; neither retroactively validates the other.

For the representative GB10 calibration, the guide specifies 512 calibration rows with concatenation length 2,048. Its 32-record runs are transport checks, not a production calibration recipe. GB10 tests must run sequentially through tests/models/run_w4a_gb10_safe.sh, which preserves 2 GiB physical/cgroup memory headroom, disables swap in its scope, and guards the shared-memory GPU workload. Quantize/save and reload/audit run in separate guarded processes to avoid retaining quantization objects during a second model load.

See Evaluation and Troubleshooting and FAQ. Treat campaign numbers as model-, checkpoint-, policy-, prompt-, and runtime-specific measurements rather than universal GPTQ accuracy guarantees.

GSQ refinement

GPTQConfig.gsq opts calibrated nn.Linear modules into Gumbel-Softmax refinement after the ordinary GPTQ solve: hard code assignments are re-sampled against the calibration Hessian and kept only when an export-aware candidate improves the local objective. It requires format in GPTQ, GPTQ_V2, or GPTQ_P, plain GPTQ (no gptaq, foem, or mock_quantization), and calibration activations; tensor-parallel padded projections are rejected. RTNConfig.gsq offers the same refinement on scalar GPTQ or AWQ export formats, with AWQ-family formats requiring 4 bits. GPTQConfig.gsq_training instead selects the staged dense-Llama block-training recipe (initializer gptq or rtn). See GSQ for field tables, per-module modules/dynamic selection, and staged requirements.

Pitfalls and edge cases

Symptom or assumption	What to check
“v2=True enables the current algorithm”	No such GPTQConfig field exists; use nested GPTAQConfig for GPTAQ and FORMAT.GPTQ_V2 for checkpoint zeros
“desc_act=False means natural order”	GPTQ currently resolves act_group_aware to True; explicitly disable both for natural order
Ordering configuration error	Do not explicitly enable both classic and group-aware ordering
Static groups with group_size=-1	Static precomputation expects a positive group step; use a positive group size or disable static groups
Full-row GPTQ with default ordering fails at task construction	group_size=-1 also requires act_group_aware=False; disabling static groups alone is insufficient
GPTAQ does not honor a plain-GPTQ optimization/recovery setting	Its overridden recurrence and solve do not implement group-aware ordering, chunked Hessian accumulation, shared-factor reuse, fused block updates, or the ordinary low-sample/NaN fallback policies
Cholesky failure or unexpectedly large damping	Check calibration coverage, dead channels, selected retry damping, and fallback logs; a bigger damping value is not a proof of better accuracy
Some MoE experts have little traffic	Ordinary GPTQ's default insufficient-evidence fallback can select RTN per module; inspect sample counts and dynamic fallback settings; GPTAQ does not run that early fallback
Hessian staging is bounded but memory still grows	The dense square Hessian, inverse factor, weights, native-reference caches, and other workspace still consume memory
A run completes with NaN-related recovery warnings	GPTQ can retry with mock quantization when fallback is configured; inspect logs rather than accepting completion as a quality result
Comparing GPTQ and RTN avg_loss directly	RTN reports mean absolute weight error as a string; GPTQ reports a sample-normalized Hessian reconstruction metric
Assuming mock_quantization=True is a fast accuracy mode	It uses simplified computation for validation, not the ordinary second-order solve
CUDA extension unavailable	Check compiler/build diagnostics or use the eager path; do not confuse it with the inference backend
Loading an asymmetric legacy checkpoint fails	Check original producer/version metadata and zero-point convention; do not hand-edit packed zeros
Good kernel parity but poor benchmark quality	Check full decoder coverage, calibration/evaluation disjointness, prompts/tokenization, and paired downstream measurements
W4A4 label interpreted as four-bit attention/KV/residual math everywhere	Only selected GEMM operands use the encoded formats; wider internal operators and residual references remain

Source files

Links below point to the documented revision so signatures and defaults can be audited independently of later changes.

Source	Responsibilities
gptqmodel/quantization/gptq.py	GPTQ tasks, Hessian lifecycle, inverse recovery, ordering, block loop, fallbacks, compatibility wrappers
gptqmodel/quantization/quantizer.py	Ranges, affine rounding, dead-column helpers, vectorized MSE grid
gptqmodel/quantization/gptaq.py	Native-reference cross term and activation-aware solve
gptqmodel/quantization/rtn.py	Weight-only rounding, smoothing, optional MLX dispatch
gptqmodel/quantization/fallback_smooth.py	Shared smoothing and ternary MSE range search used by RTN/fallbacks
gptqmodel/utils/fallback.py	Absolute/percentage threshold resolution and strict low-sample fallback comparison
gptqmodel/quantization/gptq_cuda.py	Optional native block-update loader
gptqmodel/utils/env.py	Boolean environment-flag parsing, including the CUDA block-update opt-out
gptqmodel_ext/gptq_block	C++ registration and CUDA within-block kernel
gptqmodel/quantization/config.py	Method-specific dataclasses, defaults, normalization, enums, serialization
gptqmodel/looper/gptq_processor.py	Task selection, dynamic config, hooks, shared inputs, quantization, packing
gptqmodel/looper/native_processor.py	GPTAQ/FOEM native input capture and placement
gptqmodel/looper/module_looper.py	Sequential subset replay and execution coordination
gptqmodel/models/base.py	Public quantization lifecycle, calibration requirements, processor construction
gptqmodel/models/loader.py and gptqmodel/utils/model.py	Checkpoint compatibility and packed-zero conversion
gptqmodel/quantization/protocol.py	Implemented protocol parser and restricted config compiler
docs/quantization_protocol.md	Broader protocol design, matching/override model, algorithm/export distinction
docs/gptq-w4a-gb10.md	W4A usage, transport, requirements, quality gates, memory-safe validation
docs/gptq-w4a-accuracy-notes.md	Recorded experiments, failed candidates, same-weight comparisons, dated full-model results
gptqmodel/nn_modules/qlinear/w4a_floatx.py and w4a_nvfp4.py	Actual FP8/NVFP4 backend support declarations and hardware validation
pyproject.toml and requirements.txt	Python and dependency requirements at this revision

See also

• What Is Quantization
• Installation and Quickstart
• QuantizeConfig Reference
• Quantization Lifecycle
• Looper Processors
• Packing Formats and Compatibility
• Kernel Backends Reference
• Loading and Inference
• Memory Offload and Multi-GPU
• FP8 and NVFP4
• MLX Apple Silicon
• AWQ
• GSQ
• Evaluation
• Troubleshooting and FAQ
• Developer Guide