Skip to content

Dtypes and Formats

Qubitium edited this page Nov 17, 2026 · 3 revisions

🧱 Dtypes and Formats

🏠 Home · 🧭 Choose a method · 🚀 Quickstart · 🧰 Help

On this page

Understand what a checkpoint actually stores before choosing how to run it. This guide is for readers comparing “four-bit” models, estimating storage, or diagnosing a format/backend mismatch.

Three reading depths: choose and understand → inspect and use → number representations, math, and advanced details.

Choose this guide when… Consider another path when…
A model name says INT4, FP4, NF4, GGUF, or NF12 and you need to know what that means You want to pick an optimizer: use Choosing a Method and Dtype
You are estimating payload versus total memory You need exact tensor names/layouts: use Packing Formats and Compatibility
You are checking whether two checkpoints are interchangeable You already know the layout and need a loader: use Loading and Inference

🧭 Choose and understand

Five questions hidden in one model label

Question Example Why it matters
What was the source dtype? BF16 source weights, or prequantized FP8/FP4 source tensors Converting a source to BF16 can lose information before a “lossless BF16” codec runs
What numeric representation is stored? Uniform INT4, E2M1, NF4 indices, vector-codebook indices The same four payload bits can decode to different numbers
Which method produced it? GPTQ, AWQ, MACaQ, VPTQ Different optimizers may use similar bit widths but different side state and constraints
Which checkpoint layout/container carries it? gptq_v2 packed words in safetensors; a standalone .gguf file Layout defines tensor organization; the file container is a different layer
What does the backend compute with? Packed weights, dynamic INT8 activations, higher-precision accumulation/output Storage dtype does not determine activation, accumulator, or output dtype

W4A16 describes nominal weight/activation precision, not a complete format specification. An INT32 qweight tensor can contain many four-bit codes; its Torch dtype is a packing word, not “32-bit weights.” A kernel may temporarily decode a bounded tile without changing the checkpoint's stored representation.

Keep three dtypes separate

  • Source dtype: the weights before the target conversion. Automatic NF12 source storage can reduce calibration-source storage while restoring original eligible BF16 bits before quantization.
  • Stored dtype/encoding: packed codes plus scales, zero points, codebooks, exceptions, transforms, or adapter state. A config's bits field generally describes the selected family—not complete file bits per weight.
  • Activation/compute dtype: what flows into and through the kernel. FP8 weights need not mean FP8 activations; QQQ dynamically quantizes activations to INT8; MACaQ can use w4a4 or w4a16. Accumulator/output rounding remains a separate runtime contract.

See FP8/NVFP4, QQQ, MACaQ, and NF12 source storage for concrete examples.

Representation field guide

Family Main payload and important side state Meaningful distinction / follow-up
GPTQ / GPTAQ / RTN scalar integers Integer codes, scales, zero-point semantics, optional group indices GPTAQ and RTN are solver/lifecycle choices, not independent number systems; GPTQ
AWQ Scalar integer codes with AWQ packing/scale conventions Activation-aware solve is distinct from GEMM/GEMV serialization; AWQ
QQQ INT4 weights, grouped/channel scale hierarchy; dynamic INT8 activations at runtime Not ordinary GPTQ INT4 with a renamed backend; QQQ
MACaQ MXFP4 E2M1 codes, one E8M0 scale per 32 values, plus enabled transform state mxfp4_macaq with transforms; mxfp4 when both are disabled. Retain method="macaq"; see MACaQ.
EXL3 Trellis-coded integers with per-module widths, scales and transform/layout state Model-average target width differs from module width and physical storage; EXL3
VPTQ Indices into vector centroids, optional residual centroids, scaling/state Index bits amortized over vector length omit codebook and metadata cost; VPTQ
Paro Four-bit symmetric packed weights plus learned rotation/scale side tensors Side tensors are required to reproduce execution; Paro
FP8 E4M3/E5M2 floating-point weight bytes with row/block/tensor scales Exponent/mantissa semantics and scale granularity differ; not a blanket FP8 matmul guarantee; FP8
NVFP4 E2M1 values with NVFP4's block/global scaling contract Native FP4 source import differs from GPTQ W4A-NVFP4, which stores GPTQ INT4 weights and quantizes activations; NVFP4
NF12 Twelve main bits/value plus variable exponent escapes and metadata Exact eligible BF16 recovery, not guaranteed fixed 12-bit lossy quantization; NF12
Tenstorrent block floats Shared exponent per 16 values plus signed mantissa codes bf8_b, bf4_b, bf2_b are not IEEE FP8/FP4/FP2; model and standalone-helper scopes differ; Tenstorrent
GGUF / GGML families Q, K, IQ, TQ and supported MXFP4/NVFP4 block layouts; each defines scales/subblocks/codebooks GGUF is a container/family label, not one dtype; GGUF lists supported aliases and import/export boundaries
bitsandbytes NF4 / FP4 / INT8 Nonuniform NF4 or FP4 code indices with block maxima/nested stats; rowwise INT8 with scales NF4 is neither NF12 nor MXFP4/NVFP4; BNB state must travel with the codes; BitsAndBytes

Nominal payload is not checkpoint size

For $P$ encoded weights with $b$ payload bits per weight, the payload alone is $Pb/8$ bytes. Real storage adds scales, zeros, indices, codebooks, exceptions, alignment, non-quantized tensors, configuration and container overhead. Runtime memory adds activations, KV cache, workspaces and possibly adapters.

Example Payload plus local block metadata What this estimate omits
Ideal INT4, group size 128, one FP16 scale/group $4+16/128=4.125$ bits/weight Zero points, group indices, padding and non-quantized tensors; not a complete GPTQ file-size claim
MXFP4 E2M1 + E8M0, blocks of 32 $4+8/32=4.25$ bits/weight MACaQ transforms, biases, alignment and other tensors
GGML Q4_0, blocks of 32 $(32\cdot4+16)/32=4.5$ bits/weight File/container metadata, unquantized tensors, alignment
NF12 Main stream is 12 bits/value; escapes add variable storage Distribution-dependent exceptions/metadata; total can erase the nominal saving
Vector codebook with $C$ centroids of length $v$ Index contribution $\log_2(C)/v$ bits/weight when $C$ is a power of two Centroid storage, residual indices/codebooks, scales, alignment

These are explanatory arithmetic examples, not measured model sizes or speed predictions. Compare actual saved artifacts and runtime measurements under the same workload.

🚀 Use it

Inspect a checkpoint before loading its weights

Prerequisite: a local GPT-QModel checkpoint directory. This standard-library-only example reads metadata, not tensor payloads; missing fields are not a reason to guess a dtype. Embedded or older metadata may live in config.json, so use the loader and compatibility reference for authoritative normalization.

import json
from pathlib import Path

checkpoint = Path("/path/to/checkpoint")
metadata_path = checkpoint / "quantize_config.json"
metadata = json.loads(metadata_path.read_text(encoding="utf-8"))
for key in ("method", "quant_method", "format", "checkpoint_format",
            "bits", "group_size", "sym", "desc_act", "pack_dtype"):
    if key in metadata:
        print(f"{key}: {metadata[key]}")

Then match method + format + bits/subtype + scale/group geometry + model + backend + device. Preserve every shard, index, config and side tensor. Renaming a file or changing a JSON string does not convert a layout.

Use a dedicated config instead of guessing from “4-bit”

These are alternative configuration examples, not a request to load three models at once:

from gptqmodel.quantization import BitsAndBytesConfig, FP8Config, NF12Config

nf4_weights = BitsAndBytesConfig(bits=4, format="nf4")
fp8_weights = FP8Config(format="e4m3", weight_scale_method="row")
lossless_bf16_storage = NF12Config()

All three use calibration-free weight-only lifecycles, but the outputs have different representations and loader contracts. Follow BitsAndBytes, FP8, or NF12 for complete load → convert → save → reload examples and prerequisites. For calibrated recipes use Choosing a Method and Dtype.

Checkpoint format versus runtime backend

METHOD, FORMAT, and BACKEND are separate registries. The pinned public FORMAT values include:

  • GPTQ-family layouts: gptq, gptq_v2, gptq_p; specialized marlin and bitblas layouts also exist.
  • AWQ-family layouts: gemm, gemv, gemv_fast, llm-awq.
  • Other families: qqq, exl3, vptq, paro, fp8, bitsandbytes, nf12, tt_blockfloat, gguf, mxfp4_macaq.

The same spelling can appear in more than one registry without making the concepts identical. Some families have subtype fields rather than one FORMAT member per numeric encoding. For example, selecting NF4 uses the bitsandbytes config subtype, while FORMAT.BITSANDBYTES identifies the checkpoint family.

GPT-QModel defaults to safetensors checkpoints, including non-quantized tensors. A standalone .gguf file is a different container: the public GGUF path supports eligible imports and GPT-QModel directory outputs, not a general standalone .gguf writer. See GGUF and the public serialization rule.

🧠 Go deeper

Scalar integers and floating code grids

For an affine integer group with positive scale $s$, zero point $z$, and $b$-bit code $q$, a common convention is

$$ q=\operatorname{clip}\left(\operatorname{round}(w/s)+z,0,2^b-1\right), \qquad \widehat w=s(q-z). $$

The exact signed convention, zero storage, packing order and group-axis mapping are format-specific. GPTQ v1/v2 zero semantics, for instance, must be handled by the proper reader, not inferred from this generic equation. GPTQ and Packing Formats give the actual contracts.

Normal binary floating-point values have sign, exponent and fraction: schematically $(-1)^s2^{e-\beta}(1+f/2^m)$, with exponent bias $\beta$ and $m$ fraction bits. Subnormal, zero, infinity and NaN rules depend on the exact format; this normal-value equation is not their decoder. E4M3 and E5M2 trade precision and exponent range differently. E2M1 FP4 grids need a scale contract too. BNB FP4, OCP MXFP4 and NVFP4 are not interchangeable merely because each has a four-bit payload.

For block floating point, the exponent is shared rather than stored independently for every value. Tenstorrent's vendor layout and rounding differ from IEEE-like FP8 and from MXFP4; use its defined representation and CPU example, not a guessed FP4 decoder.

Vector quantization and exact compression

VPTQ reconstructs a vector from a centroid selected by an index, optionally adding a residual centroid. This changes the unit of quantization from a scalar to a vector, so an index-width label alone cannot predict total memory. The VPTQ math defines shapes, weighting and residual state.

NF12 instead preserves eligible BF16 bits through a compact exponent window plus escaped groups. It does not solve the scalar integer objective above. Compression depends on the exponent distribution; the NF12 specification defines the actual tiled layout. Exact restored weights do not guarantee identical outputs across kernels with different accumulation/output rounding.

Advanced controls and common traps

  • Refinement is not a dtype: GSQ refines/allocates supported representations; EoRA is low-rank compensation with additional state.
  • Preprocessing is not a format: rotations/smoothing/norm tuning must be admitted by the selected lifecycle and preserved where execution needs them.
  • Backend is not hardware proof: use explicit NVIDIA CUDA and AMD ROCm declarations plus architecture gates. Triton availability alone establishes neither.
  • Fusion is not recompression: Projection Fusion changes eligible execution while preserving native packed-layout contracts.
  • Source compression is not output selection: auto_dtype_cast can affect calibration-source storage without changing the final target format.
  • A valid config is not a quality result: evaluate the reloaded model and report actual conditions.

Public source and next steps

Scoped to source 13ee428f2. Registry/config definitions: config.py. Float source dtype/decode helpers: dtype.py. Runtime registry: backend.py.

Next: choose a method, inspect packing compatibility, or load a checkpoint.

Clone this wiki locally