Repository navigation
CPTC
🏠 Home · 🧭 Choose a method · 🚀 Quickstart · 🧰 Help
On this page
CPTC is a lossy weight quantization method. It encodes blocks of weights with a shared codebook at integer rates from 1 to 6 bits per weight. Direct packing needs no calibration data. You can opt into two-sided YAQA calibration when you have representative samples and room for a dense teacher pass. Both checkpoint layouts run on NVIDIA SM80+ or AMD RDNA4 with native kernels, or on CPU through PyTorch.
This page follows source revision d4712ee13. For the algorithm and research measurements, see the CPTC paper and implementation notes.
from gptqmodel import BACKEND, CPTCConfig, GPTQModel
cfg = CPTCConfig(bits=4, device="cuda:0")
model = GPTQModel.load("path/to/dense-model", quantize_config=cfg)
model.quantize()
model.save("path/to/cptc-checkpoint")
reloaded = GPTQModel.load(
"path/to/cptc-checkpoint",
device="cuda:0",
backend=BACKEND.CPTC,
)
PyTorch uses the cuda:0 device spelling for both NVIDIA CUDA and ROCm. Set device="cpu" in the config to encode on CPU, or in the reload call to run inference on CPU. A smaller trellis_depth reduces the number of states the encoder searches. For the complete field constraints and per-layer bit overrides, see CPTCConfig.
Importing gptqmodel registers the Transformers loader for existing CPTC safetensors checkpoints. Use GPTQModel.quantize() to create the checkpoint.
import gptqmodel
import torch
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"path/to/cptc-checkpoint",
device_map="cpu",
dtype=torch.bfloat16,
)
from gptqmodel import CPTCConfig, CalibrationConfig, GPTQModel
cfg = CPTCConfig(bits=3, block_size=256, yaqa=True, yaqa_seed=2718, device="cuda:0")
model = GPTQModel.load("path/to/dense-model", quantize_config=cfg)
model.quantize(CalibrationConfig(dataset=calibration_samples))
model.save("path/to/calibrated-cptc")
With yaqa=True, supply a nonempty CalibrationConfig.dataset. GPT-QModel runs the dense model on the quantization device and samples target tokens from its own predictions. Backpropagation accumulates two FP32 curvature sketches per selected linear projection, one for the input and one for the output. Budget for the complete dense model and these square sketch matrices on the same device. yaqa_seed defaults to zero and controls both teacher sampling and the per-projection Hadamard signs.
YAQA rotates the weights and sketches with signed Hadamard transforms, then uses two-sided block LDL error feedback to choose targets for the same Viterbi encoder. Tiles are 8 output rows × 16 input columns at block_size=128, or 16 × 16 at block_size=256. Both projection widths must support a Hadamard transform and be divisible by their respective tile dimensions. Dynamic rules can exclude incompatible projections. Reload the checkpoint as above; inference needs no calibration dataset.
BACKEND.AUTO selects BACKEND.CPTC for this format. The CPTC backend chooses its execution path on the actual device ordinal. Activations and outputs use FP16, BF16, or FP32. Choose the dtype before the first forward and keep it fixed for that module.
| Path | When it runs |
|---|---|
| Fused CUDA MMA or ROCm WMMA | Resident packed weights and 1–16 flattened FP16 or BF16 activation rows without input-gradient tracking. Requires the native extension on NVIDIA SM80+ or ROCm gfx1200/gfx1201. |
| Native tiled decode and FP32 matmul | Other input shapes and dtypes on the same GPUs when the extension is available, including larger prefills and FP32 inputs. |
| Torch tiled decode and FP32 matmul | CPU, other CUDA or ROCm architectures, or a native extension build failure. |
Both tiled routes decode temporary FP32 weight tiles. The fused route applies block scales in FP32. All routes add bias to the FP32 reduction before casting the output to the input dtype. The native encoder and inference operators build on first use, and a build failure logs a warning before falling back to Torch. The declared execution level is eager, with no graph capture support. Backend implementation · Native extension dispatch
On a host with a supported GPU, call gptqmodel.extension.load("cptc") to build or load the native operators explicitly and surface build errors. Extension API
Direct packing splits each output row into contiguous input blocks and pads the final block with zeros. Both modes scale each block or tile and use a two-pass Viterbi search to choose a bitstream with overlapping states. The last states wrap around to the start of the same block. During decode, two multiplication constants map states into a small E2M5 codebook, and an FP32 scale reconstructs the weight range for each block or tile.
The checkpoint uses METHOD.CPTC and FORMAT.CPTC, both serialized as "cptc". Each packed projection preserves the following tensors.
| Tensor | Dtype and shape |
|---|---|
cptc_packed |
uint8, [out_features, blocks, block_bytes] for direct packing or [row_tiles, column_tiles, block_bytes] for YAQA, with block_bytes = block_size * bits / 8
|
cptc_table |
uint8, 2**table_q E2M5 codes |
cptc_coeffs |
int32, two 16-bit multiplication constants |
cptc_scale_bits |
int32 IEEE FP32 scale bit patterns, with the same first two dimensions as cptc_packed
|
cptc_input_signs / cptc_output_signs
|
YAQA only: int8 vectors of lengths in_features and out_features
|
bias |
Optional, in the original bias dtype |
For YAQA, the quantization config records yaqa and yaqa_seed. Inference transforms the input before multiplying by the rotated weights and inverts the output transform before adding bias and casting to the activation dtype.
The scale adds 0.25 bits per weight for a full 128-value block or 0.125 for a full 256-value block. End padding for direct packing, sign vectors for YAQA, per-layer tables, coefficients, bias, and unquantized tensors contribute to total checkpoint size. Codec implementation
Documentation home · Quickstart · Troubleshooting · Repository · PyPI
Public source d4712ee13 declares package version 7.12.4. Individual pages identify their source revisions. Choose a method · Dtypes and formats. Maintained by ModelCloud.ai.
🧭 1 · Choose and understand
🚀 2 · Use it
- Installation and Quickstart
- Loading and Inference
- Evaluation
- Serving
- Troubleshooting and FAQ
- Memory, Offload, Multi-GPU
- MLX / Apple Silicon
- Tenstorrent / Block Floats
- Hugging Face Integration
- Multimodal and MoE Models
🎛️ Methods and representations
Each guide: choose → examples → math / advanced
- GPTQ / GPTAQ / RTN
- AWQ
- QQQ
- MACaQ / MXFP4
- EXL3
- CPTC
- VPTQ
- Paro
- FP8 and NVFP4
- NF12: lossless BF16 storage
- GGUF families
- BitsAndBytes: NF4 / FP4 / INT8
🧰 Refinement and execution
🧠 3 · Theory and advanced reference