Repository navigation
CPTC
🏠 Home · 🧭 Choose a method · 🚀 Quickstart · 🧰 Help
CPTC is a calibration-free, lossy weight quantization method. It encodes blocks of weights with a shared codebook at integer rates from 1 to 6 bits per weight. Use it when you have a dense source model and want to evaluate low-bit compression without preparing a calibration corpus. The native GPU path supports NVIDIA SM80+ and AMD RDNA4, and the same checkpoint can run on CPU through PyTorch.
This page follows source revision 2e7992a18. For the algorithm and research measurements, see the CPTC paper and implementation notes.
from gptqmodel import BACKEND, CPTCConfig, GPTQModel
cfg = CPTCConfig(bits=4, device="cuda:0")
model = GPTQModel.load("path/to/dense-model", quantize_config=cfg)
model.quantize()
model.save("path/to/cptc-checkpoint")
reloaded = GPTQModel.load(
"path/to/cptc-checkpoint",
device="cuda:0",
backend=BACKEND.CPTC,
)PyTorch uses the cuda:0 device spelling for both NVIDIA CUDA and ROCm. Set device="cpu" in the config to encode on CPU, or in the reload call to run inference on CPU. A smaller trellis_depth reduces the number of states the encoder searches. For the complete field constraints and per-layer bit overrides, see CPTCConfig.
Importing gptqmodel registers the Transformers loader for existing CPTC safetensors checkpoints. Use GPTQModel.quantize() to create the checkpoint.
import gptqmodel
import torch
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"path/to/cptc-checkpoint",
device_map="cpu",
dtype=torch.bfloat16,
)BACKEND.AUTO selects BACKEND.CPTC for this format. The CPTC backend chooses its execution path on the actual device ordinal. Activations and outputs use FP16, BF16, or FP32. Choose the dtype before the first forward and keep it fixed for that module.
| Path | When it runs |
|---|---|
| Fused CUDA MMA or ROCm WMMA | Resident packed weights and 1–16 flattened FP16 or BF16 activation rows without input-gradient tracking. Requires the native extension on NVIDIA SM80+ or ROCm gfx1200/gfx1201. |
| Native tiled decode and FP32 matmul | Other input shapes and dtypes on the same GPUs when the extension is available, including larger prefills and FP32 inputs. |
| Torch tiled decode and FP32 matmul | CPU, other CUDA or ROCm architectures, or a native extension build failure. |
Both tiled routes decode temporary FP32 weight tiles. The fused route applies block scales in FP32. All routes add bias to the FP32 reduction before casting the output to the input dtype. The native encoder and inference operators build on first use, and a build failure logs a warning before falling back to Torch. The declared execution level is eager, with no graph capture support. Backend implementation · Native extension dispatch
On a host with a supported GPU, call gptqmodel.extension.load("cptc") to build or load the native operators explicitly and surface build errors. Extension API
The encoder splits each output row into contiguous input blocks and pads the final block with zeros. It scales each block and uses a two-pass Viterbi search to choose a bitstream with overlapping states. The last states wrap around to the start of the same block. During decode, two multiplication constants map states into a small E2M5 codebook, and an FP32 scale reconstructs the weight range for each block.
The checkpoint uses METHOD.CPTC and FORMAT.CPTC, both serialized as "cptc". Each packed projection preserves the following tensors.
| Tensor | Dtype and shape |
|---|---|
cptc_packed |
uint8, [out_features, blocks, block_size * bits / 8]
|
cptc_table |
uint8, 2**table_q E2M5 codes |
cptc_coeffs |
int32, two 16-bit multiplication constants |
cptc_scale_bits |
int32, [out_features, blocks], containing IEEE FP32 scale bit patterns |
bias |
Optional, in the original bias dtype |
The scale adds 0.25 bits per weight for a full 128-value block or 0.125 for a full 256-value block. End padding, per-layer tables, coefficients, bias, and unquantized tensors contribute to total checkpoint size. Codec implementation
Documentation home · Quickstart · Troubleshooting · Repository · PyPI
Public source d4712ee13 declares package version 7.12.4. Individual pages identify their source revisions. Choose a method · Dtypes and formats. Maintained by ModelCloud.ai.
🧭 1 · Choose and understand
🚀 2 · Use it
- Installation and Quickstart
- Loading and Inference
- Evaluation
- Serving
- Troubleshooting and FAQ
- Memory, Offload, Multi-GPU
- MLX / Apple Silicon
- Tenstorrent / Block Floats
- Hugging Face Integration
- Multimodal and MoE Models
🎛️ Methods and representations
Each guide: choose → examples → math / advanced
- GPTQ / GPTAQ / RTN
- AWQ
- QQQ
- MACaQ / MXFP4
- EXL3
- CPTC
- VPTQ
- Paro
- FP8 and NVFP4
- NF12: lossless BF16 storage
- GGUF families
- BitsAndBytes: NF4 / FP4 / INT8
🧰 Refinement and execution
🧠 3 · Theory and advanced reference