Skip to content
Qubitium edited this page Nov 21, 2026 · 2 revisions

Computed Permutation Trellis Coding (CPTC)

🏠 Home · 🧭 Choose a method · 🚀 Quickstart · 🧰 Help

On this page

CPTC is a calibration-free, lossy weight quantization method. It encodes blocks of weights with a shared codebook at integer rates from 1 to 6 bits per weight. Use it when you have a dense source model and want to evaluate low-bit compression without preparing a calibration corpus. The native GPU path supports NVIDIA SM80+ and AMD RDNA4, and the same checkpoint can run on CPU through PyTorch.

This page follows source revision 2e7992a18. For the algorithm and research measurements, see the CPTC paper and implementation notes.

Quantize, save, and reload

from gptqmodel import BACKEND, CPTCConfig, GPTQModel

cfg = CPTCConfig(bits=4, device="cuda:0")
model = GPTQModel.load("path/to/dense-model", quantize_config=cfg)
model.quantize()
model.save("path/to/cptc-checkpoint")

reloaded = GPTQModel.load(
    "path/to/cptc-checkpoint",
    device="cuda:0",
    backend=BACKEND.CPTC,
)

PyTorch uses the cuda:0 device spelling for both NVIDIA CUDA and ROCm. Set device="cpu" in the config to encode on CPU, or in the reload call to run inference on CPU. A smaller trellis_depth reduces the number of states the encoder searches. For the complete field constraints and per-layer bit overrides, see CPTCConfig.

Importing gptqmodel registers the Transformers loader for existing CPTC safetensors checkpoints. Use GPTQModel.quantize() to create the checkpoint.

import gptqmodel
import torch
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "path/to/cptc-checkpoint",
    device_map="cpu",
    dtype=torch.bfloat16,
)

Backend selection

BACKEND.AUTO selects BACKEND.CPTC for this format. The CPTC backend chooses its execution path on the actual device ordinal. Activations and outputs use FP16, BF16, or FP32. Choose the dtype before the first forward and keep it fixed for that module.

Path When it runs
Fused CUDA MMA or ROCm WMMA Resident packed weights and 1–16 flattened FP16 or BF16 activation rows without input-gradient tracking. Requires the native extension on NVIDIA SM80+ or ROCm gfx1200/gfx1201.
Native tiled decode and FP32 matmul Other input shapes and dtypes on the same GPUs when the extension is available, including larger prefills and FP32 inputs.
Torch tiled decode and FP32 matmul CPU, other CUDA or ROCm architectures, or a native extension build failure.

Both tiled routes decode temporary FP32 weight tiles. The fused route applies block scales in FP32. All routes add bias to the FP32 reduction before casting the output to the input dtype. The native encoder and inference operators build on first use, and a build failure logs a warning before falling back to Torch. The declared execution level is eager, with no graph capture support. Backend implementation · Native extension dispatch

On a host with a supported GPU, call gptqmodel.extension.load("cptc") to build or load the native operators explicitly and surface build errors. Extension API

Encoding and checkpoint layout

The encoder splits each output row into contiguous input blocks and pads the final block with zeros. It scales each block and uses a two-pass Viterbi search to choose a bitstream with overlapping states. The last states wrap around to the start of the same block. During decode, two multiplication constants map states into a small E2M5 codebook, and an FP32 scale reconstructs the weight range for each block.

The checkpoint uses METHOD.CPTC and FORMAT.CPTC, both serialized as "cptc". Each packed projection preserves the following tensors.

Tensor Dtype and shape
cptc_packed uint8, [out_features, blocks, block_size * bits / 8]
cptc_table uint8, 2**table_q E2M5 codes
cptc_coeffs int32, two 16-bit multiplication constants
cptc_scale_bits int32, [out_features, blocks], containing IEEE FP32 scale bit patterns
bias Optional, in the original bias dtype

The scale adds 0.25 bits per weight for a full 128-value block or 0.125 for a full 256-value block. End padding, per-layer tables, coefficients, bias, and unquantized tensors contribute to total checkpoint size. Codec implementation

Clone this wiki locally