Skip to content
Qubitium edited this page Nov 21, 2026 · 2 revisions

Computed Permutation Trellis Coding (CPTC)

🏠 Home · 🧭 Choose a method · 🚀 Quickstart · 🧰 Help

On this page

CPTC is a lossy weight quantization method. It encodes blocks of weights with a shared codebook at integer rates from 1 to 6 bits per weight. Direct packing needs no calibration data. You can opt into two-sided YAQA calibration when you have representative samples and room for a dense teacher pass. Both checkpoint layouts run on NVIDIA SM80+ or AMD RDNA4 with native kernels, or on CPU through PyTorch.

This page follows source revision d4712ee13. For the algorithm and research measurements, see the CPTC paper and implementation notes.

Quantize, save, and reload

from gptqmodel import BACKEND, CPTCConfig, GPTQModel

cfg = CPTCConfig(bits=4, device="cuda:0")
model = GPTQModel.load("path/to/dense-model", quantize_config=cfg)
model.quantize()
model.save("path/to/cptc-checkpoint")

reloaded = GPTQModel.load(
    "path/to/cptc-checkpoint",
    device="cuda:0",
    backend=BACKEND.CPTC,
)

PyTorch uses the cuda:0 device spelling for both NVIDIA CUDA and ROCm. Set device="cpu" in the config to encode on CPU, or in the reload call to run inference on CPU. A smaller trellis_depth reduces the number of states the encoder searches. For the complete field constraints and per-layer bit overrides, see CPTCConfig.

Importing gptqmodel registers the Transformers loader for existing CPTC safetensors checkpoints. Use GPTQModel.quantize() to create the checkpoint.

import gptqmodel
import torch
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "path/to/cptc-checkpoint",
    device_map="cpu",
    dtype=torch.bfloat16,
)

Optional YAQA calibration

from gptqmodel import CPTCConfig, CalibrationConfig, GPTQModel

cfg = CPTCConfig(bits=3, block_size=256, yaqa=True, yaqa_seed=2718, device="cuda:0")
model = GPTQModel.load("path/to/dense-model", quantize_config=cfg)
model.quantize(CalibrationConfig(dataset=calibration_samples))
model.save("path/to/calibrated-cptc")

With yaqa=True, supply a nonempty CalibrationConfig.dataset. GPT-QModel runs the dense model on the quantization device and samples target tokens from its own predictions. Backpropagation accumulates two FP32 curvature sketches per selected linear projection, one for the input and one for the output. Budget for the complete dense model and these square sketch matrices on the same device. yaqa_seed defaults to zero and controls both teacher sampling and the per-projection Hadamard signs.

YAQA rotates the weights and sketches with signed Hadamard transforms, then uses two-sided block LDL error feedback to choose targets for the same Viterbi encoder. Tiles are 8 output rows × 16 input columns at block_size=128, or 16 × 16 at block_size=256. Both projection widths must support a Hadamard transform and be divisible by their respective tile dimensions. Dynamic rules can exclude incompatible projections. Reload the checkpoint as above; inference needs no calibration dataset.

Backend selection

BACKEND.AUTO selects BACKEND.CPTC for this format. The CPTC backend chooses its execution path on the actual device ordinal. Activations and outputs use FP16, BF16, or FP32. Choose the dtype before the first forward and keep it fixed for that module.

Path When it runs
Fused CUDA MMA or ROCm WMMA Resident packed weights and 1–16 flattened FP16 or BF16 activation rows without input-gradient tracking. Requires the native extension on NVIDIA SM80+ or ROCm gfx1200/gfx1201.
Native tiled decode and FP32 matmul Other input shapes and dtypes on the same GPUs when the extension is available, including larger prefills and FP32 inputs.
Torch tiled decode and FP32 matmul CPU, other CUDA or ROCm architectures, or a native extension build failure.

Both tiled routes decode temporary FP32 weight tiles. The fused route applies block scales in FP32. All routes add bias to the FP32 reduction before casting the output to the input dtype. The native encoder and inference operators build on first use, and a build failure logs a warning before falling back to Torch. The declared execution level is eager, with no graph capture support. Backend implementation · Native extension dispatch

On a host with a supported GPU, call gptqmodel.extension.load("cptc") to build or load the native operators explicitly and surface build errors. Extension API

Encoding and checkpoint layout

Direct packing splits each output row into contiguous input blocks and pads the final block with zeros. Both modes scale each block or tile and use a two-pass Viterbi search to choose a bitstream with overlapping states. The last states wrap around to the start of the same block. During decode, two multiplication constants map states into a small E2M5 codebook, and an FP32 scale reconstructs the weight range for each block or tile.

The checkpoint uses METHOD.CPTC and FORMAT.CPTC, both serialized as "cptc". Each packed projection preserves the following tensors.

Tensor Dtype and shape
cptc_packed uint8, [out_features, blocks, block_bytes] for direct packing or [row_tiles, column_tiles, block_bytes] for YAQA, with block_bytes = block_size * bits / 8
cptc_table uint8, 2**table_q E2M5 codes
cptc_coeffs int32, two 16-bit multiplication constants
cptc_scale_bits int32 IEEE FP32 scale bit patterns, with the same first two dimensions as cptc_packed
cptc_input_signs / cptc_output_signs YAQA only: int8 vectors of lengths in_features and out_features
bias Optional, in the original bias dtype

For YAQA, the quantization config records yaqa and yaqa_seed. Inference transforms the input before multiplying by the rotated weights and inverts the output transform before adding bias and casting to the activation dtype.

The scale adds 0.25 bits per weight for a full 128-value block or 0.125 for a full 256-value block. End padding for direct packing, sign vectors for YAQA, per-layer tables, coefficients, bias, and unquantized tensors contribute to total checkpoint size. Codec implementation

Clone this wiki locally