Skip to content

OrbitQuant 0.2.0

Choose a tag to compare

@iamwavecut iamwavecut released this 10 Jul 11:36

OrbitQuant 0.2.0 extends calibration-free quantization from the paper-specific
Diffusers targets to transformer backbones across modalities.

Highlights

  • Universal structural policy for registered linear-compatible modules, with
    machine-readable coverage inspection before quantization.
  • Built-in adapters for torch.nn.Linear and Hugging Face Conv1D, plus a
    public adapter API for custom weight layouts.
  • Named W2A3, W2A4, W3A3, W4A4, and W4A6 recipes and a direct
    quantize_model() API.
  • Transformers quantize-on-load and packed save_pretrained() /
    from_pretrained() support, including base-model prefixes and transposed
    GPT-2 projections.
  • Shape-generic packed CUDA, Triton, and Metal inference paths for arbitrary
    leading dimensions, short autoregressive decode rows, and partial matrix
    tiles without full weight materialization.
  • Paper-specific FLUX.1, FLUX.2, Z-Image, and Wan policies remain unchanged and
    pass their exact module-inventory methodology gate.

Architecture compatibility does not imply that every low-bit recipe preserves
quality on every model. Inspect coverage and validate the selected bit profile
for the target task before publishing a checkpoint.