Skip to content

v3.0.1 - SDXL ConvRot INT8 / NVFP4 Checkpoint Support

Choose a tag to compare

View all tags
@ussoewwin ussoewwin released this 13 Aug 13:34
· 16 commits to main since this release
v3.0.1
2b4b237

This release adds support for loading SDXL ConvRot INT8 and SDXL ConvRot NVFP4 checkpoints (ComfyUI-native comfy_quant format) directly in A1111. Quantized weights are offline-dequantized and Hadamard-unrotated at load time, then flow through the standard float16 pipeline — with complete isolation from all other model formats.

✨ What's New

1. SDXL ConvRot INT8 Checkpoint Loading

  • New module modules/sdxl_int8_convrot.py + a fully isolated hook in modules/sd_models.py (96a80ed7)
  • Detects checkpoints via .comfy_quant keys (format int8_tensorwise, optional "convrot":true)
  • Supports all three INT8 scale layouts:
    • Plain INT8 — scalar weight_scale (tensorwise)
    • ConvRot Linear — row-wise weight_scale [out, 1], unrotated via W = W_rot @ H
    • ConvRot Conv2d — channel-wise weight_scale [out, 1, 1, 1], unrotated along in_channels
  • Sidecar keys (.weight_scale, .comfy_quant, .input_scale, .weight_scale_2, .pre_quant_scale) are fully cleaned up after dequantization

2. SDXL ConvRot NVFP4 Checkpoint Loading

  • NVFP4 (E2M1 packed) Linear layers dequantized via comfy_kitchen.dequantize_nvfp4 (27891d83)
  • Handles the mixed-pack layout: Linear = NVFP4, Conv2d = INT8 tensorwise, CLIP/VAE = float
  • Padded → logical shape crop via orig_shape from comfy_quant metadata
  • ConvRot Hadamard unrotate applied for "convrot":true layers
  • Respects hi_first nibble ordering (default true)

3. Unified Dequantize Pipeline

  • dequantize_state_dict() runs NVFP4 pass first, then INT8 pass — both in-place, before checkpoint caching and load_state_dict
  • Cache stores post-dequant float weights, so reloads skip re-processing
  • Console log reports per-format layer counts (int8_plain, int8_convrot_linear, int8_convrot_conv2d, nvfp4_plain, nvfp4_convrot, etc.)

4. Docs

  • Full technical documentation: docs/SDXL_ConvRot_INT8_NVFP4_Technical_Documentation.md (07fa0b3f)

🔒 Isolation Guarantee (Design Requirement)

  • Strict no-op for any checkpoint without .comfy_quant keys
  • Format-specific branching: only int8_tensorwise → INT8 path, only nvfp4 → NVFP4 path
  • Zero monkey-patches, zero custom layer injection, zero forward-path changes
  • After loading, the model is an ordinary float16 torch.nn.Module — LoRA, ControlNet, VAE, embeddings all work exactly as usual
  • SD1.5 / SD2 / SDXL (non-quantized) / fp16 / fp8 / bnb checkpoints are completely unaffected

🐛 Bug Fixes

  • Stats key collision between NVFP4 and INT8 passes in the combined stats dict — NVFP4 counter renamed to nvfp4_skipped (181e3783)

🧪 Testing

  • INT8 unit tests (6): detection, plain dequantize (max diff < 0.018), ConvRot Linear (< 0.040), ConvRot Conv2d (< 0.031), float-weight skip, no-op on plain state_dict
  • NVFP4 unit tests (4): plain dequantize (< 0.68), ConvRot dequantize (< 0.42), mixed-pack NVFP4+INT8, no-op on plain state_dict
  • Verified against the A1111 load pipeline: find_checkpoint_config, set_model_type, instantiate_from_config, LoadStateDictOnMeta, sd_disable_initialization (int8 Parameter error resolved), checkpoint cache, reload_model_weights

📁 Files Changed

File Status Description
modules/sdxl_int8_convrot.py New Hadamard build/unrotate, INT8 + NVFP4 dequantize, combined entry point
modules/sd_models.py Modified Isolated dequantize hook in load_model_weights()
docs/SDXL_ConvRot_INT8_NVFP4_Technical_Documentation.md New Full technical reference
md/CHANGELOG.md Modified v3.0.1 release entry

Commits

96a80ed7 feat: add SDXL INT8 ConvRot checkpoint support
27891d83 feat: add SDXL NVFP4 ConvRot checkpoint support
181e3783 fix: stats key collision between NVFP4 and INT8 skipped counters
07fa0b3f docs: add SDXL ConvRot INT8/NVFP4 technical documentation
4b4e1914 chore: add v3.0.1 changelog entry (SDXL ConvRot INT8/NVFP4 support)

⚠️ Notes

  • VRAM: weights are dequantized to float16 at load time, so VRAM usage equals a standard float16 model (no runtime quantization savings — the INT8/NVFP4 file-size benefit is preserved on disk)
  • Dependency: NVFP4 requires comfy_kitchen in the A1111 venv (already present in this fork)
  • Inference speed: identical to standard float16 models
  • Target: Python 3.14 fork (A1111-for-Python3.14)
Assets 2
Loading