Releases · ussoewwin/ComfyUI-HSWQ-Loader-and-Tools

Release list

v3.4.8 - SAM3 Nodes Removed (Stock Loader Support Confirmed) Latest

Latest

ussoewwin released this 29 Aug 21:59

v3.4.8

2bc615d

中文

HSWQ v3.4.8 — SAM3 Nodes Removed (Stock Loader Support Confirmed)

🗑️ Removed

HSWQ SAM3 Loader (ConvRot INT8) and HSWQ SAM3 Detect were removed from the repository tree.

📖 Background (why)

The SAM3 ConvRot INT8 node work was published in v3.4.7. After posting on r/StableDiffusion, a community member pointed out that these checkpoints work with the default Comfy SAM 3.1 node. Re-testing confirmed it: the stock CheckpointLoaderSimple (and the default Comfy SAM3.1 node) load ConvRot INT8 SAM3 checkpoints correctly — the dedicated loader turned out to be unnecessary.

The reason stock loaders work is the HSWQ startup patch (_patch_load_state_dict_guess_config_int8, gated by is_sam3), which automatically:

attaches MixedPrecisionOps (int8_tensorwise) so Linear layers stay true INT8 in VRAM (TensorWiseINT8Layout), and
dequantizes and remaps the CLIP keys so the text encoder loads without the "clip missing" warning.

The error seen during the earlier test was a user-side oversight, not a loader incompatibility.

🔧 What changed

Tree restored to the pre-SAM3 baseline d33862a (191ddbc).
History intentionally retained: all SAM3 work — the nodes, the patches, and the complete technical guide — remains in the git history under v3.4.7 for future reference.
CHANGELOG updated (EN + 中文).

📦 Related

Quantized models: Hugging Face — SAM3.1-ConvRot-INT8 (900 MB native ConvRot INT8)
Technical guide (kept in v3.4.7 history): HSWQ SAM3 ConvRot INT8 — Complete Technical Guide

💡 Technical notes from the experience

Supporting ConvRot INT8 for SAM3 took several days of development; unlike image-generation UNets, the ConvRot rotation handling for ControlNet and CLIP was the difficult part.
The motivation was storage savings as much as VRAM: converting CLIP and ControlNet to ConvRot INT8 freed about 40 GB on its own (the SSD was running out of space).
HSWQ quantization protects critical layers in FP16 and can mix ConvRot INT8 with NVFP4; in that case the size becomes more than 50% of FP16, but precision improves.

Assets 2

v3.4.7 - SAM3 ConvRot INT8 Nodes

ussoewwin released this 29 Aug 20:22

v3.4.7

f781d76

中文

HSWQ v3.4.7 — SAM3 ConvRot INT8 Nodes

This release adds HSWQ SAM3 Loader (ConvRot INT8) and HSWQ SAM3 Detect, letting ComfyUI load and run ConvRot / TensorWise INT8-quantized SAM3 (Segment Anything 3) checkpoints with weights kept in true 8-bit precision in VRAM.

✨ Added

HSWQ SAM3 Loader (ConvRot INT8) (HSWQSAM3Loader / HSWQLoadConvRotINT8SAM3, category loaders)
- Loads SAM3 checkpoints carrying int8_tensorwise comfy_quant metadata via MixedPrecisionOps, keeping Linear layers as QuantizedTensor (TensorWiseINT8Layout) — true INT8 in VRAM (~525 MB for SAM3.1 Multiplex, about half of FP16)
- Searches diffusion_models / sams / detection / checkpoints folders
- Non-INT8 (FP16 etc.) checkpoints load normally with stock settings
HSWQ SAM3 Detect (HSWQSAM3Detect, category HSWQ/Detection)
- Open-vocabulary detection & segmentation with text (CONDITIONING), box (BBOXES), and point prompts (JSON pixel coords)
- Outputs masks (MASK), bboxes (BBOXES), and pass-through image (IMAGE)
- Options: threshold (0.50), refine_iterations (SAM decoder refinement, default 2), individual_masks
comfy_kitchen INT8 GEMM safety fallback (_patch_comfy_kitchen_int8_gemm_fallback)
- Non-multiple-of-4 dimensions (e.g. boxRPB_embed_x K=2) automatically fall back to float precision instead of crashing cuBLAS INT8 GEMM

🐛 Fixed

SAM3 INT8 runtime crash under DynamicVRAM (aimdo) — ValueError: Buffer too small: needs 6291456 bytes, but only has 3164160. in resolve_cast_module_with_vbar (6d4f3f8)
- Root cause: the vbar buffer is allocated with the INT8 payload size (int8 data + scale); after the runtime weight guard replaces weights with FP16, the float16-sized cast geometry no longer fits
- Fix: _strip_dynamic_vram_attrs drops the vbar state (_v / _prefetch / _v_signature / _v_block, after vbar_unpin) so cast_bias_weight falls back to the regular cast path
SAM3 CLIP loading — "clip missing" → noisy salt-and-pepper masks (8c20913)
- Root cause: sd-level and _clip_stash in_proj_weight pre-splits broke ComfyUI's transformers_convert remap (it expects the fused in_proj_weight form); INT8 checkpoints store language_backbone already split into q/k/v, which transformers_convert cannot remap
- Fix: removed both pre-splits; process_clip_state_dict now remaps leftover encoder.* keys to sam3_clip.transformer.text_model.encoder.layers.N.self_attn.q_proj
- Verified: fp16 and INT8 both score ~0.98 on a real image + "person"; masks are clean uniform white (previously ~0.27 / NaN with 1000+ speckle components)

📚 Documentation

HSWQ SAM3 ConvRot INT8 — Complete Technical Guide — overview, files created/modified, full code, per-function explanations (baseline d33862a)
README / Chinese README updated: node usage, example workflow, FP16 compatibility
CHANGELOG v3.4.7 (EN + 中文)

✅ Compatibility

FP16 SAM3 checkpoints fully supported — HSWQ SAM3 Loader falls back to stock loading (no MixedPrecisionOps); HSWQ SAM3 Detect produces equivalent masks on both paths
Works with stock CheckpointLoaderSimple thanks to the CLIP remap patch (no "clip missing")
Compatible with ComfyUI native SAM3 nodes and HSWQ SAM3 Detect

Assets 2

v3.4.6 - SDXL anytest ControlLoRA on ConvRot INT8 / Hybrid ConvRot NVFP4 Bases - Complete Fix Guide (v2)

ussoewwin released this 27 Aug 08:12

v3.4.6

d33862a

中文

1. Symptoms

Two rounds of symptoms were reported for this LoRA-type ControlNet on HSWQ-quantized SDXL bases:

(fixed by c60bb0b) The control had no visible effect at all - no errors, the pipeline ran, but the image ignored the hint entirely.
(fixed by 152c1dc) After c60bb0b the control visibly worked (structure followed the hint) but the output locked onto the lineart: the result stayed black-and-white, no coloring happened, and the strength slider appeared completely dead.

Decisive observation from the user's real workflow (same base, same hint, strength 1.0):

ControlNet file	Type	User's output
`CN-anytest_v4-marged_am_dim256.safetensors`	LoRA-type (`ControlLora`)	near-grayscale (sat ~1.3/255) - lineart lock
`CN-anytest_v4-marged_pn_dim256.safetensors`	LoRA-type (`ControlLora`)	near-grayscale (sat ~1.3/255)
`CN-anytest4_illustrious2_A_convrot_int8.safetensors`	full-type (`ControlNet`)	correctly colored (sat ~59/255)

The failure is therefore specific to the LoRA-type path combined with the checkpoint loader node - not to the control file itself.

2. Root cause

2.1 `HSWQCheckpointLoaderSDXL` ignored `weight_dtype="int8_tensorwise"` (primary cause of symptom 2)

The node's load_checkpoint() called comfy.sd.load_checkpoint_guess_config(...) directly. For int8_tensorwise it passed model_options={} - only the fp8 options ever set a dtype - so nothing applied the INT8 Conv2d load path.

ConvRot INT8 checkpoints (e.g. JANKUTrainedChenkinNoobai_v777_hswq_r32_1off_convrot_int8.safetensors) store their quantized Conv2d layers in the sidecar format:

Key in checkpoint	Content
`X.weight`	raw int8 qdata
`X.weight_scale`	per-tensor scale (e.g. shape `(320,1,1,1)`)
`X.comfy_quant`	JSON: `{"format":"int8_tensorwise","convrot":true,"convrot_groupsize":64}`

Without the INT8 Conv2d load scope (_int8_quant_conv_scope() in patches/comfy_quant_int8.py) those layers are left as raw qdata reinterpreted as fp16 (absmax ~127, std ~30) in the loaded model. Verified on the user's actual base:

base UNet forward alone -> nan=65536 (the base itself is broken through this load path)
ControlLora borrowed weights for those convs are +-127 garbage, so the control output explodes: [731, 123352, 183752, NaN] (vs a sane full-type ControlNet at [325, 616, 944, 1414])

An exploded control signal (even at strength 0.1, 0.1 x 183k is far above a sane ~1k) overrides the model completely: the output becomes a copy of the hint (B&W lineart), the model can no longer add color, and the strength slider has no visible effect. That is exactly symptom 2.

The INT8-aware loader already existed: load_checkpoint_sdxl_hswq_weight_dtype() (same file), which wraps the load in _int8_quant_conv_scope() and applies the INT8 Conv2d decode. The node simply never used it.

Note: this also explains why a repro with plain load_diffusion_model / load_checkpoint_guess_config showed exploding controls on both waiIllustrious and JANKU bases, while the earlier "sane" results were obtained through the dedicated load_unet_hswq_weight_dtype path (which does set the scope).

2.2 ControlLora borrowed-weight dequant (fixed by `c60bb0b`, still required)

Recap of the earlier fix (see commit c60bb0b for full detail): ControlLora.pre_run borrows the base UNet's state_dict() and injects it into a float ControlLoraOps control model.

Bug A: comfy-kitchen's dequantize_int8_convrot_weight_dtype is 2D-only - 4D Conv2d raises NoCapableBackendError, and the old wrapper fell back to raw qdata (+-127).
Bug B: HSWQ-armed Conv2d (_hswq_convrot=True) keeps weights in the rotated basis - qt.dequantize() succeeds but returns W_rot, which is wrong for the float control model (it does not rotate activations).

The v3 wrapper (_dequantized_state_dict) now collects (qt, module) pairs, dequantizes with _manual_qt_dequant (qdata x scale), and un-rotates 4D Conv2d weights when the module is armed (_unrotate_conv2d mirrors native_convert_int8.rotate_weight_conv2d exactly). This is still necessary for any base whose weights are QuantizedTensor-wrapped (the load_unet_hswq_weight_dtype path).

2.3 Hadamard device mismatch (fixed by `152c1dc`)

_unrotate_conv2d built the Hadamard matrix on CPU while the qdata could be on CUDA, causing RuntimeError: Expected all tensors to be on the same device inside ControlLora.pre_run when the sampler invokes it. _regular_hadamard now takes the qdata's device.

3. Files modified

Commit	File	Change
`c60bb0b`	`patches/comfy_quant_int8.py`	`_patch_controllora_int8_dequant` v3: module-aware `_dequantized_state_dict`, `_regular_hadamard` / `_unrotate_conv2d` / `_manual_qt_dequant`, raw-sidecar fallback; `_CL_VER` 2 -> 3
`c60bb0b`	`__init__.py`	Install the ControlLora dequant wrapper unconditionally at startup
`152c1dc`	`__init__.py`	`HSWQCheckpointLoaderSDXL` delegates `int8_tensorwise` (or auto-detected comfy_quant INT8) to `load_checkpoint_sdxl_hswq_weight_dtype`
`152c1dc`	`patches/comfy_quant_int8.py`	`_regular_hadamard(size, device=None)` - device-aware Hadamard

Key code - `init.py` (node fix, `152c1dc`)

def _checkpoint_looks_int8(ckpt_path):
    try:
        from .patches.comfy_quant_int8 import checkpoint_looks_like_comfy_quant_int8
        return bool(checkpoint_looks_like_comfy_quant_int8(ckpt_path))
    except Exception:
        return False

# inside HSWQCheckpointLoaderSDXL.load_checkpoint:
ckpt_path = folder_paths.get_full_path_or_raise("checkpoints", ckpt_name)
# INT8 (incl. ConvRot) checkpoints MUST go through the INT8-aware loader:
# plain load_checkpoint_guess_config leaves Conv2d comfy_quant layers as RAW
# int8 qdata (absmax ~127, scale dropped), which breaks the base forward (NaN)
# and poisons the ControlLora borrowed weights (control output explodes ->
# lineart lock / no coloring / strength dead).
if weight_dtype == "int8_tensorwise" or _checkpoint_looks_int8(ckpt_path):
    from .patches.comfy_quant_int8 import load_checkpoint_sdxl_hswq_weight_dtype
    return load_checkpoint_sdxl_hswq_weight_dtype(ckpt_name, weight_dtype, device=None)

Key code - `patches/comfy_quant_int8.py` (Hadamard device fix, `152c1dc`)

def _regular_hadamard(size, device=None):
    h4 = _torch.tensor(
        [[1, 1, 1, -1], [1, 1, -1, 1], [1, -1, 1, 1], [-1, 1, 1, 1]],
        dtype=_torch.float32,
        device=device,
    )
    h = h4
    while h.shape[0] < size:
        h = _torch.kron(h, h4)
    return h / (size ** 0.5)

# in _unrotate_conv2d:
h = _regular_hadamard(gs, w.device)   # was: _regular_hadamard(gs)  -> CPU/CUDA mismatch

4. Verification

All checks run on the user's actual setup: JANKUTrainedChenkinNoobai_v777_hswq_r32_1off_convrot_int8.safetensors + CN-anytest_v4-marged_am_dim256.safetensors, 16 steps euler/simple, cfg 5.0, denoise 1.0.

Control-signal sanity (single forward)

	`input_blocks.1.0.in_layers.2` weight	control output norms (first 4)
Node path (before fix)	absmax 127.0 (raw qdata)	[731, 123,352, 183,752, NaN]
Node path (after fix)	absmax 0.50 (dequantized)	[720, 1,229, 1,415, 1,553]
Reference: full-type ControlNet	-	[325, 616, 944, 1,414]

Base UNet forward alone: NaN before fix -> std ~0.97, no NaN after fix.

End-to-end generation (user's exact workflow, strength 1.0)

Run	Saturation (mean HSV-S /255)	Structure adherence (L1 vs hint)
no control	110.6	0.472 (hint ignored)
am_dim256 (after fix)	73.5 - colored	0.299 (follows lineart)
full-type ControlNet	136.2	0.554

Visual check of the am_dim256 output (image recognition): "a color image (not a B&W line drawing) - an anime-style girl with long black hair in a sailor uniform sits in a colorful flower field." Before the fix the same workflow produced a near-grayscale copy of the hint (sat ~1.3).

5. Usage notes

Restart ComfyUI after updating the node pack - a long-running server keeps the old (broken) code in memory.
Load the INT8 checkpoint with HSWQ Checkpoint Loader (SDXL) and weight_dtype = int8_tensorwise - it is now routed through the INT8-aware loader (Conv2d decode + scope). Unet-only files keep using load_unet_hswq_weight_dtype / UNETLoader, which already set the scope.
The strength slider works normally again (the control magnitude is sane, so 0.0-1.0 scales the visible effect).
If a LoRA-type ControlNet still looks "locked" after the update, check that the base itself was loaded through one of the INT8-aware paths above (a plain stock load_checkpoint_guess_config reproduces the bug).
The full-type (non-LoRA) ControlNet path (CN-anytest4_illustrious2_A_convrot_int8) was never affected and continues to work unchanged.

Assets 2

v3.4.5 - HSWQ tcon NVFP4 2nd-Generation Noise Issue — Complete Guide

ussoewwin released this 27 Aug 04:20

v3.4.5

c3dba9e

中文

① What Was the Problem

Symptom

In workflows using tcon (Z Image TC/W4A4) NVFP4 models, the 1st generation is normal, but the 2nd generation after a DisTorch HSWQ purge is completely noise-corrupted.

Log Evidence (bake result comparison)

Generation	Bake result	State
1st	`nvfp4_baked=86 int8_baked=94` / `NVFP4_LORA_BAKE_OK`	Normal
2nd	`nvfp4_baked=0 other_qt_baked=83` / `NVFP4_LORA_BAKE_N/A`	NVFP4 layers misclassified as `other_qt`, baked without ConvRot → noise

nvfp4_baked=0 in the 2nd-generation log means not a single NVFP4 ConvRot layer was LoRA-baked. NVFP4 layers are stored with weights rotated by a Hadamard transform (ConvRot); baking a LoRA requires the sequence "unrotate → apply LoRA → re-rotate". When that sequence is skipped, raw LoRA deltas are added onto rotated weights, corrupting them and producing noise.

② Root Cause

Causal Chain (4 steps)

The purge peels HSWQ's load wrap
DisTorch's HSWQ purge calls uninstall_zimage_nvfp4_lora_bake(), which peels the wrap on ops._load_quantized_module (the wrapper that arms NVFP4 flags when HSWQ loads a quantized module). This drops the _hswq_nvfp4_full_load stamp.
apply_comfy_quant_nvfp4_patches() early-returns
This function decided "already applied" from _PATCHES_APPLIED and stack_ver alone, without checking whether the _load_quantized_module wrap was actually still in place. After a purge it therefore kept early-returning and never re-applied the wrap.
NVFP4 flags are never armed on reload
Without the _load_quantized_module wrap, arm_nvfp4_module() is not called when the model reloads, so the _hswq_nvfp4_convrot flag is never set on the individual Linear modules.
The bake function cannot detect NVFP4 and mis-bakes
During LoRA bake, _module_is_nvfp4_convrot() checks the flag. With the flag missing, NVFP4 layers are not detected, so they are handled as other_qt (other quantization); the ConvRot unrotate/re-rotate is skipped, the weights are corrupted → noise.

Deeper Background

ComfyUI caches the loader node's output (the MODEL object). Even though the purge removes the model from current_loaded_models, the loader node itself is not re-executed. As a result, load_unet never runs between "purge → next generation", so the hooks were never re-installed.
The purge's uninstall_zimage_nvfp4_lora_bake() peels the Dynamic.load wrap — this is correct design ("clean up ZI hooks before switching to SDXL"). The real problem was that there was no re-arm mechanism after peeling.

③ Files Added / Modified

ComfyUI-HSWQ-Loader-and-Tools (`1156f00` → `fdc60bc`)

File	Commit	Content
`nodes/zimage_nvfp4/load_unet.py`	`d97bb5b`	Added `_install_permanent_dynamic_load_guard()` — a permanent guard the purge cannot peel, auto-re-arming the `Dynamic.load` bake hook
`nodes/zimage_nvfp4/zi_comfy_quant_nvfp4.py`	`fdc60bc`	Added the `_load_wrap_ok` condition to the early-return in `apply_comfy_quant_nvfp4_patches()` — if the purge peeled the load wrap, fall through to a full re-apply

ComfyUI-DistorchMemoryManager (`62c7cbd` → `9854848`)

File	Commit	Content
`nodes/purge_vram.py` (primary — `__init__.py` prefers this one)	`9854848`	After the HSWQ purge completes, set the `unload_models` + `free_memory` queue flags so ComfyUI drops the cached loader-node output → the loader re-runs on the next prompt and the TC stack is rebuilt

Note: purge_vram.py (root, legacy fallback) received the same fix in 2936341, but __init__.py prefers nodes/purge_vram.py, so that is the primary file.

④ Full Code Added / Modified (no omissions)

4-1. `nodes/zimage_nvfp4/load_unet.py` (added in `d97bb5b`)

Function added immediately after _ensure_dynamic_load_bake_wrap():

def _install_permanent_dynamic_load_guard() -> None:
    """Install an outer ModelPatcherDynamic.load guard the purge cannot peel.

    The purge uninstall_zimage_nvfp4_lora_bake walks the chain of wraps stamped
    ``_hswq_zi_nvfp4_lora_bake`` and restores the unwrapped Dynamic.load. After that,
    nothing re-installs the bake hook because ComfyUI caches the loader-node output
    and never re-runs load_unet. The 2nd generation then runs without the NVFP4
    ConvRot LoRA bake and produces noise.

    This guard is a separate, permanent wrap (NOT stamped ``_hswq_zi_nvfp4_lora_bake``),
    so the purge deep-clean walks past it. On every Dynamic.load it ensures the bake
    hook is installed via ``_ensure_dynamic_load_bake_wrap()`` (a no-op when the hook
    is already armed at the current version).
    """
    try:
        import comfy.model_patcher as mp
    except ImportError:
        return
    Dynamic = getattr(mp, "ModelPatcherDynamic", None)
    if Dynamic is None:
        return
    cur = getattr(Dynamic, "load", None)
    if cur is None or getattr(cur, "_hswq_zi_rearm_guard", False):
        return

    def _guarded_load(self, *args, **kwargs):
        try:
            _ensure_dynamic_load_bake_wrap()
        except Exception:
            pass
        return cur(self, *args, **kwargs)

    _guarded_load._hswq_zi_rearm_guard = True  # type: ignore[attr-defined]
    _guarded_load._hswq_zi_rearm_guard_prev = cur  # type: ignore[attr-defined]
    Dynamic.load = _guarded_load

Call sites added (2 places):

# Inside load_unet_nvfp4_weight_dtype() (right after install_zimage_nvfp4_lora_bake)
    _ensure_dynamic_load_bake_wrap()
    _install_permanent_dynamic_load_guard()   # ← added
    reset_int8_lora_log_counters()
    reset_nvfp4_lora_log_counters()
    reset_zimage_nvfp4_lora_bake_log_counters()

# Inside the load_unet wrapper in install_zimage_nvfp4_unet_dispatch()
    def load_unet(self, unet_name, weight_dtype):
        _ensure_dynamic_load_bake_wrap()
        _install_permanent_dynamic_load_guard()   # ← added
        if weight_dtype in _fp8:
            return _prev(self, unet_name, weight_dtype)
        if weight_dtype == ZI_NVFP4_WEIGHT_DTYPE:
            return load_unet_nvfp4_weight_dtype(unet_name, weight_dtype)

4-2. `nodes/zimage_nvfp4/zi_comfy_quant_nvfp4.py` (fixed in `fdc60bc`)

    mp_fn = getattr(ops, "mixed_precision_ops", None)
    stack_ver = _effective_nvfp4_stack_ver(mp_fn)
    # The HSWQ purge peels the ops._load_quantized_module wrap (drops the
    # _hswq_nvfp4_full_load stamp) while leaving _PATCHES_APPLIED and the
    # detect_unet_config stamp intact. If the load wrap is gone, the arming
    # (arm_nvfp4_module -> _hswq_nvfp4_convrot) never fires on reload, so the
    # 2nd generation bakes NVFP4 layers as other_qt -> noise. Only early-return
    # when the load wrap is still armed; otherwise fall through to re-apply.
    _load_wrap_ok = bool(
        getattr(ops._load_quantized_module, "_hswq_nvfp4_full_load", False)
    )
    if (
        _PATCHES_APPLIED
        and _load_wrap_ok
        and getattr(model_detection.detect_unet_config, "_hswq_nvfp4_packed_dims", False)
        and stack_ver >= _NVFP4_STACK_VER
    ):
        return True

    # Already patched detect/load but LoRA bake missing: re-wrap mixed_precision_ops only.
    if _load_wrap_ok and getattr(model_detection.detect_unet_config, "_hswq_nvfp4_packed_dims", False) and stack_ver < _NVFP4_STACK_VER:
        # Z Image: INT8 wrap used to drop _hswq_nvfp4_stack_ver → false "upgrade"
        # that wrapped TC over ConvRot parity → double online rotate after refresh.
        if _mp_chain_has_comfy_only(mp_fn) or (
            _PATCHES_APPLIED
            and stack_ver == 0
            and getattr(mp_fn, "_hswq_int8_conv_patched", False)
        ):
            try:
                if mp_fn is not None:
                    mp_fn._hswq_nvfp4_stack_ver = _NVFP4_STACK_VER  # type: ignore[attr-defined]
            except Exception:
                pass
            _PATCHES_APPLIED = True
            _console(
                "[HSWQ NVFP4] stack ver stamped "
                "(skip TC upgrade; comfy_parity / INT8 chain intact)"
            )
            return True

        _orig_mp = getattr(mp_fn, "_hswq_nvfp4_orig_mp", None)
        if _orig_mp is None:
            _orig_mp = mp_fn

        def mixed_precision_ops_upgraded(*args, **kwargs):
            mp = _orig_mp(*args, **kwargs)
            Lin = mp.Linear
            # Never wrap TC over ConvRot parity (Z Image double-rotate / noise).
            if getattr(Lin.forward, "_hswq_nvfp4_convrot_parity", False):
                attach_nvfp4_linear_lora_bake(Lin)
                return mp
            if not getattr(Lin.forward, "_hswq_nvfp4_full_forward", False):
                Lin.forward = make_nvfp4_linear_forward(Lin.forward)
            attach_nvfp4_linear_lora_bake(Lin)
            return mp

        mixed_precision_ops_upgraded._hswq_nvfp4_full_forward = True  # type: ignore[attr-defined]
        mixed_precision_ops_upgraded._hswq_nvfp4_stack_ver = _NVFP4_STACK_VER  # type: ignore[attr-defined]
        mixed_precision_ops_upgraded._hswq_nvfp4_orig_mp = _orig_mp  # type: ignore[attr-defined]
        ops.mixed_precision_ops = mixed_precision_ops_upgraded
        _PATCHES_APPLIED = True
        _console(
            "[HSWQ NVFP4] upgraded stack ver=%s "
            "(ConvRot...

Assets 2

v3.4.4: ConvRot INT8 ControlNet Loader & Full Drop-in Compatibility

ussoewwin released this 26 Aug 07:55

v3.4.4

1156f00

中文

1. Overview

v3.4.4 introduces the HSWQ Load ConvRot INT8 ControlNet Model (HSWQLoadConvRotINT8ControlNet) node.

This node enables loading and executing ConvRot / TensorWise INT8-quantized ControlNet checkpoints (e.g., Qwen Image Fun ControlNet) directly in ComfyUI. Weights are maintained in 8-bit precision (QuantizedTensor / TensorWiseINT8Layout) in VRAM, and forward execution utilizes comfy_kitchen's high-speed int8_linear GEMM kernel with online activation rotation (convrot).

Additionally, the node includes automatic format detection and fallback mechanisms, providing 100% backwards compatibility with conventional FP16 / BF16 / FP8 ControlNet models as a full drop-in replacement for ComfyUI's stock "Load ControlNet Model" node.

2. Root Cause Analysis: Stock ComfyUI ControlNet Loader Limitations

When loading INT8-quantized checkpoints via stock ComfyUI (controlnet_load_state_dict), two critical structural failures occur:

Stage	Stock ComfyUI Behavior	Root Problem & Consequence
Module Graph Construction	Sets architecture dtype via `unet_dtype = weight_dtype(sd)`	For INT8 checkpoints, `weight_dtype(sd)` returns `torch.int8`. Instantiating PyTorch module graphs with `torch.int8` fails immediately: `RuntimeError: Only Tensors of floating point and complex dtype can require gradients`.
Quant-Aware Ops Dispatch	ControlNet loading path lacks `model_config.quant_config` and never passes `custom_operations`	Even if module initialization is forced to a float type, the `.comfy_quant` and `.weight_scale` tensors in the checkpoint are ignored, preventing quantized tensor attachment.

3. Architecture & Implementation

HSWQLoadConvRotINT8ControlNet addresses both issues through the following architecture:

[ safetensors Checkpoint ]
           │
           ▼
[_has_int8_comfy_quant] ──(No INT8 metadata)──► [ Stock load_controlnet_state_dict ] (FP16/FP8 Pass-through)
           │ (INT8 detected)
           ▼
[ model_options Construction ]
  ├── dtype = torch.bfloat16  (Prevents PyTorch INT8 gradient initialization crash)
  └── custom_operations = _int8_mixed_precision_ops() (Injects MixedPrecisionOps)
           │
           ▼
[ comfy.controlnet.load_controlnet_state_dict ]
           │
           ▼ (Consumes *.comfy_quant & *.weight_scale)
[ VRAM: TensorWiseINT8Layout (8-bit) ]
           │
           ▼ (Inference)
[ comfy_kitchen.int8_linear + ConvRot Online Activation Rotation ]
           │
           ▼
[ Output: Standard CONTROL_NET ] ──► [ Apply ControlNet Node ]

Core Components

Automatic Metadata Detection (_has_int8_comfy_quant):
- Inspects *.comfy_quant keys in the state dict and decodes the JSON payload.
- Triggers the INT8 quantization loader path if format == "int8_tensorwise"; otherwise falls back seamlessly to standard loading.
Explicit MixedPrecisionOps Construction (_int8_mixed_precision_ops):
- Instantiates a mixed_precision_ops class configured with QUANT_ALGOS["int8_tensorwise"].
- During _load_from_state_dict, every Linear layer attaches a QuantizedTensor (TensorWiseINT8Layout).
Float Graph Instantiation (dtype = torch.bfloat16):
- Forces the initial module graph to construct in torch.bfloat16, cleanly bypassing PyTorch INT8 parameter initialization errors before attaching quantized weights.
Standard ComfyUI Compatibility:
- Returns a standard CONTROL_NET object, seamlessly interoperating with standard Apply ControlNet nodes and existing workflows.

4. Feature Matrix

Feature	Stock Load ControlNet Model	HSWQ Load ConvRot INT8 ControlNet
ConvRot INT8 ControlNet Support	❌ (Crashes on init)	✅ (Native INT8 execution)
VRAM Footprint	N/A (Cannot load)	8-bit (`TensorWiseINT8Layout`)
Execution Kernel	—	`comfy_kitchen.int8_linear` + ConvRot online act rotation
Conventional FP16 / BF16 ControlNet	✅	✅ (Automatic fallback)
FP8 ControlNet	✅	✅ (Automatic fallback)
Output Type	`CONTROL_NET`	`CONTROL_NET` (100% compatible)

5. Added & Modified Files

Type	File	Description
Added	`nodes/hswq_load_convrot_int8_controlnet.py`	Implementation of `HSWQLoadConvRotINT8ControlNet`
Added	`png/convrot_int8_controlnet.png`	Node workflow screenshot
Added	`zhmd/v3.4.4.md`	Chinese Release Notes
Modified	`__init__.py`	Node registration & bump version to 3.4.4
Modified	`pyproject.toml`	Bump package version to 3.4.4
Modified	`changelog.md` / `zhmd/CHANGELOG.md`	Version 3.4.4 changelog entries
Modified	`README.md` / `zhmd/README.md`	Documentation and compatibility guide

6. Links

ComfyUI Loader Repository: ComfyUI-HSWQ-Loader-and-Tools
Upstream HSWQ Project: Hybrid-Sensitivity-Weighted-Quantization
Chinese Release Notes: zhmd/v3.4.4.md

Assets 2

v3.4.3 - Z Image Hybrid ConvRot NVFP4: Tensor Core (TC / W4A4) opt-in

ussoewwin released this 24 Aug 20:34

v3.4.3

9fee210

中文

1. Overview

v3.4.3 adds an opt-in Tensor Core (TC / W4A4) execution path for the Z Image / ZIT Hybrid ConvRot NVFP4 models. Until now these models ran only on the Comfy parity path — W4A16 (NVFP4 weights × fp16 activations, stock GEMM + online act rotate) — which keeps the 4-bit weight compression but never uses the FP4 Tensor Core. With a calibrated per-layer input_scale, the Linear hot path now switches to W4A4 TC (NVFP4 weights × 4-bit rotated activations on the raw cublas_gemm_blockwise_fp4 GEMM), unlocking the real Blackwell FP4 Tensor Core throughput.

The path is gated by trajectory-fidelity validation (a deterministic per-step latent-divergence comparison): TC vs parity final-cosine is 0.951 vs 0.952 (noise-level difference, 0 / 20 bifurcations), so TC adds no systematic quality loss while delivering the Tensor Core speedup.

2. Parity vs TC

	parity (previous default)	TC (new opt-in)
Weights	NVFP4 (E2M1 + block scale)	NVFP4 (E2M1 + block scale)
Activations	fp16	4-bit NVFP4 (rotated, E2M1)
GEMM	stock `a @ w.T` (fp16)	raw `cublas_gemm_blockwise_fp4` (FP4)
Tensor Core FP4	✗	✓
Prerequisite	—	calibrated `input_scale`

Parity was the default because TC needs an activation quantization scale (input_scale) that the earlier published checkpoints did not carry. Forcing TC on an uncalibrated checkpoint collapses quality (decoded SSIM ≈ 0.18) — this is why the path is opt-in and gated.

3. `input_scale` calibration

input_scale = amax / 2688, measured in the rotated domain (the same domain the ConvRot weights are stored in).
Produced by a standalone step: Z_Image/calib_input_scale_nvfp4.py — it samples the model's per-layer activation amax and writes *.input_scale keys into a *_calib.safetensors checkpoint.
Measured, not searched: input_scale is a running absmax ÷ 2688, not a histogram / quality knob — there is no per-layer search.
The loader reads *.input_scale from the checkpoint to decide TC eligibility.

4. Loader opt-in

Priority: HSWQ_ZI_FORCE_PARITY=1 > HSWQ_ZI_FORCE_TC=1 > auto-detect *.input_scale.
checkpoint_has_input_scale() detects the calibrated keys; zi_use_tensorcore() resolves the effective mode.
HSWQ_ZI_FORCE_PARITY=1 keeps the old W4A16 behaviour; HSWQ_ZI_FORCE_TC=1 forces W4A4 even without calibration (intentionally allowed, but collapses quality on uncalibrated files).
A warning-filter patch (_patch_load_model_weights_warnings()) silences the expected input_scale / comfy_quant load warnings that the parity path previously spammed.

5. Quality gate — trajectory divergence

Tool: benchmark/zi_convrot_nvfp4_traj_compare.py — a deterministic per-step latent trajectory comparator (torch.backends.cudnn.deterministic = True, fixed seeds), measuring the final-latent cosine vs the reference.
Bifurcation = a single-step cosine drop > 0.05 (divergence from the reference trajectory).
Same-image threshold = final-cos ≥ 0.98.
Results (hybrid nv89, 20 seeds):
- TC: final-cos mean 0.95137, min 0.86664, 0 / 20 bifurcations
- parity: final-cos mean 0.95233, min 0.86509, 0 / 20 bifurcations
- → TC ≈ parity (mean difference 0.001, noise-level).
Contrast — native full-NVFP4 (180 layers, no INT8 protection): mean 0.90945, min 0.60344, 1 / 20 bifurcations (seed 12 diverged). HSWQ hybrid (INT8-protected high-impact layers) removes the bifurcation.

6. Benchmark (test5 = moodyProMix nv90 config)

Decoded SSIM 0.9772
VRAM 12368 → 5935 MB (−52%) vs FP16
Wall time 3.39 s vs 7.18 s (2.12×)
TC GEMM hits 2160, dequant fallbacks 0
160.5 TFLOPS (~18% of Blackwell FP4 peak)

7. Files added / modified

Node repo (ComfyUI-HSWQ-Loader-and-Tools):

Kind	File	Change
Modified	`nodes/zimage_nvfp4/load_unet.py`	TC (W4A4) opt-in gating (`checkpoint_has_input_scale()` / `zi_use_tensorcore()`), `HSWQ_ZI_FORCE_TC=1` override, `input_scale`/`comfy_quant` load-warning filter
Modified	`nodes/zimage_nvfp4/zi_nvfp4_forward.py`	accumulate TC GEMM FLOPs (`_TC_FLOPS`); `nvfp4_forward_stats()` returns `tc_flops`

Upstream HSWQ repo (Hybrid-Sensitivity-Weighted-Quantization) — tools / validation:

Kind	File	Purpose
New	`Z_Image/calib_input_scale_nvfp4.py`	measure `input_scale = amax / 2688` (rotated domain)
New	`benchmark/zi_convrot_nvfp4_bench_v3.py`	`--tc` force-TensorCore bench + TFLOPS
New	`benchmark/zi_convrot_nvfp4_traj_compare.py`	deterministic trajectory divergence comparator
Modified	`nvfp4_addmm_patch.py` / `nvfp4_comfy_parity.py` (vendored)	GEMM-mode counters (`scaled_mm` hits vs dequant fallbacks; parity NVFP4/INT8 forward)

8. Links

ComfyUI loader: ComfyUI-HSWQ-Loader-and-Tools
Upstream HSWQ: Hybrid-Sensitivity-Weighted-Quantization
Model pack: Hybrid-Sensitivity-Weighted-Quantization-Z-Image-Hybrid-ConvRot-NVFP4
How-to (quantization): How to quantize Z Image - Hybrid NVFP4.md

Assets 2

v3.4.2 - Technical Guide: ComfyUI-HSWQ-Loader-and-Tools NVFP4 torch.compile Fix

ussoewwin released this 24 Aug 20:07

v3.4.2

359a89e

中文

① Error Contents

This fix resolves two independent errors.

Error A: `AssertionError: Mixing fake modes NYI` (`BackendCompilerFailed` with backend='inductor')

torch._dynamo.exc.BackendCompilerFailed:
backend='inductor' failed:
  ...
  torch/_functorch/_aot_autograd/...  (run_functionalized_fw_and_collect_metadata)
  comfy_kitchen/tensor/base.py:362 in __torch_dispatch__
  nodes/krea2_convrot_nvfp4/nvfp4_gemm.py:105 in dequantize_nvfp4
      out = torch.nn.functional.embedding(out.int(), lut).squeeze(-1)
  torch/nn/functional.py:2615 in embedding
  torch/_compile.py:54 in inner
      return disable_fn(*args, **kwargs)
  ...
AssertionError: Mixing fake modes NYI

(It reports that two distinct FakeTensorModes are mixed, in the form x.fake_mode=0x... vs self=0x.... The address values vary between runs.)

Error B: `UnicodeDecodeError: 'cp932' codec can't decode byte 0x94`

File "C:\...\repro_crash.py", line 38, in <module>
    f = torch.compile(linear_qt, backend="inductor", dynamic=False)
  File "torch\__init__.py", line 2836, in compile
  ...
  File "torch\_inductor\kernel\mm_grouped.py", line 142, in <module>
    source=load_kernel_template("cutedsl_mm_grouped"),
  File "torch\_inductor\utils.py", line 4770, in load_template
    return f.read()
UnicodeDecodeError: 'cp932' codec can't decode byte 0x94 in position 618: illegal multibyte sequence

Both are "fatal". Error A is the actual compilation failure. Error B only occurs on Japanese Windows (cp932 locale) and kills torch.compile the moment it tries to read a kernel template (in a real workflow, without the env vars this one fires first and masks Error A).

② Root Causes

Root cause of Error A

When torch.compile(linear_qt, backend="inductor") compiles F.linear(x_qt, w_qt, bias) (where x_qt / w_qt are QuantizedTensor subclasses):

Dynamo does not trace into a tensor subclass's __torch_dispatch__. It records torch._C._nn.linear as an opaque FX node only.
The inductor backend (= aot_autograd) runs the "metadata collection pass" (run_functionalized_fw_and_collect_metadata), re-executing the FX graph under a new FakeTensorMode (mode B).
During this re-execution, torch._C._nn.linear is invoked, dispatching to QuantizedTensor.__torch_dispatch__ → the HSWQ addmm handler → hswq_scaled_mm_nvfp4 → dequantize_nvfp4, which runs eagerly (in Python).
dequantize_nvfp4 performs its LUT decode using F.embedding. PyTorch wraps torch.embedding in torch._compile.disable; when executed under mode B, the raw torch.embedding re-enters dispatch.
At that point the arguments are:
- out.int() — a FakeTensor derived from the QT's inner tensor _qdata (mode A = the mode dynamo used when it first faked the QT)
- lut — a tensor just created (mode B)
  so two FakeTensorModes are mixed. FakeTensorMode's argument validation (validate_and_convert_non_fake_tensors) detects this and raises AssertionError: Mixing fake modes NYI.

Fundamental cause: the QT's inner tensors (_qdata, _params.scale, _params.block_scale) were faked under dynamo's mode A, but the AOT metadata pass re-fakes only the outer args under mode B, without re-faking the subclass's inner tensors. As a result, any aten op inside __torch_dispatch__ that mixes an inner tensor (mode A) with a freshly-created tensor (mode B) crashes.

The crash surfaces at F.embedding because that is the op wrapped in torch._compile.disable; it re-enters the fake mode and triggers validation.

Root cause of Error B

On first compilation, torch._inductor reads the Triton/CUTLASS kernel templates (*.py.jinja). This read is done by load_template in torch/_inductor/utils.py, which calls:

with open(template_dir / f"{name}.py.jinja") as f:
    return f.read()

i.e. it uses the builtin open() without an explicit encoding. The default encoding of open() is locale-dependent, and on Japanese Windows it is cp932. The template files are written in UTF-8, so decoding their UTF-8 byte sequences (e.g. 0x94) as cp932 raises UnicodeDecodeError.

load_kernel_template (kernel/mm_common.py) and load_flex_template (kernel/flex/common.py) are both functools.partial(load_template, template_dir=...), so all template loading funnels through this single load_template.

③ Fix Overview

Fix A (Mixing fake modes)

The FP4 decode was registered as a torch.library.custom_op (hswq::dequantize_nvfp4) with a register_fake meta kernel.

During fake tracing, the implementation body is never executed; only a shape-only fake kernel runs. This prevents the mixed-mode dispatch re-entry.
The real implementation (LUT decode) runs only on real tensors.
Numerics are bit-identical to the previous LUT path (verified: torch.equal matches, max|diff| = 0.0).

Fix B (cp932)

Python's UTF-8 mode (PYTHONUTF8=1) cannot be enabled after interpreter startup, so a new module win_utf8_patch.py replicates it at runtime, applied from the earliest possible point in the node.

Set the PYTHONUTF8 / PYTHONIOENCODING env vars (helps subprocesses).
Reconfigure stdin/stdout/stderr to UTF-8 (avoids UnicodeEncodeError on non-ASCII output to a cp932 console).
Monkey-patch io.open / builtins.open so a text-mode open(path) with no explicit encoding defaults to UTF-8 instead of the locale (binary mode / explicit encoding / file objects / descriptors are left untouched).

It is applied in two places: prestartup_script.py (which ComfyUI runs before __init__.py) and __init__.py (just before import torch). It is idempotent, so being loaded multiple times is safe.

④ Files Added / Modified

All under D:\USERFILES\ComfyUI\ComfyUI\custom_nodes\ComfyUI-HSWQ-Loader-and-Tools.

Kind	File	Change
Modified	`nodes\krea2_convrot_nvfp4\nvfp4_gemm.py`	Converted FP4 decode into the `hswq::dequantize_nvfp4` custom op
New	`win_utf8_patch.py`	Forces the process-wide default text encoding to UTF-8
Modified	`prestartup_script.py`	Added loading of `win_utf8_patch.py` at the top
Modified	`__init__.py`	Added loading of `win_utf8_patch.py` at the top (before `import torch`)
Restored	`nodes\krea2_convrot_nvfp4\nvfp4_addmm_patch.py`	Reverted a mistaken edit made during investigation back to the correct original state (no net change)

⑤ Full Text of the Added / Modified Code

5-1. `nodes\krea2_convrot_nvfp4\nvfp4_gemm.py` (full text, after fix)

"""HSWQ-owned NVFP4 GEMM helpers (ConvRot NVFP4).

ComfyUI / comfy_kitchen do **not** ship ConvRot×NVFP4 load+forward.
The Linear hot path lives in ``nvfp4_forward._tc_forward_pooled`` →
``nvfp4_runtime.scaled_mm_nvfp4_pooled`` (raw ``_C.cublas_gemm_blockwise_fp4``
with pre-validated shapes; weight stays packed). Never torch native
``F.scaled_mm`` FP4 / registry dispatch — path A sticky-poisons SM120.

This module owns:
  - NVFP4 unpack / dequant (FP4 E2M1 + block scales)
  - one-shot weight bake (replace QT Parameter → dense float; free packed) —
    TC-failure fallback only, never the hot path
  - float GEMM ``a @ b.T`` (+ optional bias) for residual QT×QT addmm edges

Runtime patches under ``benchmark/krea2_nvfp4`` must use these entry points.
"""
from __future__ import annotations

from typing import Optional

import torch

# FP4 E2M1 decode LUT (same values as kitchen float_utils / eager dequant).
_E2M1_VALUES = (
    0.0,
    0.5,
    1.0,
    1.5,
    2.0,
    3.0,
    4.0,
    6.0,
    -0.0,
    -0.5,
    -1.0,
    -1.5,
    -2.0,
    -3.0,
    -4.0,
    -6.0,
)
_E2M1_LUT_CACHE: dict = {}

# dtype ↔ int code for the ``hswq::dequantize_nvfp4`` custom-op schema.
_DTYPE_TO_CODE = {
    torch.float32: 0,
    torch.float16: 1,
    torch.bfloat16: 2,
    torch.float64: 3,
}
_CODE_TO_DTYPE = {v: k for k, v in _DTYPE_TO_CODE.items()}


def _ceil_div(a: int, b: int) -> int:
    return (a + b - 1) // b


def from_blocked(blocked_matrix, num_rows: int, num_cols: int):
    """Reverse cuBLAS 32×4×4 block-scale swizzle → (num_rows, num_cols)."""
    n_row_blocks = _ceil_div(num_rows, 128)
    n_col_blocks = _ceil_div(num_cols, 4)
    padded_rows = n_row_blocks * 128
    padded_cols = n_col_blocks * 4

    step1 = blocked_matrix.reshape(-1, 32, 16)
    step2 = step1.reshape(-1, 32, 4, 4).transpose(1, 2)
    step3 = step2.reshape(n_row_blocks, n_col_blocks, 4, 32, 4)
    step4 = step3.reshape(n_row_blocks, n_col_blocks, 128, 4)
    step5 = step4.permute(0, 2, 1, 3)
    unblocked = step5.reshape(padded_rows, padded_cols)
    return unblocked[:num_rows, :num_cols]


def clear_cuda_sticky_error() -> None:
    try:
        if not torch.cuda.is_available():
            return
        try:
            torch.cuda.synchronize()
        except RuntimeError:
            pass
    except Exception:
        pass


def _dequantize_nvfp4_op_impl(
    qx: torch.Tensor,
    per_tensor_scale: torch.Tensor,
    block_scales: torch.Tensor,
    output_dtype_code: int,
    hi_first: bool,
) -> torch.Tensor:
    """Real (eager) NVFP4 dequant body — same math as the pre-compile LUT path."""
    output_type = _CODE_TO_DTYPE[output_dtype_code]

    key = (str(qx.device), output_type)
    lut = _E2M1_LUT_CACHE.get(key)
    if lut is None:
        lut = torch.tensor(
...

Assets 2

v3.4.1 - HSWQ Z Image / ZIT Hybrid ConvRot NVFP4 Quantization & Model Release

ussoewwin released this 17 Aug 20:24

v3.4.1

0cd6ca3

中文

1. Overview

v3.4.1 publishes the HSWQ Z Image / ZIT Hybrid ConvRot NVFP4 quantization line and its first public model pack. It is a high-fidelity hybrid quantization for Z-Image-Turbo diffusion UNets: Linear layers → NVFP4 (Tensor Core scaled_mm_nvfp4), while the sensitivity-selected remainder stays as native ConvRot INT8 protection.

Image fidelity: decoded SSIM 0.97–0.99 (per-seed, measured on real pixels with the VAE attached)
VRAM: ~53–58% savings vs FP16
File size: ~40% smaller than FP16 (≈60% of FP16, mixed)
Runtime: loaded via HSWQ ConvRot INT8/ConvRot NVFP4 UNet Loader on the bench-matched Comfy parity path (stock GEMM + online act rotate) — a fully separated path from the SDXL Tensor Core product stack

2. Quantization Method — Reverse Hybrid NVFP4

This method is fundamentally different from the conventional "protect the top-important layers" approach (histogram MSE / cosine / SVD saliency). It is a reverse method:

Start from a complete ConvRot INT8 model (error ≈ 0) and convert layers to NVFP4 in ascending order of per-layer trajectory impact.

The conventional method ignores inter-layer interactions and is not sufficient for this hybrid. The reverse method stays in the low-error regime where additivity holds, so single-layer ranking is valid. Pass only if every seed meets decoded SSIM ≥ 0.95.

2.1 Step 1 — Per-layer impact measurement (`Z_Image/diag_impact.py`, ~12 min)

Inject NVFP4 error (e4m3, group-256 reconstruction) into one layer at a time, run a fixed-seed 4-step denoising trajectory, and measure how far the final latent drifts (relative MSE). That value is the layer's true importance under real trajectory propagation. This writes impact_<unet>.json (all 208 layers).

Typical ranking tendencies (always re-measure per checkpoint; ranking is not transferable):

Smallest / safest to convert: noise_refiner.*.attention.qkv-class layers
Largest / must protect: t_embedder.mlp.2, final_layer.linear, final_layer.adaLN_modulation.1

2.2 Step 2 — Reverse conversion (`Z_Image/gen_reverse_nvfp4.py`, ~1 min)

Rank layers ascending (lowest impact first), then convert the K lowest-impact layers from INT8 to NVFP4:

INT8 dequant (q × scale → rotated W@H^T)
Re-quantize with Kitchen TensorCoreNVFP4Layout (format: nvfp4, convrot: true, groupsize: 256)
without re-rotating — the INT8 weights are already stored rotated

Result: (208 − K) INT8 + K NVFP4 layers.

2.3 On-disk format of converted layers

Key	Layout
`.weight`	U8 packed `[out, in/2]`
`.weight_scale`	F8_E4M3 `[out, in/16]`
`.weight_scale_2`	F32
`.comfy_quant`	U8 tensor `{"format": "nvfp4", "convrot": true, "convrot_groupsize": 256}`

Weights are stored rotated (W@H^T); a large dequant-vs-FP16 deviation is expected.

2.4 Step 3 — Quality gate (bench)

Bench with the ComfyUI standard pipeline (ModelPatcher → KSampler → VAEDecode), all 5 seeds, --steps 12 --native-dtype --vae. Pass only if every seed's decoded SSIM ≥ 0.95 (the latent-view SSIM is blind to scale/shift collapse — always judge with --vae).

K is checkpoint-specific and searched, not a fixed number. The quality surface is often not a single cliff: failing seeds can change with K, and quality can recover then fail again (error cancellation). Treat "islands" as the default search assumption.

3. Published Models

Filename	Base Model	Version	License
`moodyProMix_zitV13_hswq_hybrid_nv80_convrot_nvfp4.safetensors`	Moody Pro Mix	zit v1.3 (nv80)	CreativeML Open RAIL++-M
`moodyProMix_collectorsEdition_hswq_hybrid_nv90_convrot_nvfp4.safetensors`	Moody Pro Mix	Collector's Edition (nv90)	CreativeML Open RAIL++-M
`moodyRealMix_zitV7_hswq_hybrid_nv100_convrot_nvfp4.safetensors`	Moody Real Mix	zit v7.0 (nv100)	CreativeML Open RAIL++-M
`moodyRealMix_xhsEdition_hswq_hybrid_nv110_convrot_nvfp4.safetensors`	Moody Real Mix	XHS Edition (nv110)	CreativeML Open RAIL++-M
`darkBeast30BF16INT8_dbzit9DIMRclaw_hswq_hybrid_nv100_convrot_nvfp4.safetensors`	DarkBeast	(nv100)	—

Base models: Moody Pro Mix / Moody Real Mix by catlover1937. All packs are derivatives of their respective original creators; see the HF model card for full credits and licensing.

4. Links

Quantization method (How-to): How to quantize Z Image - Hybrid NVFP4.md
Upstream HSWQ: Hybrid-Sensitivity-Weighted-Quantization
Model pack: Hybrid-Sensitivity-Weighted-Quantization-Z-Image-Hybrid-ConvRot-NVFP4
ComfyUI loader: ComfyUI-HSWQ-Loader-and-Tools
Benchmark results: benchmark result/benchmark_zi_nvfp4.md

Assets 2

v3.4.0 - HSWQ SDXL ConvRot NVFP4 Blackwell Tensor Core Acceleration (Tensor Boost) Technical Guide

ussoewwin released this 08 Aug 10:47

v3.4.0

af9365d

中文

1. Overview

To maximize HSWQ SDXL ConvRot NVFP4 inference performance on NVIDIA Blackwell (SM >= 100: B200 / GB200, RTX 5090 / SM120), this stack introduces a Per-Weight CUDA Graph auto-dispatch mechanism (Tensor Boost).

The feature is a closed, protected design inside nodes/nvfp4/ only (SDXL Product Tensor Core path). It does not affect Z Image ConvRot NVFP4 (nodes/zimage_nvfp4/ comfy-parity path), SDXL ConvRot INT8, FP8, or stock FP16/BF16 paths — a fully separated architecture.

To balance sampling speed with VRAM cost (Tensor Boost ON adds several GB) and system-RAM spill during upscale (USDU: Ultimate SD Upscale), independent BOOLEAN toggle switches are provided on the sampler (HSWQSampler) and the upscale node (HSWQUltimateSDUpscale). RTX 5090 with 32 GB+ is recommended when using Tensor Boost / high-res tiled upscale.

2. Architectural Background and Design

2.1 Limitations of the previous CUDA Graph (shape-shared)

The previous SDXL NVFP4 CUDA Graph path was shape-shared (_GRAPH_CACHE) and copied the full weight tensor with static_w.copy_(w_qdata) on every call.
That weight-copy overhead made CUDA Graph (~13.05 s) slower than the eager pooled path (~11.8 s).

2.2 Introducing Per-Weight CUDA Graph (`nvfp4_quant_mm_cudagraph_perweight`)

On Blackwell (SM100 / SM120), FP4 (E2M1) Tensor Core throughput is much higher, so host / PyTorch overhead and weight transfer copies dominate.
During sampling, model weight addresses (data_ptr) stay stable in VRAM, so nvfp4_quant_mm_cudagraph_perweight was added to capture weights directly into the graph without copying them.

Zero weight copy on replay: Only activation x and scales (scale_a, alpha, bias) are copied each replay; weight transfer overhead is eliminated.
Larger M coverage: Adaptive cap _PER_WEIGHT_GRAPH_MAX_M = 16384 covers all SDXL UNet Linear shapes at 1024×1024 and USDU tiles ($M = 8192, 4096, 2048, 512$, etc.).

3. Memory Characteristics and VRAM Saturation Mitigation

3.1 Eager Pooled vs CUDA Graph (Tensor Boost)

Item	Eager Pooled (`tensor_boost = False`)	Tensor Boost (`tensor_boost = True`)
Allocation style	Single buffer (`_ACT_Q_POOL`) reused across ~140 layers	PyTorch non-freed static allocator (CUDA Graph arena)
VRAM	No CUDA Graph arena stack (Eager pooled reuse)	Several GB more (CUDA Graph arenas; stacks further if shapes change)
CPU launch latency	Present (15–30 μs / layer)	Zero (GPU batch replay)
Speed	Baseline	Fastest (~15%–25% faster)
Recommended use	· USDU tiled upscale (keep OFF) · Changing input shapes (tiling)	· RTX 5090 32 GB+ recommended · First-pass 1024×1024 single resolution · Continuous sampling at max speed

3.2 USDU (tiled upscale) VRAM blow-up and mitigation

Tensor Boost ON already adds several GB of VRAM. In multi-tile USDU, edge handling and similar cases also feed different input shapes ($M$) per tile.
PyTorch CUDA Graphs capture a separate graph per shape, so arenas stack — dedicated VRAM saturates and spill into shared GPU memory (system RAM) is common on cards without 5090-class headroom.

Setting tensor_boost = False (default) on the USDU node clears the CUDA Graph cache as soon as upscale starts and runs Eager Pooled, so per-tile shape changes do not keep stacking Graph arenas.

4. UI Node Toggles and Control Layout

The intended workflow is: speed up the base pass with Tensor Boost, then turn it OFF only for upscale to avoid VRAM blow-up. Node roles:

4.1 Node roles

graph TD
    A["HSWQ Checkpoint Loader (SDXL)<br>(model load only / no toggle)"] --> B["HSWQ Sampler<br>(first-pass 1024x1024)"]
    B --> C["HSWQ Ultimate SD Upscale<br>(USDU tiled upscale)"]
    
    subgraph "First pass (speed)"
        B -- "tensor_boost = True (ON)" --> B1["CUDA Graph ON<br>several GB more VRAM"]
    end
    
    subgraph "Tiled upscale (VRAM safety)"
        C -- "tensor_boost = False (OFF)" --> C1["Eager Pooled<br>clear Graph arenas"]
    end

HSWQ Checkpoint Loader (SDXL):
- No toggle. Loads the model and installs NVFP4 operators only.
- Keeping the toggle off the loader avoids locking the whole graph OFF from load time when USDU needs OFF for upscale.
HSWQ Sampler (first sampling node):
- tensor_boost (BOOLEAN toggle).
- ON (True): first 1024×1024 sampling at full Tensor Boost (CUDA Graph) speed; VRAM rises by several GB. Sampler path: 16 GB+ recommended.
HSWQ Ultimate SD Upscale (USDU node):
- tensor_boost (BOOLEAN toggle) (default: False).
- OFF (False): on upscale start, sets HSWQ_NVFP4_TENSORBOOST=0 and runs clear_nvfp4_cudagraphs(), so tiles do not stack Graph arenas / spill. ON on this path needs RTX 5090 32 GB+ because Tensor Boost alone already costs several GB.

4.2 Environment variable interface

UI toggles map to env vars that gate the lower dispatch path:

HSWQ_NVFP4_TENSORBOOST=1 / HSWQ_NVFP4_CUDAGRAPH=1: Tensor Boost on
HSWQ_NVFP4_TENSORBOOST=0 / HSWQ_NVFP4_CUDAGRAPH=0: Tensor Boost off (Eager Pooled)

5. Logging and Diagnostics

Tensor Boost status is visible in the console / ComfyUI log in real time.

5.1 Toggle state logs (at NVFP4 load — see §11)

Toggle ON:

[HSWQ NVFP4 Tensor Boost] Tensor Boost Toggle ON: CUDA Graph Tensor Boost ACTIVE

Toggle OFF:

[HSWQ NVFP4 Tensor Boost] Tensor Boost Toggle OFF: Eager Pooled Path ACTIVE (Graph arenas cleared)

5.2 Capture and hit statistics

Capture log:

[HSWQ NVFP4 Tensor Boost] Captured Blackwell per-weight CUDA Graph #1 (shape M=8192 K=2048 N=2048, w_ptr=0x..., device=cuda:0)

Hit milestones (100, 500, 1000, …):

[HSWQ NVFP4 Tensor Boost] Running CUDA Graph accelerated GEMM (100 hits active)

nvfp4_forward_stats() dict:
- "blackwell_graph_hits": cumulative Blackwell CUDA Graph replay count
- "blackwell_tensor_boost_active": GPU class flag (True / False)

6. Path Isolation and Safety Guarantees

Path	Flag / condition	Tensor Boost	Memory protection
SDXL ConvRot NVFP4	`module._hswq_nvfp4 = True`	✅ Sampler / USDU toggle	`_PER_WEIGHT_GRAPH_CACHE.clear()` + `empty_cache()`
Z Image ConvRot NVFP4	Parity path (`_hswq_nvfp4 = False`)	❌ Fully excluded (Comfy Parity)	Cannot enter `_tc_forward_pooled`
SDXL ConvRot INT8	ComfyUI MixedPrecision / INT8 Ops	❌ Fully excluded	Separate bindings
FP8 / Native FP16	Stock ComfyUI Ops	❌ Fully excluded	Stock ComfyUI ops

Load / inference code for Z Image NVFP4, INT8, and FP8 is not touched. Tensor Boost runs only inside the SDXL ConvRot NVFP4 product path, so other formats and model structures are not polluted.

7. Addendum Policy (this section onward)

§1–§6 remain the authoritative design summary, tables, log strings, and recommended workflow.
From this section on, the guide adds file names, full relevant code, and meaning matched to the implementation so diagnostics can be cross-checked against source.

Control flow (summary):

UI (Sampler / USDU).tensor_boost
  → os.environ["HSWQ_NVFP4_TENSORBOOST"] = "1"|"0"
  → (when OFF) clear_nvfp4_cudagraphs()
  → Linear.forward (make_nvfp4_linear_forward)  [requires: module._hswq_nvfp4]
  → _tc_forward_pooled
  → is_nvfp4_cudagraph_enabled() × is_blackwell_gpu()
  → nvfp4_quant_mm_cudagraph_perweight  OR  eager pooled

The Loader has no toggle. At NVFP4 checkpoint load, the current env is also read and the same §5.1 Toggle lines are printed (details in §11).

8. Created / Modified Files (Tensor Boost)

Kind	Path	Role for Tensor Boost
Core / modified	`nodes/nvfp4/nvfp4_runtime.py`	`_PER_WEIGHT_*` cache, `clear_nvfp4_cudagraphs`, `nvfp4_quant_mm_cudagraph_perweight`
Core / modified	`nodes/nvfp4/nvfp4_forward.py`	`_tc_forward_pooled` dispatch, hit stats, `nvfp4_forward_stats`
Core / modified	`nodes/nvfp4/nvfp4_conf.py`	`is_blackwell_gpu`, `is_nvfp4_cudagraph_enabled` (env read)
Core / existing (gate)	`nodes/nvfp4/nvfp4_load.py`	`arm_nvfp4_module` sets `_hswq_nvfp4 = True` (TC entry condition)
UI / modified	`nodes/hswq_sampler.py`	Optional `tensor_boost`; env + clear-on-OFF at start of `sample`
UI / modified	`nodes/nunchaku_usdu.py`	Required `tensor_boost` (default False); env + clear at start of `upscale`
Load diagnostics / modified	`nodes/nvfp4/comfy_quant_nvfp4.py`	§5.1 Toggle ON/OFF log at NVFP4 load
Loader (no toggle)	`__init__.py`	`HSWQCheckpointLoaderSDXL` — `ckpt_name` / `weight_dtype` / `device` only
Isolated (not applied)	`nodes/zimage_nvfp4/`	Comfy-parity; does not use Product TC (`_tc_forward_pooled`)

The “new” core is nvfp4_quant_mm_cudagraph_perweight plus the per-weight cache set. UI, env, and dispatch are wiring onto the existing SDXL NVFP4 stack.

9. Per File: Full Code and Meaning

9.1 `nodes/nvfp4/nvfp4_conf.py` — GPU detection and env gate

Meaning

is_blackwell_gpu(): compute capability major ≥ 10 (SM100 / SM120, etc.). GPU condition to enter per-weight graphs.
**`is...

Assets 2

v3.3.9 - HSWQ ComfyUI 0.30.2 Compatibility Fix — Complete Technical Reference

ussoewwin released this 07 Aug 10:27

v3.3.9

228c9fa

中文

1. What Went Wrong

Problem 1: Krea2 ConvRot INT8 Extremely Slow (Primary Symptom)

After updating ComfyUI to the 0.30.x series, Krea2 ConvRot INT8 inference became extremely slow.
Two root causes.

(a) Full-module scan on every load_models_gpu patch invocation

patches/comfy_quant_int8.py monkey-patches comfy.model_management.load_models_gpu,
and on every call it ran:

_model_has_int8_quantized_weights(model) — walks every module via model.named_modules()
(thousands to tens of thousands for Krea2) searching for QuantizedTensor
_model_is_nunchaku_svdq(model) — same full-module scan

In ComfyUI 0.30.x, model load / memory management frequency increased, so these O(n) scans
executed each time, accumulating to massive overhead.

(b) CPU→GPU transfer of Hadamard matrix on every forward pass

ConvRot (Hadamard rotation) rotates activations as x_rot = x @ H.
The old rotate_activation() called h_matrix.to(dtype, device) every time,
transferring a CPU-built Hadamard matrix to the GPU on every forward pass.
Additionally, the HSWQ-injected Conv2d forward also called build_hadamard(..., device="cpu")
each time. GPU transfers involve device synchronization; the cost multiplied by layer
count × step count.

Problem 2: ZI NVFP4 VRAM Growth (Secondary Symptom)

nodes/zimage_nvfp4/nvfp4_lora_bake.py's install_load_models_gpu_bake_hook
scanned all current_loaded_models on every load_models_gpu call:

_nvfp4_convrot_diag(model) — full-module scan (no cache)
run_zimage_nvfp4_lora_bake_on_patcher() — fallback path called
_patcher_has_quant_via_keys() which walks all LoRA patch keys,
calling get_key_weight() (expensive QT unwrap) for each key

Repeating this caused GPU memory fragmentation and unnecessary weight movement,
cumulatively increasing VRAM usage.

Problem 3 (Latent Bug): `get_hadamard_on_device` Referenced But Undefined

In the first fix commit 21792a8, the HSWQ-injected Conv2d forward in
patches/comfy_quant_int8.py was changed to call nc.get_hadamard_on_device(...),
but the patch to add the function definition in native_convert_int8.py
silently failed (CRLF line-ending mismatch in PowerShell Replace).
Only the _HADAMARD_GPU_CACHE dict (+3 lines) was committed.

Krea2 (DiT) does not use the Conv2d injection path, so no symptom was observed
("speed restored" appeared correct)
Using SDXL ConvRot INT8 would trigger AttributeError on first forward — latent bug

Problem 4 (Latent Bug): `weight_inner` Referenced But Not Defined

In the same commit 21792a8, _bake_int8_patches_on_dynamic_patcher had
isinstance(weight, QuantizedTensor) changed to isinstance(weight_inner, ...),
but the patch to add the definition line
weight_inner = weight.data if hasattr(weight, "data") else weight
was not applied ("pattern not found").

SDXL / INT8 + LoRA Dynamic bake path would trigger NameError — latent bug

Problem 5 (Compatibility): ComfyUI 0.30.2 API Changes

mixed_precision_ops's disabled argument is expected to be a set in 0.30.2
(old HSWQ code passed [])
LowVramPatch.__call__ → comfy.lora.calculate_weight added original_weights
parameter
In 0.30.2, quantized weights may be stored as Parameter(QuantizedTensor),
so isinstance(w, QuantizedTensor) alone fails to detect them
_quantized_weight_state_dict added extra_quant_params parameter
LoRA modules relocated to comfy.weight_adapter.lora

2. Files Created / Modified

File	Type	Content
`native_convert_int8.py`	Modified	GPU-side Hadamard cache (`get_hadamard_on_device`), `rotate_activation` uses GPU cache
`patches/comfy_quant_int8.py`	Modified	Early-return/cache, `disabled` set normalization, 0.30.2 compat (Parameter.data, original_weights, extra_quant_params), `weight_inner` definition, parity contamination peel for Krea2
`nodes/zimage_nvfp4/nvfp4_lora_bake.py`	Modified	`load_models_gpu` bake hook fast-skip
`__init__.py`	Modified	`comfy.weight_adapter.lora` import fallback, `calculate_weight` signature fix

No new files (all existing files modified in-place).

3. Full Code Changes

3-1. `native_convert_int8.py`

(a) Module top (GPU cache dict added)

_DEFAULT_GROUPSIZE = 256
_HADAMARD_CACHE: dict[tuple[int, str, torch.dtype], torch.Tensor] = {}
# GPU-side cache: avoids CPU→GPU transfer on every rotate_activation call.
# Keyed by (size, device_str, dtype) – same as CPU cache but on target device.
_HADAMARD_GPU_CACHE: dict[tuple[int, str, torch.dtype], torch.Tensor] = {}

(b) New: `get_hadamard_on_device()` (immediately after `build_hadamard`)

def get_hadamard_on_device(
    size: int,
    device: str | torch.device = "cpu",
    dtype: torch.dtype = torch.float32,
) -> torch.Tensor:
    """Return Hadamard matrix on target device, with GPU-side caching.

    Builds on CPU (via build_hadamard) once, then transfers to the target
    device and caches there. Subsequent calls with the same
    (size, device, dtype) hit the GPU cache and skip CPU→GPU transfer.
    """
    cache_key = (size, str(device), dtype)
    cached = _HADAMARD_GPU_CACHE.get(cache_key)
    if cached is not None:
        return cached
    h = build_hadamard(size, device="cpu", dtype=torch.float32)
    h = h.to(dtype=dtype, device=device)
    _HADAMARD_GPU_CACHE[cache_key] = h
    return h

(c) `rotate_activation()` (switched to GPU cache)

def rotate_activation(
    x: torch.Tensor, h_matrix: torch.Tensor, group_size: int
) -> torch.Tensor:
    """Online Linear: x_rot = x @ H (last dim = features)."""
    orig_shape = x.shape
    features = orig_shape[-1]
    if features % group_size != 0:
        raise ValueError(f"features {features} not divisible by group_size {group_size}")
    group_count = features // group_size
    x_grouped = x.reshape(-1, group_count, group_size)
    # GPU-cached Hadamard: build/transfer once, reuse on every call
    h = get_hadamard_on_device(group_size, device=x.device, dtype=x.dtype)
    return torch.matmul(x_grouped, h).reshape(orig_shape)

3-2. `patches/comfy_quant_int8.py`

(a) `_model_is_nunchaku_svdq()` early exit

    seen = set()
    _checked = 0
    _MAX_CHECK_SVDQ = 100  # Early exit: SVDQ modules are typically at the top
    for root in roots:
        rid = id(root)
        if rid in seen:
            continue
        seen.add(rid)
        try:
            named = root.named_modules()
        except Exception:
            continue
        for _, module in named:
            cls_name = type(module).__name__
            if (
                "SVDQ" in cls_name
                or "Nunchaku" in cls_name
                or cls_name.startswith("ComfyNunchaku")
            ):
                return True
            mod = getattr(type(module), "__module__", "") or ""
            if _module_path_is_real_nunchaku_package(mod):
                return True
            _checked += 1
            if _checked >= _MAX_CHECK_SVDQ:
                break
    return False

(b) `_model_has_int8_quantized_weights()` early exit + 0.30.2 support

    if _model_is_nunchaku_svdq(model):
        return False
    try:
        from comfy.quant_ops import QuantizedTensor
    except ImportError:
        return False
    _checked = 0
    _MAX_CHECK = 200  # Early exit: only scan first 200 modules
    for _, module in model.named_modules():
        cls_name = type(module).__name__
        if "SVDQ" in cls_name or "Nunchaku" in cls_name:
            continue
        w = getattr(module, "weight", None)
        if w is None:
            continue
        if isinstance(w, QuantizedTensor):
            return True
        # 0.30.2: Parameter wrapping QuantizedTensor
        if hasattr(w, "data") and isinstance(w.data, QuantizedTensor):
            return True
        # Fast path: layout_type set means quantized
        if getattr(module, "layout_type", None) is not None:
            return True
        _checked += 1
        if _checked >= _MAX_CHECK:
            break
    return False

(c) Injected Conv2d `state_dict()` (0.30.2 `_quantized_weight_state_dict` compat)

        def state_dict(self, *args, destination=None, prefix="", **kwargs):
            sd = destination if destination is not None else {}
            sd = _quantized_weight_state_dict(self, sd, prefix,
                extra_quant_params=("input_scale", "pre_quant_scale"))
            # Re-stamp ConvRot on export (Params.convrot cleared for safe 4D dequant).
            if getattr(self, "_hswq_convrot", False):
                cq_key = f"{prefix}comfy_quant"
                conf = {
                    "format": "int8_tensorwise",
                    "convrot": True,
                    "convrot_groupsize": int(
                        getattr(self, "_hswq_convrot_groupsize", 256) or 256
                    ),
                }
                sd[cq_key] = torch.tensor(
                    list(json.dumps(conf, separators=(",", ":")).encode("utf-8")),
                    dtype=torch.uint8,
                )
            return sd

(d) Injected Conv2d `forward_comfy_cast_weights()` (GPU cache usage)

        def forward_comfy_cast_weights(self, input):
            if getattr(self, "_hswq_convrot", False):
          ...

Assets 2

Releases: ussoewwin/ComfyUI-HSWQ-Loader-and-Tools

Release list

v3.4.8 - SAM3 Nodes Removed (Stock Loader Support Confirmed)

HSWQ v3.4.8 — SAM3 Nodes Removed (Stock Loader Support Confirmed)

🗑️ Removed

📖 Background (why)

🔧 What changed

📦 Related

💡 Technical notes from the experience

Uh oh!

v3.4.7 - SAM3 ConvRot INT8 Nodes

HSWQ v3.4.7 — SAM3 ConvRot INT8 Nodes

✨ Added

🐛 Fixed

📚 Documentation

✅ Compatibility

Uh oh!

v3.4.6 - SDXL anytest ControlLoRA on ConvRot INT8 / Hybrid ConvRot NVFP4 Bases - Complete Fix Guide (v2)

1. Symptoms

2. Root cause

2.1 HSWQCheckpointLoaderSDXL ignored weight_dtype="int8_tensorwise" (primary cause of symptom 2)

2.2 ControlLora borrowed-weight dequant (fixed by c60bb0b, still required)

2.3 Hadamard device mismatch (fixed by 152c1dc)

3. Files modified

Key code - __init__.py (node fix, 152c1dc)

Key code - patches/comfy_quant_int8.py (Hadamard device fix, 152c1dc)

4. Verification

Control-signal sanity (single forward)

End-to-end generation (user's exact workflow, strength 1.0)

5. Usage notes

Uh oh!

v3.4.5 - HSWQ tcon NVFP4 2nd-Generation Noise Issue — Complete Guide

① What Was the Problem

Symptom

Log Evidence (bake result comparison)

② Root Cause

Causal Chain (4 steps)

Deeper Background

③ Files Added / Modified

ComfyUI-HSWQ-Loader-and-Tools (1156f00 → fdc60bc)

ComfyUI-DistorchMemoryManager (62c7cbd → 9854848)

④ Full Code Added / Modified (no omissions)

4-1. nodes/zimage_nvfp4/load_unet.py (added in d97bb5b)

4-2. nodes/zimage_nvfp4/zi_comfy_quant_nvfp4.py (fixed in fdc60bc)

Uh oh!

v3.4.4: ConvRot INT8 ControlNet Loader & Full Drop-in Compatibility

1. Overview

2. Root Cause Analysis: Stock ComfyUI ControlNet Loader Limitations

3. Architecture & Implementation

Core Components

4. Feature Matrix

5. Added & Modified Files

6. Links

Uh oh!

v3.4.3 - Z Image Hybrid ConvRot NVFP4: Tensor Core (TC / W4A4) opt-in

1. Overview

2. Parity vs TC

3. input_scale calibration

4. Loader opt-in

5. Quality gate — trajectory divergence

6. Benchmark (test5 = moodyProMix nv90 config)

7. Files added / modified

8. Links

Uh oh!

v3.4.2 - Technical Guide: ComfyUI-HSWQ-Loader-and-Tools NVFP4 torch.compile Fix

① Error Contents

Error A: AssertionError: Mixing fake modes NYI (BackendCompilerFailed with backend='inductor')

Error B: UnicodeDecodeError: 'cp932' codec can't decode byte 0x94

② Root Causes

Root cause of Error A

Root cause of Error B

③ Fix Overview

Fix A (Mixing fake modes)

Fix B (cp932)

④ Files Added / Modified

⑤ Full Text of the Added / Modified Code

5-1. nodes\krea2_convrot_nvfp4\nvfp4_gemm.py (full text, after fix)

Uh oh!

v3.4.1 - HSWQ Z Image / ZIT Hybrid ConvRot NVFP4 Quantization & Model Release

1. Overview

2.1 `HSWQCheckpointLoaderSDXL` ignored `weight_dtype="int8_tensorwise"` (primary cause of symptom 2)

2.2 ControlLora borrowed-weight dequant (fixed by `c60bb0b`, still required)

2.3 Hadamard device mismatch (fixed by `152c1dc`)

Key code - `init.py` (node fix, `152c1dc`)

Key code - `patches/comfy_quant_int8.py` (Hadamard device fix, `152c1dc`)

ComfyUI-HSWQ-Loader-and-Tools (`1156f00` → `fdc60bc`)

ComfyUI-DistorchMemoryManager (`62c7cbd` → `9854848`)

4-1. `nodes/zimage_nvfp4/load_unet.py` (added in `d97bb5b`)

4-2. `nodes/zimage_nvfp4/zi_comfy_quant_nvfp4.py` (fixed in `fdc60bc`)

3. `input_scale` calibration

Error A: `AssertionError: Mixing fake modes NYI` (`BackendCompilerFailed` with backend='inductor')

Error B: `UnicodeDecodeError: 'cp932' codec can't decode byte 0x94`

5-1. `nodes\krea2_convrot_nvfp4\nvfp4_gemm.py` (full text, after fix)

2.1 Step 1 — Per-layer impact measurement (`Z_Image/diag_impact.py`, ~12 min)

2.2 Step 2 — Reverse conversion (`Z_Image/gen_reverse_nvfp4.py`, ~1 min)

2.2 Introducing Per-Weight CUDA Graph (`nvfp4_quant_mm_cudagraph_perweight`)