Skip to content

v2.6.0

Choose a tag to compare

View all tags
@ussoewwin ussoewwin released this 28 Aug 11:29
· 57 commits to main since this release
v2.6.0
0c9aec8
EN 中文

Overview

This release:

  1. Renames the ControlNet node from NunchakuQI&ZITDiffsynthControlnet to Nunchaku ZI Diffsynth Controlnet&Krea2 LoRA ControlNet (internal class NunchakuQwenImageDiffsynthControlnet is unchanged, so existing workflows load without modification).
  2. Clarifies the supported model scope of each route precisely (ZI vs QI vs Krea2).
  3. Fixes route classification so the standard (non-Nunchaku) QwenImageTransformer2DModel is routed to the qwenimage_standard path instead of failing with unknown.

1. Node Rename

Before After
NunchakuQI&ZITDiffsynthControlnet Nunchaku ZI Diffsynth Controlnet&Krea2 LoRA ControlNet
  • Internal class name: NunchakuQwenImageDiffsynthControlnet (unchanged).
  • Workflow compatibility: existing saved workflows referencing the old display name still load because the class name is the stable identifier.
  • The name was misleading: the old name implied the node handled "Nunchaku Qwen Image", but the DiffSynth route does not work with Nunchaku Qwen Image. The new name reflects the actual capability: Z-Image-Turbo (ZI) DiffSynth ControlNet + Krea2 LoRA ControlNet.

2. Clarified Supported Model Scope

The node has three routes, each with a strictly defined model scope:

Z-Image-Turbo (ZI) route

  • Supports Nunchaku (quantized) Z-Image-Turbo models and all non-quantized / quantized Z-Image models.
  • Example: svdq-fp4_r128-z-image-turbo.safetensors (Nunchaku) with Z-Image-Turbo-Fun-Controlnet-Union model patches.

Qwen Image (QI) route

  • Supports all non-Nunchaku Qwen Image models:
    • Non-quantized: standard bf16 / fp16 Qwen Image models (e.g. qwnImageEdit_v16Bf16.safetensors).
    • Quantized: HSWQ ConvRot INT8 models (e.g. qwnImageEdit_v16Bf16_convrot_int8.safetensors).
  • Nunchaku Qwen Image is NOT supported by the DiffSynth route. See Section 4 for the technical reason.

Krea2 route (depth / openpose)

  • Unchanged, fully isolated route via Krea2ControlNetLoraLoader (krea2-depth-control-lora.safetensors / krea2_turbo_openpose_controlnet.safetensors).

3. Route Classification Fix

The _classify_controlnet_target helper now recognizes the standard (non-Nunchaku) diffusion model class:

if dm_name in ("ComfyQwenImageWrapper", "NunchakuQwenImageTransformer2DModel", "QwenImageTransformer2DModel"):
    return "qwenimage_standard"

Before: a standard bf16 Qwen Image model (QwenImageTransformer2DModel) fell through to unknown, which raised a RuntimeError and aborted the workflow.

After: it is correctly routed to qwenimage_standard, which uses the standard ComfyUI patches["double_block"] path - the same mechanism the ComfyUI reference QwenImageDiffsynthControlnet node uses. This makes qwen_image_canny_diffsynth_controlnet_convrot_int8.safetensors etc. work with standard bf16 Qwen Image models via Model Patch Loader (ModelPatchLoaderCustom) + this node.

4. Technical Background: Why Nunchaku Qwen Image is Incompatible with the DiffSynth Route

Investigation showed the failure is not in the ControlNet model itself (the QwenImageBlockWiseControlNet decode is correct) but in the nunchaku W4A4-quantized Qwen Image model's internal hidden-state scale:

  • The nunchaku SVDQW4A4Linear (W4A4, activation also quantized) amplifies hidden-states inside each transformer block (measured: to_qkv ~29x, MLP ~760x amplification; block output mean ~1.4e5, whereas Z-Image stays at standard scale ~0.3-5).
  • The hidden-states also have severely non-uniform per-channel scales (channel RMS spread up to ~1500x), caused by the accumulated W4A4 group scales.
  • BlockWise ControlNet (QwenImageBlockWiseControlNet) is trained for standard-scale hidden-states. Its residual is ~0.3% of the nunchaku hidden-state magnitude, so the control is effectively invisible; naive residual rescaling breaks the channel structure and produces glitched output.
  • Z-Image is unaffected because its nunchaku implementation (NextDiT) keeps hidden-states at standard scale.

Conclusion: For the QI route, use a standard (non-Nunchaku) bf16 Qwen Image model. For the ZI route, Nunchaku Z-Image-Turbo works as expected. This is a nunchaku Qwen Image limitation (similar to upstream issue #711), not a bug in the ControlNet model itself.

Docs

  • README (EN + ZH) updated: node image, accurate supported-scope notation, usage notes.
  • Changelog updated (EN + ZH).

Compatibility

  • ComfyUI-nunchaku (official) - ZI route
  • Standard ComfyUI Qwen Image bf16 - QI route
  • HSWQ quantized Qwen Image (ConvRot INT8) - QI route
  • Krea2 control LoRAs (depth / openpose) - Krea2 route
Assets 2
Loading