Repository navigation
| EN | 中文 |
1. What changed, and why
1.1 Background: ComfyUI v0.44.0 supplies weights through a cast contract
Since ComfyUI v0.44.0 (dynamic VRAM / aimdo enabled by default), layer weights are no longer guaranteed to sit on the compute device when a module's forward runs. Weights stay in their file form (CPU / virtual-memory backed) until the module takes part in the core weight-supply contract:
# comfy/ops.py (v0.44.0, disable_weight_init.Linear.forward)
def forward(self, *args, **kwargs):
run_every_op()
if self.comfy_cast_weights or len(self.weight_function) > 0 or len(self.bias_function) > 0:
return self.forward_comfy_cast_weights(*args, **kwargs) # contract path
else:
return super().forward(*args, **kwargs) # reads self.weight directlyThe contract path (forward_comfy_cast_weights -> CastBiasWeightContext -> cast_bias_weight) is what moves the weight to the input device: vbar-managed layers go through comfy/ops.py:396-414, plain tensors through comfy/ops.py:417-462. A module that never opts in reads self.weight directly and gets whatever device the weight happens to be on.
Modules created and loaded through a ModelPatcher are armed by the core. The problem sits outside that path.
1.2 The symptom
Qwen Image ControlNet does not call the diffusion model through the wrapper's forward. It reaches into the base model directly (comfy/ldm/qwen_image/controlnet.py:151-162):
hidden_states = base_model.img_in(hidden_states)
encoder_hidden_states = base_model.txt_norm(context)
encoder_hidden_states = base_model.txt_in(encoder_hidden_states)
temb = base_model.time_text_embed(timesteps, hidden_states)base_model here is this package's ComfyQwenImageWrapper, whose __getattr__ forwards unknown attributes to the inner nunchaku model. The borrowed img_in is a plain disable_weight_init.Linear with comfy_cast_weights = False - it never joined the contract. The failure looks like this:
File "comfy/samplers.py", line 330, in _calc_cond_batch
c['control'] = control.get_control(input_x, timestep_, c, len(cond_or_uncond), transformer_options)
File "comfy/controlnet.py", line 320, in get_control
control = self.control_model(x=x_noisy.to(dtype), hint=self.cond_hint, ...)
File "comfy/ldm/qwen_image/controlnet.py", line 151, in forward
hidden_states = base_model.img_in(hidden_states)
File "comfy/ops.py", line 641, in forward
return super().forward(*args, **kwargs)
RuntimeError: Expected all tensors to be on the same device, but got mat1 is on cuda:0,
different from other tensors on cpu (when checking argument in method wrapper_CUDA_addmm)
1.3 Why the second run fails while the first succeeds
Measured on the affected workflow (10 consecutive runs, ComfyUI v0.44.0, torch 2.15.0+cu134):
| Run | ControlNet loader re-executed | Result | Wall time |
|---|---|---|---|
| 1 | yes | OK | 106.06 s |
| 2 | no (node cache) | FAIL | 31.56 s |
| 3 | yes | OK | 68.24 s |
| 4-10 | no (node cache) | FAIL | 5.5 - 9.5 s |
- When the loader re-executes, the ControlNet state dict is re-read (
Found quantization metadata version 1) and the weights reach the GPU through the load path before sampling. - When ComfyUI's node-output cache returns the cached
CONTROL_NET/MODELobjects, the loader does not run again. The workflow's end-of-run purge nodes have already unloaded the models, so the cached objects carry weights that were dropped back to CPU / file form - and the borrowed modules still read them directly.
This is a lifecycle gap: cache reuse + purge + a direct attribute borrow that sits outside the core's contract. It only became fatal when v0.44.0 changed the default weight placement.
2. The fix: opt the borrowed modules into the core contract
wrappers/qwenimage.py - ComfyQwenImageWrapper.__init__ now calls one static helper:
# wrappers/qwenimage.py (v2.6.8)
_CONTROLNET_BORROWED_MODULES = (
"img_in",
"txt_in",
"txt_norm",
"time_text_embed",
"pe_embedder",
)
@staticmethod
def _enable_weight_supply_contract(model):
"""Opt the modules ControlNet borrows into ComfyUI's cast contract."""
for attribute in _CONTROLNET_BORROWED_MODULES:
root = getattr(model, attribute, None)
if root is None:
continue
seen = set()
stack = [root]
while stack:
module = stack.pop()
if id(module) in seen:
continue
seen.add(id(module))
if hasattr(module, "comfy_cast_weights") and not module.comfy_cast_weights:
module.comfy_cast_weights = True
stack.extend(module.children())
# ComfyQwenImageWrapper.__init__, last line:
self._enable_weight_supply_contract(model)Properties:
- Scoped: only the five attributes ControlNet actually accesses on
base_model(img_in,txt_in,txt_norm,time_text_embed,pe_embedder) and their children are touched; the rest of the model stays exactly as constructed (verified below). - Idempotent: modules already opted in are skipped.
- Follows the core instead of fighting it: dynamic VRAM stays enabled - no launcher flag, no ComfyUI-core edit, no nunchaku edit, no workflow edit. Whenever the core moves a weight,
cast_bias_weightsupplies it to the input device on every forward, including the ControlNet path.
3. Repository state after this release
| Path | Change | Note |
|---|---|---|
wrappers/qwenimage.py |
+38 / -0 | ComfyQwenImageWrapper opts the borrowed modules into the contract |
| ComfyUI core / nunchaku package / workflows | unchanged | no file outside this package is modified |
RELEASE_NOTES/changelog.md / zhmd/changelogzh.md |
changelog | v2.6.8 entries (16 insertions, 2 deletions combined) |
pyproject.toml / __init__.py |
version | 2.6.7 -> 2.6.8 |
4. What was verified
- Contract-off reproduction: with the weight on CPU and
comfy_cast_weights = False,F.linearraises the exactRuntimeErrorshown above (python_embeded, torch 2.15.0+cu134, cuda:0). - Contract-on: the same layer returns a CUDA result; against an identity weight the output equals the input (
maxdiff0.0). - Scope:
img_in/txt_inflipFalse -> True; a sibling Linear outside the five borrowed attributes staysFalse. - Deployed copy: byte-identical to the repo file (SHA256
20061EBC58DBFCC1BDCAF33EC8628F6AA86F7339B5229BDEBBF9328F55903262),py_compileclean. - Real workflow: repeated executions of the affected i2i workflow (ControlNet + Nunchaku Qwen Image + LoRA stack + EasyCache) complete normally on ComfyUI v0.44.0 - no
RuntimeError, no fallback-latent output, an image produced on every run.
5. Action required
- Update this package to v2.6.8 (ComfyUI Manager / git pull) and restart ComfyUI.
- No workflow edits, no ComfyUI-core changes, no nunchaku changes, no launcher flags are needed.
6. Notes
- The five attributes in
_CONTROLNET_BORROWED_MODULESare the module attributes Qwen Image ControlNet accesses onbase_modeltoday (comfy/ldm/qwen_image/controlnet.py:143-162;process_imgis a method call and needs no contract). If upstream adds more borrowed attributes, add them to the same tuple. - The workflow's purge nodes and ComfyUI's node-output cache are left untouched; with the contract in place they no longer matter for this path.