Nodes loading ok but workflows not (SUPIR model loader v2 / SUPIR conditioner errors)

[GastonSan79]

First at all, thanks for your work.
I'm completely lost here, the nodes load into comfyui ok, but loading the example workflow from the custom node subfolder gives the next error:

Error occurred when executing SUPIR_model_loader_v2:

Failed to load first clip model from SDXL checkpoint

  File "H:\AI\COMFY-PORTABLE\ComfyUI\execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "H:\AI\COMFY-PORTABLE\ComfyUI\execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "H:\AI\COMFY-PORTABLE\ComfyUI\custom_nodes\ComfyUI-0246\utils.py", line 381, in new_func
    res_value = old_func(*final_args, **kwargs)
  File "H:\AI\COMFY-PORTABLE\ComfyUI\execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "H:\AI\COMFY-PORTABLE\ComfyUI\custom_nodes\ComfyUI-SUPIR\nodes_v2.py", line 925, in process
    raise Exception("Failed to load first clip model from SDXL checkpoint")

The loaded model is a SDXL.
If I replicate the old example workflow from the repository, the models load ok but at the SUPIR conditioner stage I get the error:

Error occurred when executing SUPIR_conditioner:

Cannot copy out of meta tensor; no data!

  File "H:\AI\COMFY-PORTABLE\ComfyUI\execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "H:\AI\COMFY-PORTABLE\ComfyUI\execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "H:\AI\COMFY-PORTABLE\ComfyUI\custom_nodes\ComfyUI-0246\utils.py", line 381, in new_func
    res_value = old_func(*final_args, **kwargs)
  File "H:\AI\COMFY-PORTABLE\ComfyUI\execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "H:\AI\COMFY-PORTABLE\ComfyUI\custom_nodes\ComfyUI-SUPIR\nodes_v2.py", line 571, in condition
    SUPIR_model.conditioner.to(device)
  File "H:\AI\COMFY-PORTABLE\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1160, in to
    return self._apply(convert)
  File "H:\AI\COMFY-PORTABLE\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    module._apply(fn)
  File "H:\AI\COMFY-PORTABLE\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    module._apply(fn)
  File "H:\AI\COMFY-PORTABLE\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    module._apply(fn)
  File "H:\AI\COMFY-PORTABLE\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 833, in _apply
    param_applied = fn(param)
  File "H:\AI\COMFY-PORTABLE\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1158, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)

All the nodes are correctly connected.
Python version: 3.10.11 / ComfyUI Revision: 2021 [36f7face] | Released on '2024-02-26

Thanks in advance.

[Amit30swgoh]

Error occurred when executing SUPIR_sample:

No operator found for memory_efficient_attention_forward with inputs:
query : shape=(40, 126, 1, 64) (torch.float16)
key : shape=(40, 126, 1, 64) (torch.float16)
value : shape=(40, 126, 1, 64) (torch.float16)
attn_bias :
p : 0.0
decoderF is not supported because:
xFormers wasn't build with CUDA support
attn_bias type is
operator wasn't built - see python -m xformers.info for more info
flshattF@0.0.0 is not supported because:
xFormers wasn't build with CUDA support
operator wasn't built - see python -m xformers.info for more info
cutlassF is not supported because:
xFormers wasn't build with CUDA support
operator wasn't built - see python -m xformers.info for more info
smallkF is not supported because:
max(query.shape[-1] != value.shape[-1]) > 32
xFormers wasn't build with CUDA support
dtype=torch.float16 (supported: {torch.float32})
operator wasn't built - see python -m xformers.info for more info
unsupported embed per head: 64

File "/content/drive/MyDrive/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/content/drive/MyDrive/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-0246/utils.py", line 381, in new_func
res_value = old_func(*final_args, **kwargs)
File "/content/drive/MyDrive/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/nodes_v2.py", line 481, in sample
_samples = self.sampler(denoiser, noised_z, cond=positive[i], uc=negative[i], x_center=sample.unsqueeze(0), control_scale=control_scale_end,
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/sampling.py", line 441, in call
x = self.sampler_step(
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/sampling.py", line 418, in sampler_step
denoised = self.denoise(x, denoiser, sigma_hat, cond, uc, control_scale=control_scale)
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/sampling.py", line 400, in denoise
denoised = denoiser(*self.guider.prepare_inputs(x, sigma, cond, uc), control_scale)
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/nodes_v2.py", line 455, in
denoiser = lambda input, sigma, c, control_scale: SUPIR_model.denoiser(SUPIR_model.model, input, sigma, c, control_scale)
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/denoiser.py", line 73, in call
return network(input * c_in, c_noise, cond, control_scale) * c_out + input * c_skip
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/wrappers.py", line 96, in forward
out = self.diffusion_model(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/SUPIR/modules/SUPIR_v0.py", line 654, in forward
h = self.project_modules[adapter_idx](control[control_idx], h, control_scale=control_scale)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/SUPIR/modules/SUPIR_v0.py", line 147, in forward
x = self.attn(x, context)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/attention.py", line 365, in forward
out = xformers.ops.memory_efficient_attention(
File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/init.py", line 247, in memory_efficient_attention
return _memory_efficient_attention(
File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/init.py", line 365, in _memory_efficient_attention
return _memory_efficient_attention_forward(
File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/init.py", line 381, in _memory_efficient_attention_forward
op = _dispatch_fw(inp, False)
File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/dispatch.py", line 125, in _dispatch_fw
return _run_priority_list(
File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/dispatch.py", line 65, in _run_priority_list
raise NotImplementedError(msg)

[tapistefan]

Error occurred when executing SUPIR_model_loader_v2:

Try going into Manager and "Update All". It fixed the error for me.

[kijai]

I don't really know what could cause this, I can't replicate it, current version with the example workflow runs fine for me.

[Sostay]

Exactly the same situation, when I use the model of fp16.

[SRagy]

I also have issues with these nodes, although the error messages are not the same. On the first time I queue something, I get the following error for the loader:

Error occurred when executing SUPIR_model_loader_v2:

Failed to load SDXL model

File "/notebooks/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/notebooks/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/notebooks/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/notebooks/ComfyUI/custom_nodes/ComfyUI-SUPIR/nodes_v2.py", line 893, in process
raise Exception("Failed to load SDXL model")

On the second time, this error disappears, which is a bit perplexing, but I become subsequently stuck at the conditioner with the following error:

Error occurred when executing SUPIR_conditioner:

'NoneType' object is not callable

File "/notebooks/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/notebooks/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/notebooks/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/notebooks/ComfyUI/custom_nodes/ComfyUI-SUPIR/nodes_v2.py", line 617, in condition
_c, _uc = SUPIR_model.conditioner.get_unconditional_conditioning(cond, uncond)
File "/notebooks/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/encoders/modules.py", line 187, in get_unconditional_conditioning
c = self(batch_c)
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/notebooks/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/encoders/modules.py", line 208, in forward
emb_out = embedder(batch[embedder.input_key])
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/notebooks/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/encoders/modules.py", line 490, in forward
batch_encoding = self.tokenizer(

This happens when running the sample workflow, so I don't think it's a problem with how I've set the nodes up. My python environment is potentially a bit janky though, because I use paperspace which has some inflexibility.

[moguldigital]

@SRagy Did you find some solution to this? I'm having this same issue.

[Sostay]

@SRagy Did you find some solution to this? I'm having this same issue.

It seems that there are some environment-dependent problems. I adjusted the network environment and sometimes it can work normally.

[moguldigital]

@SRagy I'm using runpod with 3090 gpu, can you tell what things you changed because I can't figure it out. I also don't know coding much but if you can give a direction of what to do then I can chatgpt my way through it.

[SRagy]

In the end I used the legacy node from V1, which worked fine for me.

[firofame]

seems duplicate of #72

[t1804330987]

最后，我使用了 V1 的遗留节点，这对我来说效果很好。

如何使用V1遗留节点，我也卡在这了