How to use gpu when clip encoding?

[bbaudio-2025]

Custom Node Testing

• I have tried disabling custom nodes and the issue persists (see how to disable custom nodes if you need help)

Your question

I have set 'device' as 'default' in the node 'Load CLIP', but it seemed that comfyui was using CPU to encode, and was very slow. There is no 'GPU' or 'CUDA' in device options, how can I switch to GPU for text encoding?

Logs

Using scaled fp8: fp8 matrix mult: False, scale input: False
Requested to load WanTEModel
loaded completely; 95367431640625005117571072.00 MB usable, 6419.48 MB loaded, full load: True
CLIP/text encoder model load device: cpu, offload device: cpu, current: cpu, dtype: torch.float16

Other

No response

[bbaudio-2025]

Temporary solution, modify nodes.py under /comfyui , in line 948

        if device == "cpu":
            model_options["load_device"] = model_options["offload_device"] = torch.device("cpu")
        else:
            model_options["load_device"] = model_options["offload_device"] = torch.device("cuda")

[sebas1111111]

Thank you so much for this! In my case I had to write "cuda:0" at the end instead of just "cuda" for it to work.

Temporary solution, modify nodes.py under /comfyui , in line 948

        if device == "cpu":
            model_options["load_device"] = model_options["offload_device"] = torch.device("cpu")
        else:
            model_options["load_device"] = model_options["offload_device"] = torch.device("cuda")

[bbaudio-2025]

Thank you so much for this! In my case I had to write "cuda:0" at the end instead of just "cuda" for it to work.

Temporary solution, modify nodes.py under /comfyui , in line 948

        if device == "cpu":
            model_options["load_device"] = model_options["offload_device"] = torch.device("cpu")
        else:
            model_options["load_device"] = model_options["offload_device"] = torch.device("cuda")

Yep if you have multiple GPUs

[bbaudio-2025]

But there is another problem, after text encoding with GPU, the CLIP model doesn't offload from VRAM, make the available VRAM decreased for generation steps.

[alFrame]

I am seeing this as well.

loaded completely; 95367431640625005117571072.00 MB usable, 9319.23 MB loaded, full load: True
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16

The above fix also works for me. But I wonder about the large memory number available on my gigantuous GPU, I don't have...

And ComfyUI core dumps past v3.68 of ComfyUI especially using FLUX. I have not tried 3.75 yet.

Is this somehow acknowledged as a bug and addressed? Or is it a Cuda/Pytorch version issue?

P.S. Sorry, did not mean to hijack the thread. Just didn't see this memory usage mentioned anywhere else...

P.P.S. I have now test run ComfyUI 3.75, while I do still see the gigantic GPU memory number, it doesn't core dump on a simple FLUX workflow, just using ComfyUI core nodes.

[github-actions]

This issue is being marked stale because it has not had any activity for 30 days. Reply below within 7 days if your issue still isn't solved, and it will be left open. Otherwise, the issue will be closed automatically.

[hum-ma]

@bbaudio-2025 was this actually resolved for you in ComfyUI?

If it's still an issue, do you have an older GPU? There's a function in comfy/model_management.py called should_use_fp16() which seems to assume that all old GPUs (compute major version < 6) cannot load a model in float16 so it ends up on the CPU. Also all of Nvidia 10x0 series on Linux.

I was thinking of creating a new issue or a small PR to get this fixed because otherwise we have to keep either manually patching ComfyUI with every new release or use a custom node just to get CLIP loaded on GPU correctly. But it might be good to have a relevant open issue.

[TheElo]

Also have that issue with the new ComfyUI, but remember it worked on the old version

[bbaudio-2025]

@bbaudio-2025 was this actually resolved for you in ComfyUI?

If it's still an issue, do you have an older GPU? There's a function in comfy/model_management.py called should_use_fp16() which seems to assume that all old GPUs (compute major version < 6) cannot load a model in float16 so it ends up on the CPU. Also all of Nvidia 10x0 series on Linux.

I was thinking of creating a new issue or a small PR to get this fixed because otherwise we have to keep either manually patching ComfyUI with every new release or use a custom node just to get CLIP loaded on GPU correctly. But it might be good to have a relevant open issue.

My GPU is 3090ti, it supports both FP16 and BF16. I hope this information is helpful to you.
It seems the problem has been solved in recent version.

[robustini]

Same problem here with RTX 5090, latest version of ComfyUI, not fixed.

[auf1r2]

Same here on Strix Halo. From some update it gets broken. For Flux.2 workflow I fixed it by using GGUF Clip loader. But this solution doesn't work for Krea2 / Ideogram.4 ;(