How does this work: "Always put text encoder on GPU when dynamic vram on."

[SRStwo]

For people with systems that have inadequate VRAM the solution has been to have the text encoder stay in main system RAM and use the CPU, right?

On a system with 64 GB of system RAM and a 12 GB GPU, how does it make sense to put the text encoder on the GPU?

[chaObserv]

Partial modules are transferred to and offloaded from VRAM during computation as usual. Before that PR, some devices were forced to run text encoders on the CPU by default. It just made them work like the main diffusion/flow model, which can run on the GPU even when the entire model cannot fit into VRAM.

[SRStwo]

What about scenarios where it seems to make more sense to leave the main model in VRAM and the text encoder in system RAM? For example, on a system with 16 GB of VRAM and the main model fits in VRAM but the text encoder does not, then why would one want to evict the main model for every generation, especially if one has a fast CPU like a 9800X3D and plenty of system RAM?

Is running a text encoder on the CPU so much slower that it's worth moving a main model out of VRAM every time — even if the text encoder is being run at 32-bit precision? (I assume disk swap space would never be used to accomplish this, instead it would stream (disk reads not writes) from the SSD to both VRAM and system RAM if the text encoder is too large to fit in VRAM.)

With Wan, if someone wants to they should be able to put both the high and low models into VRAM on a 5090 (using INT8convrot models) and keep them there, keeping the text encoder and VAE in system RAM to run those at FP32 precision. With that, nothing has to move for each generation. (My extremely brief testing hasn't impressed me with INT8convrot for Wan 2.2. t2v but for this example let's assume it doesn't have as severe a quality problem as I seem to have found.)

Or, with H3, someone could fit the model into VRAM on some systems but not the large text encorder, especially if they want to use a higher precision for the text encoder.

This issue seems to be particularly prominent on systems with low-VRAM GPUs and plenty of system RAM. Someone might want to run, for instance Q8_0 GGUF (which, from my very brief testing is usable for quality versus the unusable for quality INT8convrot, at least for t2v) Wan 2.2 on a 12 GB VRAM GPU and keep the text encoder on the CPU to run it at 16-bit precision or 32-bit precision (depending upon how much system RAM one has).

[chaObserv]

Load Clip nodes (single and dual) in core should have a device option for CPU. That would allow the text encoder to run on the CPU if needed. The PR just made the default one able to run on the GPU instead.

Text encoders for recent models are LLMs now. On my devices, running TextEncodeAceStepAudio1.5 in the default template with GPU is 5–6x faster than running it on CPU. As another example, there is an execution time reference for text encoding in MiniMax Music 3: #15607 (comment). In your case, even if the main model is kept in VRAM, the text encoder on CPU may become the bottleneck when switching prompts. It might need to be tested, though.

[159753a52]

Some numbers for the 16 GB case you describe, using Qwen-Image 2.1 at 1024×1024 as the example (file sizes are exact, peaks come from reported runs):

• Q4_K_M DiT (3.9 GiB) + INT8 Qwen3-VL-8B encoder (8.7 GiB) + BF16 VAE, everything resident: 13.2 GiB of weights, 15.3–17.9 GiB peak. That's exactly the "the main model fits, both together don't" situation on a 16 GB card.
• INT8 DiT + INT8 encoder with the encoder unloaded after the prompt: 11.4–14.0 GiB peak.
• Encoder on the CPU: 6.1–6.9 GiB peak (an RTX 3090 with --lowvram measured 6.14 GiB).

Two practical points:

1. The swap only costs you when the prompt changes. If you requeue with the same prompt, the text-encode output is normally cached, so the encoder doesn't run and the DiT stays loaded.
2. With dynamic VRAM on, --lowvram on its own no longer moves the encoder to the CPU; you need --disable-dynamic-vram --lowvram globally, or the CPU device option on the Load CLIP node that chaObserv mentioned, per workflow.

The trade-off gets much worse with big encoders: MiniMax H3's Qwen3-VL-32B is 14.6 GiB even as NVFP4, which is a lot of CPU work per prompt; an 8B encoder is far more tolerable. Timing both on your machine is the only way to know.

I keep these per-file tables here (I built it): https://modelvram.com/qwen-image-2-1-vram-calculator/

[stanleys12]

The text encoder does not have to fit in your 12 GB. With dynamic VRAM the weights stay in system RAM and are streamed to the GPU as each layer is needed, the same way the diffusion model is handled. So only the compute moves to the GPU, which is much faster than running the encoder on the CPU. This matches what chaObserv said.

The change is in text_encoder_device():

ComfyUI/comfy/model_management.py

Lines 1219 to 1230 in b87fe48

	def text_encoder_device():
	if args.gpu_only:
	return get_torch_device()
	if comfy.memory_management.aimdo_enabled:
	return get_torch_device()
	elif vram_state in (VRAMState.HIGH_VRAM, VRAMState.NORMAL_VRAM):
	if should_use_fp16(prioritize_performance=False):
	return get_torch_device()
	else:
	return torch.device("cpu")
	else:
	return torch.device("cpu")

if args.gpu_only:
    return get_torch_device()
if comfy.memory_management.aimdo_enabled:
    return get_torch_device()
elif vram_state in (VRAMState.HIGH_VRAM, VRAMState.NORMAL_VRAM):
    if should_use_fp16(prioritize_performance=False):
        return get_torch_device()
    else:
        return torch.device("cpu")
else:
    return torch.device("cpu")

Before commit d39cdfd (#16374), dynamic VRAM was OR'd with the HIGH/NORMAL VRAM check, so it still went through the should_use_fp16 test and could fall back to the CPU. Now dynamic VRAM (aimdo) returns the GPU unconditionally.

The encoder is also not loaded onto the GPU up front. In text_encoder_initial_device() (

ComfyUI/comfy/model_management.py

Lines 1232 to 1234 in b87fe48

	def text_encoder_initial_device(load_device, offload_device, model_size=0):
	if comfy.memory_management.aimdo_enabled:
	return offload_device

), when dynamic VRAM is on the initial device is the offload device, which is the CPU unless --gpu-only is set. The weights start in RAM and are pulled to the GPU on demand.

Without dynamic VRAM the old behaviour is unchanged, so the CPU is still used in low VRAM modes.

(I looked this up in the code with help from an AI assistant; the links are to the lines I checked.)