Dynamic VRAM + Symlinked Model causes model reload on every prompt after workflow/model switch (v0.23)

[toorn]

Bug Report: Dynamic VRAM causes model reload on every prompt after workflow/model switch

Regarding Custom Nodes

I was unable to test with custom nodes disabled, as the workflow requires them to function. However, based on the logs, the trigger point correlates exactly with a model dtype change (fp16 → bf16+nvfp4 mixed), not with any custom node behavior. The slowdown occurs at the ComfyUI core level (Dynamic VRAM model management), and I believe this is a core bug rather than a custom node issue.

Environment

• ComfyUI Version: v0.23.0-10-gbd7da053a (Released 2026-06-02)
• OS: Windows 11
• Python: 3.13.11
• PyTorch: 2.12.0+cu130
• GPU: NVIDIA GeForce RTX 5070 Ti (16GB VRAM)
• RAM: 64GB
• Driver: 610.47

Expected Behavior

After the first load, models (ZImageTEModel_, Lumina2) should remain staged in Dynamic VRAM and not reload on every prompt. Repeated runs with only a seed change should complete quickly (~23-28 seconds in my case).

Actual Behavior

After switching workflows or loading a different model mid-session, models begin reloading on every subsequent prompt. Once triggered, the slowdown persists for the rest of the session without recovery.

• Normal runs: ~23-28 seconds
• After trigger: ~120-190 seconds per run

Steps to Reproduce

1. Start ComfyUI with Dynamic VRAM enabled (no --disable-dynamic-vram)
2. Run a workflow several times — runs are fast after first load
3. Switch to a different workflow or load a model with different dtype/quantization (e.g. switching between fp16 and bf16/nvfp4 mixed models)
4. Return to original workflow
5. Observe that Requested to load now appears on every prompt and execution time increases 5-8x

Key Observation

The trigger appears to be a model dtype/quantization change mid-session. In the log below, normal fast runs are interrupted when a workflow switch causes:

[INFO] model weight dtype torch.bfloat16, manual cast: None
[INFO] model_type FLOW
[INFO] Requested to load ZImageTEModel_   ← starts appearing every run from this point

Before this point, Requested to load does NOT appear on repeated runs.

Note: This issue does NOT occur when using --disable-dynamic-vram. However, per the warning message, that flag is a temporary workaround and will be removed.

---

Log

Section 1: Normal behavior (fast runs after first load)

[INFO] got prompt
[INFO] Using pytorch attention in VAE
[INFO] Using pytorch attention in VAE
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
[INFO] Requested to load ZImageTEModel_
[INFO] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged. 0 patches attached. Force pre-loaded 145 weights: 383 KB.
[INFO] model weight dtype torch.float16, manual cast: None
[INFO] model_type FLOW
[INFO] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged. 0 patches attached. Force pre-loaded 145 weights: 383 KB.
[INFO] Found quantization metadata version 1
[INFO] Detected mixed precision quantization
[INFO] Using mixed precision operations
[INFO] Native ops: mxfp8, nvfp4, float8_e5m2, float8_e4m3fn
[INFO] model weight dtype torch.bfloat16, manual cast: torch.bfloat16
[INFO] model_type FLOW
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged. 0 patches attached. Force pre-loaded 265 weights: 1046 KB.
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged. 0 patches attached. Force pre-loaded 265 weights: 1046 KB.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 3/3 [00:07<00:00,  2.44s/it]
[INFO] Requested to load AutoencodingEngine
[INFO] 0 models unloaded.
[INFO] Model AutoencodingEngine prepared for dynamic VRAM loading. 159MB Staged.
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 11738MB Staged. 180 patches attached.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 6/6 [00:11<00:00,  1.87s/it]
[INFO] 0 models unloaded.
[INFO] Model AutoencodingEngine prepared for dynamic VRAM loading. 159MB Staged.
[INFO] Prompt executed in 167.80 seconds  ← first run (includes initial load)

[INFO] got prompt
[INFO] Requested to load ZImageTEModel_
[INFO] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged.
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 3/3 [00:07<00:00,  2.58s/it]
[INFO] 0 models unloaded.
[INFO] Model AutoencodingEngine prepared for dynamic VRAM loading. 159MB Staged.
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 11738MB Staged. 180 patches attached.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 6/6 [00:10<00:00,  1.82s/it]
[INFO] 0 models unloaded.
[INFO] Model AutoencodingEngine prepared for dynamic VRAM loading. 159MB Staged.
[INFO] Prompt executed in 23.16 seconds  ← fast

[INFO] got prompt
[INFO] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged.   ← no "Requested to load"
[INFO] 0 models unloaded.
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
[INFO] 0 models unloaded.
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 3/3 [00:12<00:00,  4.17s/it]
[INFO] 0 models unloaded.
[INFO] Model AutoencodingEngine prepared for dynamic VRAM loading. 159MB Staged.
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 11738MB Staged. 180 patches attached.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 6/6 [00:10<00:00,  1.82s/it]
[INFO] 0 models unloaded.
[INFO] Prompt executed in 27.95 seconds  ← fast, no "Requested to load" at all

Section 2: Trigger point — workflow/model switch causes dtype reinitialization

[INFO] got prompt
[INFO] model weight dtype torch.bfloat16, manual cast: None   ← model reinit triggered by workflow switch
[INFO] model_type FLOW
[INFO] Requested to load ZImageTEModel_   ← "Requested to load" returns
[INFO] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged.
[INFO] Found quantization metadata version 1
[INFO] Detected mixed precision quantization
[INFO] Using mixed precision operations
[INFO] Native ops: mxfp8, nvfp4, float8_e5m2, float8_e4m3fn
[INFO] model weight dtype torch.bfloat16, manual cast: torch.bfloat16
[INFO] model_type FLOW
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 3/3 [00:07<00:00,  2.62s/it]
[INFO] Requested to load AutoencodingEngine
[INFO] 0 models unloaded.
[INFO] Model AutoencodingEngine prepared for dynamic VRAM loading. 159MB Staged.
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 11738MB Staged. 180 patches attached.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 6/6 [01:52<00:00, 18.78s/it]   ← suddenly 18s per step vs 1.8s before
[INFO] Prompt executed in 186.56 seconds  ← 8x slower

Section 3: Slowdown persists — no recovery without restart

[INFO] got prompt
[INFO] Requested to load ZImageTEModel_   ← persists every run
[INFO] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged.
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 6/6 [01:52<00:00, 18.69s/it]
[INFO] Prompt executed in 183.31 seconds

[INFO] got prompt
[INFO] Requested to load ZImageTEModel_
...
[INFO] Prompt executed in 176.45 seconds

[INFO] got prompt
[INFO] Requested to load ZImageTEModel_
...
[INFO] Prompt executed in 124.70 seconds

[INFO] got prompt
[INFO] Requested to load ZImageTEModel_
...
[INFO] Prompt executed in 130.12 seconds

---

Additional Notes

• Issue does NOT reproduce on a fresh ComfyUI start for the first few runs
• Issue triggers reliably after switching workflows that involve different model dtypes (fp16 → bf16+nvfp4 mixed)
• Once triggered, only a full ComfyUI restart restores normal speed
• --disable-dynamic-vram prevents the issue but is documented as a temporary flag to be removed
• This behavior was NOT present in v0.22 with the same workflows and hardware

Workflow : Z+Z V3.zip

[coderabbitai]

🔗 Related PRs

#13371 - Add new ComfyUI blueprints and fix subgraph naming [merged]
#13570 - Add New Blueprints [merged]
#14052 - fixup threaded loader with worksplit multi-gpu [merged]
#14116 - Threaded Loader performance fixes / improvements (+ Aimdo 0.4.6) [merged]
#14183 - Revert deprecation of non-dynamic smart memory (CORE-152 (revert)) [merged]

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[toorn]

Additional finding - Root cause identified:

I have identified the root cause. The slowdown only occurs when loading models via symbolic links. Models stored as actual files load normally without any issues.

• Models loaded via symbolic links → triggers reload on every prompt (slow)
• Models loaded as actual files → stays cached, no reload (fast)
• Using --disable-dynamic-vram flag → no slowdown even with symbolic links

The symbolic links point to files on a different drive from ComfyUI. This worked correctly in v0.22 without any issues.

It appears that Dynamic VRAM in v0.23 does not correctly track/cache models loaded via symbolic links, causing it to treat them as new models on every prompt execution.

[rattus128]

Additional finding - Root cause identified:

I have identified the root cause. The slowdown only occurs when loading models via symbolic links. Models stored as actual files load normally without any issues.

* Models loaded via symbolic links → triggers reload on every prompt (slow)

* Models loaded as actual files → stays cached, no reload (fast)

* Using `--disable-dynamic-vram` flag → no slowdown even with symbolic links

The symbolic links point to files on a different drive from ComfyUI. This worked correctly in v0.22 without any issues.

It appears that Dynamic VRAM in v0.23 does not correctly track/cache models loaded via symbolic links, causing it to treat them as new models on every prompt execution.

windows have two types of symbolic links. Junctions and symlinks. Also can you double confirm the type and if the link is on an enclosing directory or the model file itself?

I routinely do windows cross drive directory junction so thats probably good, but if we can confirm your precise symlink topology we can maybe corner it.

[toorn]

Further investigation results:

After more testing, I can now provide a complete picture:

Behavior by link type:

• Real files → normal, model stays cached in VRAM ✅
• Directory junction → normal, model stays cached in VRAM ✅
• Symbolic links (both individual files AND folders) → triggers reload on every prompt ❌

Critical finding - session contamination:

Once a symbolic link model is loaded even once, the session becomes "contaminated". After that point, switching back to real files or junctions does NOT fix the issue — the reload behavior persists for the rest of the session. Only a full ComfyUI restart clears the problem.

This suggests that loading a symbolic link model corrupts some internal cache or tracking table in Dynamic VRAM, and that corrupted state persists until restart.

Summary:

• Symbolic links (file or folder) = broken
• Junctions = works fine
• Real files = works fine
• One symlink load = entire session broken until restart

[rattus128]

I cant repro this yet unfortunately. I symlinked locally in the folder. My model library is already cross-file-system from comfy itself behind a junction too.

I tried z_image_symlink x2, ideogram then z_image_symlink x2 again and the model stays hot in VRAM for me.

Your contamination almost sounds like a VRAM leak? Is it actually loading to VRAM on those unwanted reloads or running real slow in offload?

Can you install the ComfyUI-MemoryVisualizer and see if you can see the model loading patterns? It shows what is loaded at any given time. If theres a before and after video snap of the critical moment in time thats ideal.

Can you check if edit-> Unload Model or Unload Models and Cache are able to recover the session? Usually they do, if neither level recovers we have a very special form of global resource leak.

Also worth a --disable-all-custom-nodes run just to make sure we don't have a customs monkey patch in play.

[toorn]

I followed your suggestions and ran several tests.

When I tested with --disable-all-custom-nodes, everything ran without any problems. At that point, I assumed it was a custom node conflict and was quite disappointed, as I have too many custom nodes to check them individually.

After using Edit → Unload Models and Cache, the issue did not occur for about a day, so I initially reported it as resolved. However, the same issue has returned.

This time, Edit → Unload Models and Cache does not resolve the problem either.

I did record some video footage during the testing process, but have not uploaded it yet. Should I upload it now?

---

At this point, --disable-dynamic-vram is the only working solution for me. The model only takes time on the first load, and runs very fast after that. My custom nodes also work without any issues with this option. Without it, every run takes as long as the initial load.

I would like to completely remove v0.24 and revert to v0.22 to test, but it feels too daunting to attempt.

[brendanhoar]

I would like to completely remove v0.24 and revert to v0.22 to test, but it feels too daunting to attempt.

Anytime you want to return to the current git master state:
git switch - (or sometimes git switch master)
git pull

I used the following to move back to v0.2.2, after disabling any auto-updates I had:
git checkout 26515ac

If I need to test a particular pr:
e.g. if PR is number 1234
git switch - (or sometimes git switch master)
git pull
git fetch origin "pull/1234/head:pr/1234"
git switch pr/1234

[toorn]

I downgraded to v0.22. The issue is resolved.

After a symlinked model is loaded into VRAM, subsequent runs no longer reload the model and complete almost instantly. Compared to v0.24, image generation speed is approximately 3x faster.

Using ComfyUI-MemoryVisualizer, I noticed one clear difference. In v0.24, RAM usage barely changed when loading symlinked models (it stayed around 24GB from the start based on my recorded footage). In v0.22, RAM usage increases significantly (up to 47.7GB).

I am not sure what this means technically, but this is what I was able to observe. I apologize as I am not an expert.

In any case, it is satisfying to see images generated almost instantly without the model reloading on every run.

[brendanhoar]

Past v0.2.2, we need a startup flag for: --please-allow-comfyui-to-cache-model-files-or-tensors-in-RAM-and-stop-going-to-disk-constantly ... i've got 512GB of RAM here, so this multi-week performance tanking situation is just blowing my mind.