What ComfyUI startup command line parameters should I set to make the NVIDIA driver's “Prefer System Fallback” setting work correctly, so that what doesn't fit in VRAM is offloaded to RAM during workflow execution?

[rollingcookies]

Your question

Hello!

System
Windows 10
Nvidia driver 555.36
Total VRAM 12288 MB, total RAM 65409 MB
pytorch version: 2.2.0+cu121
xformers version: 0.0.24
ComfyUI: 2921e83063

VRAM is sufficient for most tasks, but if I use Redux and then enable IPadapter v2 for Flux, OOM happens.

I managed to get it to work on the startup parameters:
.\python_embeded\python.exe -s ComfyUI\main.py --windows-standalone-build --disable-auto-launch --disable-cuda-malloc --force-fp16

But that's only because everything fits in VRAM and there is no OOM crash.

The --force-fp16 setting also affects the quality of the VAE output. I would like to run with --fp32-vae, but now it causes OOM.

I see a solution in setting correct unloading partially from VRAM > RAM, but it works rarely, often just crashes with OOM.

Could you please tell me the correct settings for the command line parameters so that unloading to RAM works and also VAE is in fp32.

“Prefer System Fallback” in the Nvidia driver settings for python.exe is enabled!

Logs

No response

Other

No response

[github-actions]

This issue is being marked stale because it has not had any activity for 30 days. Reply below within 7 days if your issue still isn't solved, and it will be left open. Otherwise, the issue will be closed automatically.