FLUX Models Ignore Precision Arguments and Always Default to bfloat16 on RTX 40 Series

[SYN-Sunyata]

Custom Node Testing

• I have tried disabling custom nodes and the issue persists (see how to disable custom nodes if you need help)

Your question

OS: Windows 11
GPU: NVIDIA RTX 4060 Ti 16GB
ComfyUI Version: Latest at this date

Hi everyone,

I'm experiencing a behavior where all variants of the FLUX models (fp32, fp32_pruned, fp8) produce a pixel-perfect identical image. The generation time per iteration is also identical, regardless of the model's native precision.

This seems to be caused by a forced manual cast to bfloat16 (not sure at 100% this is the main cause), as shown in the log, which occurs even when trying to manually override the precision.

What I've Tried:
Default Launch: Loading any FLUX model (fp32, fp8, etc.) results in the bfloat16 cast.
Forcing Precision with Arguments: Launching with command-line arguments like --fp32-unet or --fp8_e4m3fn-unet has no effect. The log still shows the model is cast to bfloat16.
Clean Installations: The behavior is identical on a fresh, clean portable install from the official website, confirming it's not an issue with my setup or custom nodes.

Here some visual explanation of the problem.
[](https://www.reddit.com/r/comfyui/comments/1nsq3gc/exact_same_result_bug_with_fp32_an_fp8_models/)

Expected Behavior:
The model should run in its native precision, or the precision forced by the command-line argument. The output images and performance should differ between fp8 and fp32 versions.

Why this is a problem:

This forced casting to bfloat16 creates a situation where the specific benefits of using different model precisions are completely negated, defeating the user's objectives in both scenarios:

1)A user choosing an FP32 model does so to achieve the highest possible quality and mathematical precision. This objective is undermined when the model is automatically downcast to the less precise bfloat16 format.

2)Conversely, a user choosing an FP8 model is aiming for maximum speed and the lowest possible memory footprint. This objective is defeated when the model is upcast to the significantly heavier bfloat16 format for computation.

As a result, neither of the desired outcomes—maximum quality or maximum performance—is achievable. All model variants are funneled into a single "one-size-fits-all" execution path, which nullifies the very purpose of creating and distributing these specialized model files.

Additional Investigation:
A deep dive into the source code suggests this is a hard-coded behavior. The Flux class in supported_models.py explicitly lists [torch.bfloat16, torch.float16, torch.float32] as the only supported_inference_dtypes, excluding FP8. It seems this rule is so stringent that it even overrides the command-line arguments, likely within the FLUX-specific node loading logic itself.

model weight dtype torch.float8_e4m3fn, manual cast: torch.bfloat16
model_type FLUX

Anyway, this lines are the same for any flux models.

Is this intentional for stability on FLUX models, or is there a way to truly run them in their native FP8 precision to leverage their full potential on compatible hardware like the RTX 40 series?

Thanks in advance for any information.

Logs

(venv) PS C:\Users\user\Documents\depannage\ComfyUI> python main.py --fp8_e4m3fn-unet
[START] Security scan
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2025-09-28 20:31:23.443
** Platform: Windows
** Python version: 3.12.10 (tags/v3.12.10:0cc8128, Apr  8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
** Python executable: C:\Users\kgero\Documents\depannage\ComfyUI\venv\scripts\python.exe
** ComfyUI Path: C:\Users\kgero\Documents\depannage\ComfyUI
** ComfyUI Base Folder Path: C:\Users\kgero\Documents\depannage\ComfyUI
** User directory: C:\Users\kgero\Documents\depannage\ComfyUI\user
** ComfyUI-Manager config path: C:\Users\kgero\Documents\depannage\ComfyUI\user\default\ComfyUI-Manager\config.ini
** Log path: C:\Users\kgero\Documents\depannage\ComfyUI\user\comfyui.log

Prestartup times for custom nodes:
   0.0 seconds: C:\Users\kgero\Documents\depannage\ComfyUI\custom_nodes\rgthree-comfy
   1.8 seconds: C:\Users\kgero\Documents\depannage\ComfyUI\custom_nodes\ComfyUI-Manager

C:\Users\user\Documents\depannage\ComfyUI\venv\Lib\site-packages\torch\cuda\__init__.py:61: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
Checkpoint files will always be loaded safely.
Total VRAM 16380 MB, total RAM 32449 MB
pytorch version: 2.7.1+cu128
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4060 Ti : cudaMallocAsync
Using pytorch attention
Python version: 3.12.10 (tags/v3.12.10:0cc8128, Apr  8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
ComfyUI version: 0.3.60
ComfyUI frontend version: 1.26.13
[Prompt Server] web root: C:\Users\kgero\Documents\depannage\ComfyUI\venv\Lib\site-packages\comfyui_frontend_package\static
[Crystools INFO] Crystools version: 1.27.3
[Crystools INFO] Platform release: 11
[Crystools INFO] JETSON: Not detected.
[Crystools INFO] CPU: AMD Ryzen 5 7500F 6-Core Processor - Arch: AMD64 - OS: Windows 11
[Crystools INFO] pynvml (NVIDIA) initialized.
[Crystools INFO] GPU/s:
[Crystools INFO] 0) NVIDIA GeForce RTX 4060 Ti
[Crystools INFO] NVIDIA Driver: 581.29
### Loading: ComfyUI-Manager (V3.37)
[ComfyUI-Manager] network_mode: public
### ComfyUI Revision: 3951 [b8730510] *DETACHED | Released on '2025-09-23'
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json

[rgthree-comfy] Loaded 48 fantastic nodes. 🎉


Import times for custom nodes:
   0.0 seconds: C:\Users\kgero\Documents\depannage\ComfyUI\custom_nodes\websocket_image_save.py
   0.2 seconds: C:\Users\kgero\Documents\depannage\ComfyUI\custom_nodes\rgthree-comfy
   0.3 seconds: C:\Users\kgero\Documents\depannage\ComfyUI\custom_nodes\ComfyUI-Crystools
   0.4 seconds: C:\Users\kgero\Documents\depannage\ComfyUI\custom_nodes\ComfyUI-Manager

Context impl SQLiteImpl.
Will assume non-transactional DDL.
No target revision found.
Starting server

To see the GUI go to: http://127.0.0.1:8188
FETCH ComfyRegistry Data: 5/98
FETCH ComfyRegistry Data: 10/98
FETCH ComfyRegistry Data: 15/98
FETCH ComfyRegistry Data: 20/98
FETCH ComfyRegistry Data: 25/98
FETCH ComfyRegistry Data: 30/98
FETCH ComfyRegistry Data: 35/98
FETCH ComfyRegistry Data: 40/98
FETCH ComfyRegistry Data: 45/98
FETCH ComfyRegistry Data: 50/98
FETCH ComfyRegistry Data: 55/98
FETCH ComfyRegistry Data: 60/98
FETCH ComfyRegistry Data: 65/98
FETCH ComfyRegistry Data: 70/98
FETCH ComfyRegistry Data: 75/98
FETCH ComfyRegistry Data: 80/98
FETCH ComfyRegistry Data: 85/98
FETCH ComfyRegistry Data: 90/98
FETCH ComfyRegistry Data: 95/98
FETCH ComfyRegistry Data [DONE]
[ComfyUI-Manager] default cache updated: https://api.comfy.org/nodes
FETCH DATA from: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json [DONE]
[ComfyUI-Manager] All startup tasks have been completed.
got prompt
Using pytorch attention in VAE
Using pytorch attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
Requested to load FluxClipModel_
loaded completely 9.5367431640625e+25 9319.23095703125 True
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cuda:0, dtype: torch.float16
clip missing: ['text_projection.weight']
model weight dtype torch.float8_e4m3fn, manual cast: torch.bfloat16
model_type FLUX
Requested to load Flux
loaded completely 13474.938900817871 11350.067443847656 True
100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [00:58<00:00,  2.36s/it]
Requested to load AutoencodingEngine
loaded completely 611.5643920898438 159.87335777282715 True
Prompt executed in 90.90 seconds

Other

No response

[rulebreakerdude]

1. Run with the --fast option enabled
2. If still experiencing issue, update Pytorch to nightly 2.10.0+cu130 (Cuda 13) and re-run with --fast --disable-all-custom-nodes.
   (The bug you are experiencing can be caused by NVIDIA drivers updating above v581, I faced the same issue)

Also note, that there will still be a cast to torch.float16 (instead of torch.bfloat16) because even though fp8 matrix multiplication is supported, the accumulation is done in torch.float16 at a minimum. The --fast option will enable fast_fp16_accumulation which will also help with inference times.

Tested on 4060 with 8GB VRAM with flux fp8 and generation times went down from 3.7s/it to 2.9s/it. (With 20 iterations, thats 1 image per minute)

python .\main.py --fast --disable-all-custom-nodes
Checkpoint files will always be loaded safely.
Total VRAM 8188 MB, total RAM 65298 MB
pytorch version: 2.10.0.dev20250928+cu130
Enabled fp16 accumulation.
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4060 : cudaMallocAsync
Using pytorch attention
torchaudio missing, ACE model will be broken
torchaudio missing, ACE model will be broken
Python version: 3.12.9 (tags/v3.12.9:fdb8142, Feb  4 2025, 15:27:58) [MSC v.1942 64 bit (AMD64)]
ComfyUI version: 0.3.60
ComfyUI frontend version: 1.26.13

Context impl SQLiteImpl.
Will assume non-transactional DDL.
No target revision found.
Starting server

To see the GUI go to: http://127.0.0.1:8188
got prompt
model weight dtype torch.float8_e4m3fn, manual cast: torch.float16
model_type FLUX
Using pytorch attention in VAE
Using pytorch attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load FluxClipModel_
loaded completely 5677.8 4777.53759765625 True
Requested to load Flux
loaded partially 5438.066919891357 5437.908264160156 0
100%|███████| 20/20 [00:58<00:00,  2.91s/it]
Requested to load AutoencodingEngine
loaded completely 205.259765625 159.87335777282715 True

[ailthrim]

I've tried these exact tweaks. Like you said, it still casts to float16. It's slightly faster but it's not actually saving us the VRAM, so it's still not good enough. I really hope we can get comfyui to just use the fp8 weights on cards that support it. The fp8_e4m3fn is still doubled in size with the manual cast to torch.float16. My 11.1 GB model still becomes 22.2 GB just like the native float16 version. This is with a 4070 TI OC 12 GB:

--supports-fp8-compute --fast fp16_accumulation fp8_matrix_mult

torch 2.10.0.dev20251026+cu130

| NVIDIA-SMI 581.57                 Driver Version: 581.57         CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4070 Ti   WDDM  |   00000000:01:00.0 Off |                  N/A |
|  0%   37C    P0             31W /  285W |     857MiB /  12282MiB |      1%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

It's sometimes easier for me to use the native float16 version to avoid the minimal overhead of upcasting for no reason despite it sharding over PCIe.

[github-actions]

This issue is being marked stale because it has not had any activity for 30 days. Reply below within 7 days if your issue still isn't solved, and it will be left open. Otherwise, the issue will be closed automatically.