AMD graphics cards are forced to switch to FP16 precision mode when using FP8 models.

[PLASSICAxu]

Custom Node Testing

• I have tried disabling custom nodes and the issue persists (see how to disable custom nodes if you need help)

Expected Behavior

Use the FP8 model at FP8 precision
model weight dtype torch.float8_e4m3fn, manual cast: None

Actual Behavior

Models with FP8 precision are forced to load and compute at FP16 precision.
model weight dtype torch.float8_e4m3fn, manual cast: torch.bfloat16

Steps to Reproduce

model using FP8 can be reproduced; here I use Qwen's official image edit workflow.

Debug Logs

(.venv) (base) PS E:\ComfyUI-master> python main.py --use-pytorch-cross-attention --disable-all-custom-nodes                                                           
Checkpoint files will always be loaded safely.
Total VRAM 16304 MB, total RAM 32599 MB
pytorch version: 2.9.0+rocmsdk20251116
Set: torch.backends.cudnn.enabled = False for better AMD performance.
AMD arch: gfx1201
ROCm version: (7, 1)
Set vram state to: NORMAL_VRAM
Device: cuda:0 AMD Radeon RX 9070 XT : native
Enabled pinned memory 14669.0
Using pytorch attention
Python version: 3.12.9 (tags/v3.12.9:fdb8142, Feb  4 2025, 15:27:58) [MSC v.1942 64 bit (AMD64)]
ComfyUI version: 0.6.0
ComfyUI frontend version: 1.34.9
[Prompt Server] web root: E:\ComfyUI-master\.venv\Lib\site-packages\comfyui_frontend_package\static
Total VRAM 16304 MB, total RAM 32599 MB
pytorch version: 2.9.0+rocmsdk20251116
Set: torch.backends.cudnn.enabled = False for better AMD performance.
AMD arch: gfx1201
ROCm version: (7, 1)
Set vram state to: NORMAL_VRAM
Device: cuda:0 AMD Radeon RX 9070 XT : native
Enabled pinned memory 14669.0
Skipping loading of custom nodes
Context impl SQLiteImpl.
Will assume non-transactional DDL.
No target revision found.
Starting server

To see the GUI go to: http://127.0.0.1:8188
got prompt
Using split attention in VAE
Using split attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
Requested to load WanVAE
loaded completely; 12472.04 MB usable, 242.03 MB loaded, full load: True
Found quantization metadata version 1
Using MixedPrecisionOps for text encoder
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load QwenImageTEModel_
loaded completely; 14165.88 MB usable, 7910.29 MB loaded, full load: True
model weight dtype torch.float8_e4m3fn, manual cast: torch.bfloat16
model_type FLUX
Requested to load QwenImage
Interrupting prompt 44a39768-fd55-43e3-8319-771d3839678e
loaded partially; 14136.05 MB usable, 14026.95 MB loaded, 5457.00 MB offloaded, 108.01 MB buffer reserved, lowvram patches: 0

Other

I know the author previously added FP8 support for AMD graphics cards based on the GFX 1201 architecture, but it doesn't seem to be working.

[jahu00]

I;m trying to figure out this myself. I see multiple issue related to this problem (#10158 and #8242). It seems there used to be a flag --supports-fp8-compute, but it should be unnecessary now for gfx1201. From what I've read in other issues, fp8 might have worked with ROCm 6.4, but doesn't with 7 for some reason.

Just to make sure if it works or not, I'll have to do some tests with --supports-fp8-compute and --fast flags.

[jahu00]

After some more tests, I'm not sure if any flags are necessary. Models seem to load into VRAM as fp8 so take only as much memory as the file (this also worked when I disabled automatic fp8 support through code). There used to be an issue on RDNA2/3 where model would load as fp16 (so would take double the size in VRAM). Might be old news, haven't tested this in ages.

The most I can get out of those fp8 models is by using fp8_e4m3fn_fast weight_type with Load Diffusion Model node.

With everything disabled I was getting 35s on 1024x1024 20 step Flux1 dev image gen on my R9700. But when using fp8_e4m3fn_fast I get 30s. I verified this with all kinds of combination of flags and disabling the code that automatically enables fp8 on RDNA4. I've expected a more dramatic increase in speed (fp8 should have 2x the TFLOPS of fp16), but I will take whatever is available.

fp8_e4m3fn_fast changes the output somewhat compared to fp8_e4m3fn. Maybe this can be mitigated by using a scaled model (not sure if mine are or not).

I haven't compared it to fp16/bf16 yet.

EDIT: fp16 model loaded as bf16 without a cast (maybe it was bf16 all along) taking a significant chunk of VRAM. Generating 20 step Flux1 dev took only slightly above 36s (that 1s might be due to partial load).

In short:

• fp8 support is enabled by default on gfx1201 (flag --supports-fp8-compute is not needed).
• flag --fast seems to force the same mode that is used with fp8_e4m3fn_fast weight_type (fp8_e4m3fn becomes fp8_e4m3fn_fast), so isn't needed either

[jahu00]

After some more tests, it appears that fp8 currently runs at the same speed as fp/bf16 for AMD GPUs outside of fp8_matrix_mult optimization which gives some 15% boost in speed. This worked for me with Flux1 and Z-Image Turbo, but made no difference for Qwen Image.

[HDXUT]

Same problem,recognize that when i use fp8 model and start it the VRAM and RAM usage have surged to fp16 levels

[hakan4vg]

Bumping this with technical details - RX 9070 XT owner hitting this exact issue

1. Code Location
The AMD FP8 block is in comfy/model_management.py, function supports_fp8_compute() (around line 350):

if not is_nvidia():
    return False  # This forces all AMD GPUs to BF16 fallback

This blanket NVIDIA check prevents RDNA4 (gfx1201) from using FP8 compute despite hardware support.

2. Hardware Capability Proof
ROCm 7.2 includes FP8 primitives for gfx1201. The vLLM team proved this works by forcing the MI300X code path on consumer RDNA4 cards:

• Reference: https://blog.vllm.ai/2025/02/24/ptpc-fp8-rocm.html
• vLLM treats gfx1201 as MI300X for FP8 operations and achieves working FP8 inference on RX 9070 XT hardware
• This confirms the silicon supports FP8, but ROCm artificially gates it behind data-center-only detection

3. Current Situation

• Linux bare metal: FP8 works with hacks (vLLM approach), but ComfyUI's is_nvidia() check prevents users from accessing it
• Windows: ROCm 7.2 lacks FP8 compute exposure for gfx1201 entirely (storage/dequant only, no compute kernels)
• ComfyUI: Currently forces manual cast: torch.bfloat16 for all AMD GPUs, wasting 2x VRAM and compute performance

This would enable Linux users to access working FP8 immediately, and be ready for Windows when AMD enables it in ROCm 7.3.

[leovanalphen]

Bumping this with technical details - RX 9070 XT owner hitting this exact issue

1. Code Location The AMD FP8 block is in comfy/model_management.py, function supports_fp8_compute() (around line 350):

if not is_nvidia():
    return False  # This forces all AMD GPUs to BF16 fallback

This blanket NVIDIA check prevents RDNA4 (gfx1201) from using FP8 compute despite hardware support.

2. Hardware Capability Proof ROCm 7.2 includes FP8 primitives for gfx1201. The vLLM team proved this works by forcing the MI300X code path on consumer RDNA4 cards:

* Reference: https://blog.vllm.ai/2025/02/24/ptpc-fp8-rocm.html

* vLLM treats `gfx1201` as `MI300X` for FP8 operations and achieves working FP8 inference on RX 9070 XT hardware

* This confirms the silicon supports FP8, but ROCm artificially gates it behind data-center-only detection

3. Current Situation

* **Linux bare metal**: FP8 works with hacks (vLLM approach), but ComfyUI's `is_nvidia()` check prevents users from accessing it

* **Windows**: ROCm 7.2 lacks FP8 compute exposure for gfx1201 entirely (storage/dequant only, no compute kernels)

* **ComfyUI**: Currently forces `manual cast: torch.bfloat16` for all AMD GPUs, wasting 2x VRAM and compute performance

This would enable Linux users to access working FP8 immediately, and be ready for Windows when AMD enables it in ROCm 7.3.

This is not entirely correct — at least the “This specific function in ComfyUI blocks FP8” part.

In the current ComfyUI code, supports_fp8_compute() starts with:

def supports_fp8_compute(device=None):
    if SUPPORT_FP8_OPS:
        return True

    if not is_nvidia():
        return False

So if SUPPORT_FP8_OPS == True, the if not is_nvidia() branch is never reached. I verified this on Windows using the ComfyUI Desktop bundled environment:

import comfy.model_management as mm
import torch

print("SUPPORT_FP8_OPS:", getattr(mm, "SUPPORT_FP8_OPS", None))
print("torch:", torch.__version__)
print("hip:", getattr(torch.version, "hip", None))

float8 = [n for n in dir(torch) if n.startswith("float8_")]
print("torch float8 dtypes:", float8[:20], "... total:", len(float8))

Output:

SUPPORT_FP8_OPS: True
torch: 2.9.0+rocmsdk20251116
hip: 7.1.52802-561cc400e1
torch float8 dtypes: ['float8_e4m3fn', 'float8_e4m3fnuz', 'float8_e5m2', 'float8_e5m2fnuz', 'float8_e8m0fnu'] ...

So this function is not the blocker (at least in the current release + this environment).

Current situation (updated):

• Linux bare metal: FP8 works with hacks (vLLM approach), but ComfyUI's is_nvidia() check prevents users from accessing it
• Windows: ROCm 7.2 lacks FP8 compute exposure for gfx1201 entirely (storage/dequant only, no compute kernels)
• ComfyUI: I also see the model weight dtype torch.float8_e4m3fn, manual cast: torch.bfloat16 behavior on RX 9070 XT (but that is on Windows)

So the issue seems to be why ComfyUI ends up doing BF16 compute even when FP8 dtypes/flags are present, rather than this specific gate.

Also: TheRock has docs for generating Triton kernels on Linux in their repo somewhere; in the past I was able to build kernels for my 7800 XT and copy them to Windows. I haven’t tested this flow on RX 9070 XT, but it may be relevant if Triton is the missing piece for FP8 kernels on Windows.