Experimental ROCM & PYTORCH builds for ALL GFX103X (rdna2)

[patientx]

THIS IS NOW SUPERSEEDED BY THE COMFYUI-ROCM FORK.

*** https://github.com/patientx-cfz/comfyui-rocm ***

THIS IS NOW SUPERSEEDED BY THE COMFYUI-ROCM FORK.

It is maintained actively and my recommendation for everyone above from vega to latest 9000s. Has auto installation, portable venv (self-contained) and necessary packages installed and updated automatically...

• https://app.mediafire.com/folder/mvrwkgj96lkua

• First make sure you have these installed :

git :: https://git-scm.com/download/win
python 3.12 :: https://www.python.org/ftp/python/3.12.10/python-3.12.10-amd64.exe
Visual C++ Runtime Library :: https://aka.ms/vs/17/release/vc_redist.x64.exe
Visual Studio Build Tools :: https://aka.ms/vs/17/release/vs_BuildTools.exe

Then download all of the files from the mediafire folder above, follow the instructions. (https://app.mediafire.com/folder/mvrwkgj96lkua)

Open a commandline window in the folder / drive you want to install comfyui to.
Copy-paste and "enter" after each command.

git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI

• (create a folder inside this empty comfyui folder called rocm and copy the files downloaded from mediafire into it)

md .triton
py -3.12 -m venv venv
venv\Scripts\activate
python.exe -m pip install --upgrade pip --quiet
pip install -r requirements.txt --quiet
pip uninstall torch torchvision torchaudio -y --quiet
cd rocm
pip install rocm-7.12.0.dev0.tar.gz rocm_sdk_core-7.12.0.dev0-py3-none-win_amd64.whl rocm_sdk_devel-7.12.0.dev0-py3-none-win_amd64.whl rocm_sdk_libraries_gfx103x_all-7.12.0.dev0-py3-none-win_amd64.whl
pip install "torch-2.10.0+devrocm7.12.0.dev0-cp312-cp312-win_amd64.whl" "torchaudio-2.10.0+devrocm7.12.0.dev0-cp312-cp312-win_amd64.whl" "torchvision-0.25.0+devrocm7.12.0.dev0-cp312-cp312-win_amd64.whl"
pip install triton-windows==3.6.0.post25 --quiet
pip install sageattention==1.0.6 --quiet
pip install https://github.com/0xDELUXA/bitsandbytes_win_rocm/releases/download/v0.49.2.dev0-py312-rocm7.12/bitsandbytes-0.49.2.dev0-cp312-cp312-win_amd64.whl --quiet
cd..
del venv\Lib\site-packages\sageattention\attn_qk_int8_per_block.py
curl -s -o venv\Lib\site-packages\sageattention\attn_qk_int8_per_block.py https://raw.githubusercontent.com/patientx/ComfyUI-Zluda/refs/heads/master/comfy/customzluda/sa/attn_qk_int8_per_block.py
del venv\Lib\site-packages\sageattention\attn_qk_int8_per_block_causal.py
curl -s -o venv\Lib\site-packages\sageattention\attn_qk_int8_per_block_causal.py https://raw.githubusercontent.com/patientx/ComfyUI-Zluda/refs/heads/master/comfy/customzluda/sa/attn_qk_int8_per_block_causal.py
del venv\Lib\site-packages\sageattention\quant_per_block.py
curl -s -o venv\Lib\site-packages\sageattention\quant_per_block.py https://raw.githubusercontent.com/patientx/ComfyUI-Zluda/refs/heads/master/comfy/customzluda/sa/quant_per_block.py

now to run this easily, download the attached comfyrun.txt , change it's extension to comfyrun.bat and change the folder and if you want the arguments in the file to your needs and you can run comfy with it from anywhere.

comfyrun.txt

---

EDIT : Thanks to @0xDELUXA ; we have bitsandbytes for rnda2 & above.

https://github.com/mengqin/ComfyUI-UnetBnbModelLoader

I tested this model for example;

https://huggingface.co/Jianqiao1/Z-Image-Turbo-bnb-nf4/blob/main/z_image_turbo_nf4_v2.safetensors

It not faster than for example gguf_q8 (about 5 to 10% slower BUT uses half the vram. And now we have more options to choose from. Better than nothing.

---

[AlessandroMIlani]

Ok, the packages seem to work.

Some info about the system

• OS: Windows 11
• GPU: RX6750XT, driver 26.1.1
• Python 3.12.10

These is the terminal output for the first generations

• Model is an illustrious
• Vae decode is tiled
• gen 1 and 2 at: 832 x 1216
• gen 3 and 4 at: 1024x1024

(venv) PS C:\Games\AI\ComfyUI> python main.py
Checkpoint files will always be loaded safely.
Found comfy_kitchen backend triton: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8']}
Found comfy_kitchen backend cuda: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8']}
Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'scaled_mm_nvfp4']}
Total VRAM 12272 MB, total RAM 32693 MB
pytorch version: 2.10.0+devrocm7.12.0.dev0
AMD arch: gfx1031
ROCm version: (7, 2)
Set vram state to: NORMAL_VRAM
Device: cuda:0 AMD Radeon RX 6750 XT : native
Using async weight offloading with 2 streams
Enabled pinned memory 14712.0
Using sub quadratic optimization for attention, if you have memory or speed issues try using: --use-split-cross-attention
Python version: 3.12.10 (tags/v3.12.10:0cc8128, Apr  8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
ComfyUI version: 0.12.3
****** User settings have been changed to be stored on the server instead of browser storage. ******
****** For multi-user setups add the --multi-user CLI argument to enable multiple user profiles. ******
ComfyUI frontend version: 1.38.13
[Prompt Server] web root: C:\Games\AI\ComfyUI\venv\Lib\site-packages\comfyui_frontend_package\static

Import times for custom nodes:
   0.0 seconds: C:\Games\AI\ComfyUI\custom_nodes\websocket_image_save.py

Context impl SQLiteImpl.
Will assume non-transactional DDL.
Context impl SQLiteImpl.
Will assume non-transactional DDL.
Running upgrade  -> 0001_assets, Initial assets schema
Revision ID: 0001_assets
Revises: None
Create Date: 2025-12-10 00:00:00
Database upgraded from None to 0001_assets
Assets scan(roots=['models']) completed in 0.040s (created=11, skipped_existing=0, orphans_pruned=0, total_seen=11)
Starting server

To see the GUI go to: http://127.0.0.1:8188
got prompt
Failed to validate prompt for output 19:
* VAELoader 25:
  - Value not in list: vae_name: 'fixFP16ErrorsSDXLLowerMemoryUse_v10.safetensors' not in ['pixel_space']
Output will be ignored
invalid prompt: {'type': 'prompt_outputs_failed_validation', 'message': 'Prompt outputs failed validation', 'details': '', 'extra_info': {}}
got prompt
model weight dtype torch.float16, manual cast: None
model_type EPS
Using split attention in VAE
Using split attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.float32
Requested to load SDXLClipModel
loaded completely;  1560.80 MB loaded, full load: True
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cuda:0, dtype: torch.float16
Requested to load SDXLClipModel
Requested to load SDXL
loaded completely; 7966.92 MB usable, 4897.05 MB loaded, full load: True
  0%|                                                                                                                                                                          | 0/20 [00:00<?, ?it/s]warning: xnack 'Off' was requested for a processor that does not support it!
warning: xnack 'Off' was requested for a processor that does not support it!
warning: xnack 'Off' was requested for a processor that does not support it!
warning: xnack 'Off' was requested for a processor that does not support it!
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [01:10<00:00,  3.51s/it]
Requested to load AutoencoderKL
0 models unloaded.
loaded completely; 4728.70 MB usable, 319.11 MB loaded, full load: True
warning: xnack 'Off' was requested for a processor that does not support it!
warning: xnack 'Off' was requested for a processor that does not support it!
Prompt executed in 410.98 seconds
got prompt
Requested to load SDXL
loaded completely; 8861.86 MB usable, 4897.05 MB loaded, full load: True
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [01:42<00:00,  5.11s/it]
0 models unloaded.
Prompt executed in 108.28 seconds
got prompt
Requested to load SDXLClipModel
loaded completely; 9939.62 MB usable, 1560.80 MB loaded, full load: True
Requested to load SDXL
Unloaded partially: 84.03 MB freed, 1476.77 MB remains loaded, 15.64 MB buffer reserved, lowvram patches: 0
loaded completely; 7514.58 MB usable, 4897.05 MB loaded, full load: True
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [01:02<00:00,  3.13s/it]
Requested to load AutoencoderKL
0 models unloaded.
loaded completely; 4723.88 MB usable, 319.11 MB loaded, full load: True
Prompt executed in 156.68 seconds
got prompt
Requested to load SDXLClipModel
loaded completely; 9939.62 MB usable, 1560.80 MB loaded, full load: True
Requested to load SDXL
Unloaded partially: 211.09 MB freed, 108.02 MB remains loaded, 27.01 MB buffer reserved, lowvram patches: 0
loaded completely; 7389.53 MB usable, 4897.05 MB loaded, full load: True
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:17<00:00,  1.12it/s]
0 models unloaded.
loaded completely; 4700.68 MB usable, 319.11 MB loaded, full load: True
Prompt executed in 25.93 seconds

I've altro tried the Klein 4b distilled default workflow, but i get Warning: Ran out of memory when regular VAE decoding, retrying with tiled VAE decoding. then the system start to stutter so I had to kill the process.

[patientx]

with klein I "rx 6800" have to use tiled -long before these specific rocm packages- most of the time so that's common ...

[mrparsons]

On a 6650xt under windows 11, doing the flux2 distilled default hedgehog pic takes about 80 seconds, which I think is generally the same as what zluda was giving.

[rez3vil]

Okay so after installing rocm and pytorch from your method, I wanted to test a simple tensor on my gpu -->

import torch

print(torch.cuda.is_available())
print(f"Number of devices: {torch.cuda.device_count()}")
print(f"CUDA is available! Using device: {torch.cuda.get_device_name(0)}")
print(f"CUDA is available! Using device: {torch.cuda.get_device_name(1)}")
print(f"PyTorch version with CUDA: {torch.version.cuda}")

device = torch.device("cuda")
x = torch.randn(3, 3, device=device)

but I get following output and error -->

True
Number of devices: 2
CUDA is available! Using device: AMD Radeon(TM) Graphics
CUDA is available! Using device: AMD Radeon RX 6700S
PyTorch version with CUDA: None

---------------------------------------------------------------------------
AcceleratorError                          Traceback (most recent call last)
Cell In[24], line 10
      7 print(f"PyTorch version with CUDA: {torch.version.cuda}")
      9 device = torch.device("cuda")
---> 10 x = torch.randn(3, 3, device=device)

AcceleratorError: HIP error: invalid kernel file
Search for `hipErrorInvalidKernelFile' in https://rocm.docs.amd.com/projects/HIP/en/latest/index.html for more information.
HIP kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing AMD_SERIALIZE_KERNEL=3
Compile with `TORCH_USE_HIP_DSA` to enable device-side assertions.

Do I need to install something else?

[patientx]

nope that should work, it is not my method fyi, it is how the comfy or amd installs these as well, if you like you can try the other rocm libs their supported gpu's claims 6700s. As I said I am not sure if those or these ones here would work with any gpu....

EDIT: you have two "gpu"s , probably apu is being used from your cpu. check the readme to setup the other device correctly.

If you have an integrated GPU by AMD (e.g. AMD Radeon(TM) Graphics) you need to add HIP_VISIBLE_DEVICES=1 to your environment variables. Other possible variables to use : ROCR_VISIBLE_DEVICES=1 HCC_AMDGPU_TARGET=1 . This basically tells it to use 1st gpu -this number could be different if you have multiple gpu's- Otherwise it will default to using your iGPU, which will most likely not work. This behavior is caused by a bug in the ROCm-driver.

so you "set HIP_VISIBLE_DEVICES=1" or one of the others before you start comfy. and make sure to do the "rocm-init" after activating venv. the device number could be different, 0 , 1 I am not sure.

[rez3vil]

Okay so that did work, I was testing it on windows 11 and tested that from poweshell. I used -->

$env:CUDA_VISIBLE_DEVICES = "1"
$env:HIP_VISIBLE_DEVICES = "1"

and it did ran the simple tensor on cuda:0 -->

print(torch.version.hip)
print(torch.version.rocm)

print(torch.cuda.is_available())
print(f"Number of devices: {torch.cuda.device_count()}")
print(f"CUDA is available! Using device: {torch.cuda.get_device_name()}")
print(f"PyTorch version with CUDA: {torch.version.cuda}")

device = torch.device("cuda")
x = torch.randn(3, 3, device=device)

print(x)

Output -->

7.2.0
7.12.0
True
Number of devices: 1
CUDA is available! Using device: AMD Radeon RX 6700S
PyTorch version with CUDA: None
tensor([[ 0.3258, -1.7096, -1.2975],
        [ 0.0837,  2.5402, -0.1996],
        [-0.3048,  1.1795, -0.6170]], device='cuda:0')

Thank you so much! 😊
I have tried training a simple LLM model, it used to take nearly 20 minutes on my cpu, but today it took only 3 minutes on my machine! 😁
Finally I can train neural netowork models on RX 6700s after 4 years literally haha!

[VeteranXT]

It works fine on RX 6600 XT. I can confirm it works.

[viinaji]

Works great on 6700 XT, Thank you @patientx.
Klein 9B gguf quad attention -> Zluda 3m 57s | This way 59s.
If i use sage or pytorch attention, throws memory exceeded error. any suggestion?