[Bug]: rocBLAS error: Cannot read ...\AMD\ROCm\6.1\bin\/rocblas/library/TensileLibrary.dat: No such file or directory for GPU arch : gfx1036

[marnel-estrada]

Checklist

• The issue exists after disabling all extensions
• The issue exists on a clean installation of webui
• The issue is caused by an extension, but I believe it is caused by a bug in the webui
• The issue exists in the current version of the webui
• The issue has not been reported before recently
• The issue has been reported before but has not been fixed yet

What happened?

I followed this guide. When I run webui-user.bat, this error is the end result. I searched for this error and I found this. However, I don't understand what it means to "setting the environment variable export HSA_OVERRIDE_GFX_VERSION=10.3.0". I don't think that's the same with setting the PATH variable. I have updated to the current drivers. My GPU is RX7800.

Steps to reproduce the problem

1. Follow the guide here.
2. The error shows when running webui-user.bat.

What should have happened?

I expected it to proceed with no errors?

What browsers do you use to access the UI ?

Mozilla Firefox

Sysinfo

I don't know what to do here. The WebUI is not opened. I tried --dump-sysinfo but there are Python errors:

D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\lib\site-packages\timm\models\layers\__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
  warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
  rank_zero_deprecation(
Traceback (most recent call last):
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\launch.py", line 48, in <module>
    main()
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\launch.py", line 29, in main
    filename = launch_utils.dump_sysinfo()
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\launch_utils.py", line 700, in dump_sysinfo
    text = sysinfo.get()
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\sysinfo.py", line 46, in get
    res = get_dict()
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\sysinfo.py", line 119, in get_dict
    "Extensions": get_extensions(enabled=True, fallback_disabled_extensions=config.get('disabled_extensions', [])),
AttributeError: 'str' object has no attribute 'get'
Press any key to continue . . .

Console logs

venv "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\Scripts\Python.exe"
WARNING: ZLUDA works best with SD.Next. Please consider migrating to SD.Next.
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Version: v1.10.1-amd-12-gae5ff7a2
Commit hash: ae5ff7a232cd898f653e4fffb36f507b54de8b72
Using ZLUDA in D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\.zluda
ROCm agents: ['gfx1036', 'gfx1101'], using gfx1036
Skipping onnxruntime installation.
You are up to date with the most recent release.
D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\lib\site-packages\timm\models\layers\__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
  warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
  rank_zero_deprecation(
Launching Web UI with arguments: --use-zluda --update-check --skip-ort

rocBLAS error: Cannot read C:\Program Files\AMD\ROCm\6.1\bin\/rocblas/library/TensileLibrary.dat: No such file or directory for GPU arch : gfx1036

rocBLAS error: Could not initialize Tensile host:
regex_error(error_backref): The expression contained an invalid back reference.
Press any key to continue . . .

Additional information

I updated the GPU driver prior to following the steps.

[marnel-estrada]

I tried adding this in webui.bat but it still showed the same error:
set HSA_OVERRIDE_GFX_VERSION=10.3.0

[lshqqytiger]

Don't set HSA_OVERRIDE_GFX_VERSION. Instead, add --device-id 1 or set HIP_VISIBLE_DEVICES=1.

[marnel-estrada]

I used those and got another set of errors:

venv "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\Scripts\Python.exe"
WARNING: ZLUDA works best with SD.Next. Please consider migrating to SD.Next.
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Version: v1.10.1-amd-12-gae5ff7a2
Commit hash: ae5ff7a232cd898f653e4fffb36f507b54de8b72
Using ZLUDA in D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\.zluda
ROCm agents: ['gfx1101'], using gfx1101
Skipping onnxruntime installation.
You are up to date with the most recent release.
D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\lib\site-packages\timm\models\layers\__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
  warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
  rank_zero_deprecation(
Invalid device id: str
Traceback (most recent call last):
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\errors.py", line 98, in run
    code()
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\devices.py", line 118, in enable_tf32
    if cuda_no_autocast():
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\devices.py", line 30, in cuda_no_autocast
    torch.cuda.get_device_capability(device_id) == (7, 5)
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\lib\site-packages\torch\cuda\__init__.py", line 430, in get_device_capability
    prop = get_device_properties(device)
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\lib\site-packages\torch\cuda\__init__.py", line 447, in get_device_properties
    raise AssertionError("Invalid device id")
AssertionError: Invalid device id

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\launch.py", line 48, in <module>
    main()
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\launch.py", line 39, in main
    prepare_environment()
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\launch_utils.py", line 671, in prepare_environment
    from modules import devices
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\devices.py", line 125, in <module>
    errors.run(enable_tf32, "Enabling TF32")
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\errors.py", line 100, in run
    display(task, e)
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\errors.py", line 68, in display
    te = traceback.TracebackException.from_exception(e)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\traceback.py", line 572, in from_exception
    return cls(type(exc), exc, exc.__traceback__, *args, **kwargs)
AttributeError: 'str' object has no attribute '__traceback__'
Press any key to continue . . .

I also saw here that gfx1036 is not supported.

[marnel-estrada]

I edited the code in cuda/init.py to print the device values to this:

def get_device_properties(device: _device_t) -> _CudaDeviceProperties:
    r"""Get the properties of a device.

    Args:
        device (torch.device or int or str): device for which to return the
            properties of the device.

    Returns:
        _CudaDeviceProperties: the properties of the device
    """
    _lazy_init()  # will define _get_device_properties
    print("device from parameter: " + str(device))
    print("Device Count: " + str(device_count()))
    device = _get_device_index(device, optional=True)
    print("device after _get_device_index(): " + str(device))
    if device < 0 or device >= device_count():
        raise AssertionError("Invalid device id")
    return _get_device_properties(device)  # type: ignore[name-defined]

This is what I got. Not sure if it helps:

venv "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\Scripts\Python.exe"
WARNING: ZLUDA works best with SD.Next. Please consider migrating to SD.Next.
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Version: v1.10.1-amd-12-gae5ff7a2
Commit hash: ae5ff7a232cd898f653e4fffb36f507b54de8b72
Using ZLUDA in D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\.zluda
ROCm agents: ['gfx1101'], using gfx1101
Skipping onnxruntime installation.
You are up to date with the most recent release.
D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\lib\site-packages\timm\models\layers\__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
  warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
  rank_zero_deprecation(
device from parameter: 0
Device Count: 1
device after _get_device_index(): 0
device from parameter: 0
Device Count: 1
device after _get_device_index(): 0
device from parameter: 0
Device Count: 1
device after _get_device_index(): 0
device from parameter: 1
Device Count: 1
device after _get_device_index(): 1
Invalid device id: str
Traceback (most recent call last):
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\errors.py", line 98, in run
    code()
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\devices.py", line 118, in enable_tf32
    if cuda_no_autocast():
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\devices.py", line 30, in cuda_no_autocast
    torch.cuda.get_device_capability(device_id) == (7, 5)
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\lib\site-packages\torch\cuda\__init__.py", line 430, in get_device_capability
    prop = get_device_properties(device)
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\venv\lib\site-packages\torch\cuda\__init__.py", line 450, in get_device_properties
    raise AssertionError("Invalid device id")
AssertionError: Invalid device id

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\launch.py", line 48, in <module>
    main()
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\launch.py", line 39, in main
    prepare_environment()
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\launch_utils.py", line 671, in prepare_environment
    from modules import devices
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\devices.py", line 125, in <module>
    errors.run(enable_tf32, "Enabling TF32")
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\errors.py", line 100, in run
    display(task, e)
  File "D:\AI\sd\SD-Zluda\stable-diffusion-webui-amdgpu\modules\errors.py", line 68, in display
    te = traceback.TracebackException.from_exception(e)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\traceback.py", line 572, in from_exception
    return cls(type(exc), exc, exc.__traceback__, *args, **kwargs)
AttributeError: 'str' object has no attribute '__traceback__'
Press any key to continue . . .

_get_device_index() returned 1 at the end there which triggers the AssertionError.

[lshqqytiger]

--device-id is untested feature on zluda. I couldn't test as I don't have APU or another AMDGPU. Anyway, set HIP_VISIBLE_DEVICES=1 should work.

[CS1o]

HIP SDK found your APU (gfx1036) before your dedicated GPU.

You can fix that by adding: set HIP_VISIBLE_DEVICES=1
to the webui-user.bat.

Or by opening up the Device Manager and under Display Adapters disable the Radeon TM Graphics.
Then relaunch the webui-user.bat

Also its not recommended to use Python from MS Store.
From your log:

File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\traceback.py", line 572, in from_exception
    return cls(type(exc), exc, exc.__traceback__, *args, **kwargs)
AttributeError: 'str' object has no attribute '__traceback__'

It seems you have and old Python Version installed via the MS Store.
Uninstall it via Settings -> Apps -> Apps and Features.

[TiberiumCat]

Alternative option if you're not using the APU. Disable it in Device Manager.
On one hand you have to remember to disable it every time you update video drivers. On the other hand, nothing else is going to give you issues by trying to use it.

[granitbrovina]

Alternative option if you're not using the APU. Disable it in Device Manager. On one hand you have to remember to disable it every time you update video drivers. On the other hand, nothing else is going to give you issues by trying to use it.

I'm having a similar issue to OP. I installed ZLUDA and replaced the library folder in rocblas with the correct one for my card (gfx1201). Yet, when I run sdnext, it would select gfx1036 as the ROCm agent. So, I disabled the iGPU in device manager, and ran webui.bat again. Now it shows "ROCm: no agent was found". So it seems it does not detect my GPU at all?

I even tried HIP_VISIBLE_Devices=1, still not working...

[granitbrovina]

I spoke too soon!

Apparently, I had done a mistake.

This comment helped me resolve it: ROCm/ROCm#2941 (comment)

In my case, I had to set HIP_VISIBLE_DEVICES="0,1" and the issue got resolved!

[Driftui]

Alternative option if you're not using the APU. Disable it in Device Manager. On one hand you have to remember to disable it every time you update video drivers. On the other hand, nothing else is going to give you issues by trying to use it.

I'm having a similar issue to OP. I installed ZLUDA and replaced the library folder in rocblas with the correct one for my card (gfx1201). Yet, when I run sdnext, it would select gfx1036 as the ROCm agent. So, I disabled the iGPU in device manager, and ran webui.bat again. Now it shows "ROCm: no agent was found". So it seems it does not detect my GPU at all?

I even tried HIP_VISIBLE_Devices=1, still not working...

i tried adding HIP_VISIBLE_DEVICES=1 then it would say "ROCm: no agent was found" but i also get this message now

"rocBLAS error: Cannot read C:\Program Files\AMD\ROCm\6.2\bin/rocblas/library/TensileLibrary.dat: No such file or directory for GPU arch : gfx1201

rocBLAS error: Could not initialize Tensile host:
regex_error(error_backref): The expression contained an invalid back reference.
Press any key to continue . . ."

I have no idea what im doing wrong im not sure if its because im using a 9070xt and maybe it's unsupported Im not good with code so im lost on how to fix this. this is the full prompt if someone knows how to fix this.

Details

venv "C:\A1111\stable-diffusion-webui-directml\venv\Scripts\Python.exe" Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] Version: v1.10.1-amd-39-g6d85a7df Commit hash: 6d85a7d ROCm: no agent was found ROCm: version=6.2 ZLUDA support: experimental ZLUDA load: path='C:\A1111\stable-diffusion-webui-directml\.zluda' nightly=False W0812 05:59:45.603648 9356 venv\Lib\site-packages\torch\distributed\elastic\multiprocessing\redirects.py:29] NOTE: Redirects are currently not supported in Windows or MacOs. C:\A1111\stable-diffusion-webui-directml\venv\lib\site-packages\timm\models\layers\__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) no module 'xformers'. Processing without... no module 'xformers'. Processing without... No module 'xformers'. Proceeding without it. C:\A1111\stable-diffusion-webui-directml\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead. rank_zero_deprecation( Launching Web UI with arguments: C:\A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\onnx\_internal\registration.py:159: OnnxExporterWarning: Symbolic function 'aten::scaled_dot_product_attention' already registered for opset 14. Replacing the existing function with new function. This is unexpected. Please report it on https://github.com/pytorch/pytorch/issues. warnings.warn( ONNX failed to initialize: module 'optimum.onnxruntime.modeling_diffusion' has no attribute 'ORTPipelinePart'

rocBLAS error: Cannot read C:\Program Files\AMD\ROCm\6.2\bin/rocblas/library/TensileLibrary.dat: No such file or directory for GPU arch : gfx1201

rocBLAS error: Could not initialize Tensile host:
regex_error(error_backref): The expression contained an invalid back reference.
Press any key to continue . . .

[Mixcalibur]

Not sure if still relevant, I have a 6700 XT and got it to work by overwriting the ROCm bin/library folder with "NewerROCmLibs.rar" from this thread: https://github.com/brknsoul/ROCmLibs?tab=readme-ov-file

I did that after HIP_VISIBLE_DEVICES=1 made it choose gfx1031 instead of 1036 which would be the integrated gfx adapter.