Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RX 470,480,570,580 no longer supports OpenCL on ROCm 4.5 #1659

Closed
boxerab opened this issue Jan 22, 2022 · 76 comments
Closed

RX 470,480,570,580 no longer supports OpenCL on ROCm 4.5 #1659

boxerab opened this issue Jan 22, 2022 · 76 comments

Comments

@boxerab
Copy link

boxerab commented Jan 22, 2022

cf: #1608

And please don't close this issue until we have a clear answer - has polaris support been intentionally dropped from ROCm
after only 6 years, or is this an error ?

@ROCmSupport
Copy link

AFAIK, we have not removed any code intentionally. But maybe something changed in the stack and we don't validate gfx8 on ROCm, so it might not be working anymore.
One thing from support point of view, each card has some duration of support. We can not continue supporting cards for more number of years as per business standards. As new cards coming into the market, we keep adding the new ones into the supported list and keep dropping the old ones after certain amount of time, which is the process.

Anyhow, I am not closing this ticket right now. Let me wait for some time.
Thank you.

@boxerab
Copy link
Author

boxerab commented Jan 29, 2022

@ROCmSupport the RX 590 was released in 2018. This is way too early to drop support.

https://www.techpowerup.com/gpu-specs/radeon-rx-590.c3322

@xuhuisheng
Copy link
Contributor

Maybe you mean OpenCL. The MIOpen run succeed only with a small patch.

@kvirikroma
Copy link

kvirikroma commented Feb 8, 2022

@boxerab I solved this issue by using a docker image of certain version for tensorflow-rocm, when I had tried to use a tensorflow with rx570. Here's my repo with a few little scripts that I used to automate this process: https://github.com/kvirikroma/tensorflow-rocm-legacy
Maybe this will be helpful for you

@boxerab boxerab changed the title RX 470,480,570,580 no longer supported on ROCm 4.5 RX 470,480,570,580 no longer supports OpenCL on ROCm 4.5 Feb 8, 2022
@boxerab
Copy link
Author

boxerab commented Feb 8, 2022

@kvirikroma thank you. I need OpenCL support. That's gone now.

@Umio-Yasuno
Copy link

Is it possible that this commit is the cause?
ROCm/ROCclr@16044d1

@boxerab
Copy link
Author

boxerab commented Feb 10, 2022

Is it possible that this commit is the cause? ROCm-Developer-Tools/ROCclr@16044d1

This looks exactly like where OpenCL was disabled. Is it difficult to
build ROCclr with this patch reverted ?

@xuhuisheng
Copy link
Contributor

I can have a try do patch ROCclr for OpenCL, but I am not familiar with OpenCL, could you show me a demo for testing?

@Flakebi
Copy link

Flakebi commented Feb 10, 2022

Here are a few examples: https://rocmdocs.amd.com/en/latest/Programming_Guides/Opencl-programming-guide.html#example
The Example Code 1 can be compiled and run with g++ -std=c++17 -o test test.cpp -g -lOpenCL -O2 && ./test

@boxerab
Copy link
Author

boxerab commented Feb 10, 2022

I can have a try do patch ROCclr for OpenCL, but I am not familiar with OpenCL, could you show me a demo for testing?

those examples look good, also there is a clinfo binary in the /opt/rocm/opencl/bin folder, I believe, that can quickly tell
you if the card is recognized as an opencl device.

@xuhuisheng
Copy link
Contributor

xuhuisheng commented Feb 10, 2022

I am afraid of it is not work, for just revert this commits.
I comment out return false of device/device.hpp, recompiling HIP with ROCclr, but /opt/rocm/opencl/bin/clinfo cannot find any devices , either.

work@e0454438e38a:~/rocm-build$ /opt/rocm/opencl/bin/clinfo
Number of platforms:                             1
  Platform Profile:                              FULL_PROFILE
  Platform Version:                              OpenCL 2.2 AMD-APP.dbg (3361.0)
  Platform Name:                                 AMD Accelerated Parallel Processing
  Platform Vendor:                               Advanced Micro Devices, Inc.
  Platform Extensions:                           cl_khr_icd cl_amd_event_callback


  Platform Name:                                 AMD Accelerated Parallel Processing
Number of devices:                               0

My card is RX580.

@boxerab
Copy link
Author

boxerab commented Feb 10, 2022

Thanks, so it looks like AMD really did remove support

@Atemu
Copy link

Atemu commented Feb 10, 2022

And intentionally so.

@Atemu
Copy link

Atemu commented Feb 10, 2022

Has anyone tried to set ROC_ENABLE_PRE_VEGA to true? Default seems to be false.

@boxerab
Copy link
Author

boxerab commented Feb 10, 2022

hmmm, that's an interesting idea

@boxerab
Copy link
Author

boxerab commented Feb 11, 2022

@ROCmSupport can you comment on Polaris support in recently released ROCm 5.0 ?

@MathiasMagnus
Copy link
Contributor

MathiasMagnus commented Feb 14, 2022

Dear @ROCmSupport,

We can not continue supporting cards for more number of years as per business standards.

Nvidia recently dropped Kepler support in CUDA in June 2021, when Kepler was released 2012 April. That's 9 years of support. RX 470 was released June 2016, so ~5 years. A bit over half as much. People who bought an RX 590 (released in 2018 November, easily on shelves throughout 2019) only got 2 years of support. 2 years of support!

I'm teaching GPGPU to physicists at university and we have a BYOD policy (teaching OpenCL, HIP/CUDA, SYCL) and easily the students who suffer the most are those sporting AMD hardware (myself included, running an RX 580 laptop). It's increasingly hard to install and run any of these APIs. Everything stems from the spotty gfx803 support moved to "partial support" (whatever that means) way too early. AMD shouldn't release products they won't support.

(FWIW even in my professional capacity it's becoming harder to justify recommending CDNA products, due to all of them being gfx9XYZ variants for such a long time. I can't say with a straight face that MI100s/MI200s will not share the same fate as RX 590s, that MI200 successors won't sport a new ISA and gfx9XYZ will be dropped ever so swiftly, being an ancient ISA flavor. Professional Fiji owners have been burned like this before, not just consumer card owners.)

My experience trying to get gfx803 working:

  • HIP doesn't work with DKMS or with Ubuntu 20.04 upstream kernel
mate@GL702ZC:~$ /opt/rocm/bin/rocm-smi


======================= ROCm System Management Interface =======================
================================= Concise Info =================================
GPU  Temp   AvgPwr   SCLK    MCLK     Fan  Perf  PwrCap  VRAM%  GPU%  
0    55.0c  14.253W  608Mhz  2000Mhz  0%   auto  68.0W     7%   0%    
================================================================================
WARNING:  		 One or more commands failed
============================= End of ROCm SMI Log ==============================
mate@GL702ZC:~$ sudo /opt/rocm/bin/rocminfo 
ROCk module is loaded
hsa api call failure at: /long_pathname_so_that_rpms_can_package_the_debug_info/src/rocminfo/rocminfo.cc:1143
Call returned HSA_STATUS_ERROR_OUT_OF_RESOURCES: The runtime failed to allocate the necessary resources. This error may also occur when the core runtime library needs to spawn threads or create internal OS-specific events.
  • OpenCL ROCr runtime doesn't detect the device, even with ROC_ENABLE_PRE_VEGA set.
  • OpenCL orca runtime mostly works but
    • crashes inside the runtime when creating the interop context.
    • Failed to finalize BRIG when having work_group_reduce_add function in use inside the kernel. (Missing device built-in)

The user experience is bad with all APIs. This doesn't incentivize users to upgrade and buy AMD HW.

@FelixSchwarz
Copy link

Has anyone tried to set ROC_ENABLE_PRE_VEGA to true? Default seems to be false.

I tried that using a self-compiled ROCm 5.0 stack on Fedora but still no dice (same as @MathiasMagnus). However if someone is collecting patches to re-enable Polaris in ROCm, please let me know. Seems like there is no easy solution but maybe it isn't that hard...

@dagrim
Copy link

dagrim commented Feb 14, 2022

I have OpenCL working in Ubuntu 20.04 with an RX570. Here is what I did :

  • install ROCm 4.5.2
  • get the 21.10 Radeon Software for Linux for Ubuntu 20.04 .tar.xz from here : https://www.amd.com/en/support/kb/release-notes/rn-amdgpu-unified-linux-21-10
  • extract the .tar.xz
  • extract the opencl-orca-amdgpu-pro-icd_21.10-1247438_amd64.deb and ocl-icd-libopencl1-amdgpu-pro_21.10-1247438_amd64.deb files
  • check that you get those 4 files : opt/amdgpu-pro/lib/x86_64-linux-gnu/{libamdocl12cl64.so,libamdocl-orca64.so,libOpenCL.so.1.2,libOpenCL.so.1}
  • put those 4 files into a custom directory, say /opt/myopencl/lib/
  • create a new file /etc/OpenCL/vendors/myopencl.icd with only 1 line : /opt/myopencl/lib/libamdocl-orca64.so
  • try clinfo ; here is what I get :
Number of platforms                               2
  Platform Name                                   AMD Accelerated Parallel Processing
  Platform Vendor                                 Advanced Micro Devices, Inc.
  Platform Version                                OpenCL 2.1 AMD-APP (3224.4)
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_icd cl_amd_event_callback cl_amd_offline_devices 
  Platform Host timer resolution                  1ns
  Platform Extensions function suffix             AMD

  Platform Name                                   Clover
  Platform Vendor                                 Mesa
  Platform Version                                OpenCL 1.1 Mesa 21.2.0-devel (git-eb6d990 2021-05-04 focal-oibaf-ppa)
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_icd
  Platform Extensions function suffix             MESA

  Platform Name                                   AMD Accelerated Parallel Processing
Number of devices                                 1
  Device Name                                     Ellesmere
  Device Vendor                                   Advanced Micro Devices, Inc.
  Device Vendor ID                                0x1002
  Device Version                                  OpenCL 1.2 AMD-APP (3224.4)
  Driver Version                                  3224.4
  Device OpenCL C Version                         OpenCL C 1.2 
  Device Type                                     GPU
  Device Board Name (AMD)                         AMD Radeon RX 570 Series
[...]

@boxerab
Copy link
Author

boxerab commented Feb 14, 2022

I have OpenCL working in Ubuntu 20.04 with an RX570. Here is what I did :

  • install ROCm 4.5.2
  • get the 21.10 Radeon Software for Linux for Ubuntu 20.04 .tar.xz from here : https://www.amd.com/en/support/kb/release-notes/rn-amdgpu-unified-linux-21-10
  • extract the .tar.xz
  • extract the opencl-orca-amdgpu-pro-icd_21.10-1247438_amd64.deb and ocl-icd-libopencl1-amdgpu-pro_21.10-1247438_amd64.deb files
  • check that you get those 4 files : opt/amdgpu-pro/lib/x86_64-linux-gnu/{libamdocl12cl64.so,libamdocl-orca64.so,libOpenCL.so.1.2,libOpenCL.so.1}
  • put those 4 files into a custom directory, say /opt/myopencl/lib/
  • create a new file /etc/OpenCL/vendors/myopencl.icd with only 1 line : /opt/myopencl/lib/libamdocl-orca64.so
  • try clinfo ; here is what I get :
Number of platforms                               2
  Platform Name                                   AMD Accelerated Parallel Processing
  Platform Vendor                                 Advanced Micro Devices, Inc.
  Platform Version                                OpenCL 2.1 AMD-APP (3224.4)
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_icd cl_amd_event_callback cl_amd_offline_devices 
  Platform Host timer resolution                  1ns
  Platform Extensions function suffix             AMD

  Platform Name                                   Clover
  Platform Vendor                                 Mesa
  Platform Version                                OpenCL 1.1 Mesa 21.2.0-devel (git-eb6d990 2021-05-04 focal-oibaf-ppa)
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_icd
  Platform Extensions function suffix             MESA

  Platform Name                                   AMD Accelerated Parallel Processing
Number of devices                                 1
  Device Name                                     Ellesmere
  Device Vendor                                   Advanced Micro Devices, Inc.
  Device Vendor ID                                0x1002
  Device Version                                  OpenCL 1.2 AMD-APP (3224.4)
  Driver Version                                  3224.4
  Device OpenCL C Version                         OpenCL C 1.2 
  Device Type                                     GPU
  Device Board Name (AMD)                         AMD Radeon RX 570 Series
[...]

Nice. But I see that only OpenCL 1.2 is supported - used to be 2.x with earlier ROCm versions. Also, do you even need ROCm, as you are getting OpenCL from the amdgpu driver ?

@dagrim
Copy link

dagrim commented Feb 15, 2022

You're right on both points :

  1. ROCm is probably not needed, indeed. Any OpenCL icd would work.
  2. The Platform version advertises OpenCL 2.1, and the device version 1.2. In fact, I did not even notice since that solution allowed my OpenCL workflow to run successfully.

@xuhuisheng
Copy link
Contributor

xuhuisheng commented Feb 15, 2022

I think I used wrong package, after recompile rocm-opencl-runtime, clinfo can display gfx803 device.
I will upload my package in this week, everyone who is interest in opencl and gfx803 can have a test.

BTW, My card is RX580, test on ubuntu-20.04 and ROCm-5.0.0.

@boxerab
Copy link
Author

boxerab commented Feb 15, 2022

I think I used wrong package, after recompile rocm-opencl-runtime, clinfo can display gfx803 device. I will upload my package in this week, everyone who is interest in opencl and gfx803 can have a test.

BTW, My card is RX580, test on ubuntu-20.04 and ROCm-5.0.0.

Thanks. If you can document what you did to recompile the runtime, we can try it.

@Umio-Yasuno
Copy link

https://github.com/RadeonOpenCompute/ROCm-OpenCL-Runtime#building

Follow these steps:

  • Build ROCclr first. Follow the steps in the following link to build ROCclr
    ROCclr Readme
    In this step, $OPENCL_DIR and $ROCclr_DIR are defined.

  • Building OpenCL
    Run these commands:

cd "$OPENCL_DIR"
mkdir -p build; cd build
cmake -DUSE_COMGR_LIBRARY=ON -DCMAKE_PREFIX_PATH="$ROCclr_DIR/build;/opt/rocm/" ..
make -j$(nproc)

Note: For release build, add "-DCMAKE_BUILD_TYPE=Release" to the cmake command line.

@boxerab
Copy link
Author

boxerab commented Feb 15, 2022

https://github.com/RadeonOpenCompute/ROCm-OpenCL-Runtime#building

Follow these steps:

  • Build ROCclr first. Follow the steps in the following link to build ROCclr
    ROCclr Readme
    In this step, $OPENCL_DIR and $ROCclr_DIR are defined.
  • Building OpenCL
    Run these commands:
cd "$OPENCL_DIR"
mkdir -p build; cd build
cmake -DUSE_COMGR_LIBRARY=ON -DCMAKE_PREFIX_PATH="$ROCclr_DIR/build;/opt/rocm/" ..
make -j$(nproc)

Note: For release build, add "-DCMAKE_BUILD_TYPE=Release" to the cmake command line.

Did you change any settings or variables before building ?

@Umio-Yasuno
Copy link

https://github.com/RadeonOpenCompute/ROCm-OpenCL-Runtime#building

Follow these steps:

  • Build ROCclr first. Follow the steps in the following link to build ROCclr
    ROCclr Readme
    In this step, $OPENCL_DIR and $ROCclr_DIR are defined.
  • Building OpenCL
    Run these commands:
cd "$OPENCL_DIR"
mkdir -p build; cd build
cmake -DUSE_COMGR_LIBRARY=ON -DCMAKE_PREFIX_PATH="$ROCclr_DIR/build;/opt/rocm/" ..
make -j$(nproc)

Note: For release build, add "-DCMAKE_BUILD_TYPE=Release" to the cmake command line.

Did you change any settings or variables before building ?

Sorry, I haven't tried it in my environment yet.
At least, I think ROCclr and ROCm-OpenCL-Runtime need to rebuilt to enable OpenCL for gfx803, .

@xuhuisheng
Copy link
Contributor

xuhuisheng commented Feb 16, 2022

I had uploaded patched rocm-opencl-runtime package to github.
https://github.com/xuhuisheng/rocm-gfx803/releases/download/rocm500/rocm-opencl_2.0.0-local_amd64.deb

The patch file: https://github.com/xuhuisheng/rocm-build/blob/feature/build/patch/31.rocm-opencl-runtime-rocclr-gfx803-1.patch
Just change ROC_ENABLE_PRE_VEGA from false to true.

The build script: https://github.com/xuhuisheng/rocm-build/blob/feature/build/31.rocm-opencl-runtime.sh

And please make sure you are testing on ubuntu-20.04 with ROCm-5.0.0 and gfx803 card.

/opt/rocm/bin/opencl/bin/clinfo

detail

work@2ff13ecae2ec:~/rocm-build$ /opt/rocm/opencl/bin/clinfo
Number of platforms:                             1
  Platform Profile:                              FULL_PROFILE
  Platform Version:                              OpenCL 2.2 AMD-APP.dbg (3406.0)
  Platform Name:                                 AMD Accelerated Parallel Processing
  Platform Vendor:                               Advanced Micro Devices, Inc.
  Platform Extensions:                           cl_khr_icd cl_amd_event_callback


  Platform Name:                                 AMD Accelerated Parallel Processing
Number of devices:                               1
  Device Type:                                   CL_DEVICE_TYPE_GPU
  Vendor ID:                                     1002h
  Board name:                                    Radeon RX 580 Series
  Device Topology:                               PCI[ B#2, D#0, F#0 ]
  Max compute units:                             36
  Max work items dimensions:                     3
    Max work items[0]:                           1024
    Max work items[1]:                           1024
    Max work items[2]:                           1024
  Max work group size:                           256
  Preferred vector width char:                   4
  Preferred vector width short:                  2
  Preferred vector width int:                    1
  Preferred vector width long:                   1
  Preferred vector width float:                  1
  Preferred vector width double:                 1
  Native vector width char:                      4
  Native vector width short:                     2
  Native vector width int:                       1
  Native vector width long:                      1
  Native vector width float:                     1
  Native vector width double:                    1
  Max clock frequency:                           1340Mhz
  Address bits:                                  64
  Max memory allocation:                         7301444400
  Image support:                                 Yes
  Max number of images read arguments:           128
  Max number of images write arguments:          8
  Max image 2D width:                            16384
  Max image 2D height:                           16384
  Max image 3D width:                            16384
  Max image 3D height:                           16384
  Max image 3D depth:                            8192
  Max samplers within kernel:                    26591
  Max size of kernel argument:                   1024
  Alignment (bits) of base address:              1024
  Minimum alignment (bytes) for any datatype:    128
  Single precision floating point capability
    Denorms:                                     No
    Quiet NaNs:                                  Yes
    Round to nearest even:                       Yes
    Round to zero:                               Yes
    Round to +ve and infinity:                   Yes
    IEEE754-2008 fused multiply-add:             Yes
  Cache type:                                    Read/Write
  Cache line size:                               64
  Cache size:                                    16384
  Global memory size:                            8589934592
  Constant buffer size:                          7301444400
  Max number of constant args:                   8
  Local memory type:                             Scratchpad
  Local memory size:                             65536
  Max pipe arguments:                            16
  Max pipe active reservations:                  16
  Max pipe packet size:                          3006477104
  Max global variable size:                      7301444400
  Max global variable preferred total size:      8589934592
  Max read/write image args:                     64
  Max on device events:                          1024
  Queue on device max size:                      8388608
  Max on device queues:                          1
  Queue on device preferred size:                262144
  SVM capabilities:
    Coarse grain buffer:                         Yes
    Fine grain buffer:                           Yes
    Fine grain system:                           No
    Atomics:                                     No
  Preferred platform atomic alignment:           0
  Preferred global atomic alignment:             0
  Preferred local atomic alignment:              0
  Kernel Preferred work group size multiple:     64
  Error correction support:                      0
  Unified memory for Host and Device:            0
  Profiling timer resolution:                    1
  Device endianess:                              Little
  Available:                                     Yes
  Compiler available:                            Yes
  Execution capabilities:
    Execute OpenCL kernels:                      Yes
    Execute native function:                     No
  Queue on Host properties:
    Out-of-Order:                                No
    Profiling :                                  Yes
  Queue on Device properties:
    Out-of-Order:                                Yes
    Profiling :                                  Yes
  Platform ID:                                   0x7f4c522adc80
  Name:                                          gfx803
  Vendor:                                        Advanced Micro Devices, Inc.
  Device OpenCL C version:                       OpenCL C 2.0
  Driver version:                                3406.0 (HSA1.1,LC)
  Profile:                                       FULL_PROFILE
  Version:                                       OpenCL 1.2
  Extensions:                                    cl_khr_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_fp16 cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_media_ops cl_amd_media_ops2 cl_khr_image2d_from_buffer cl_khr_subgroups cl_khr_depth_images cl_amd_copy_buffer_p2p cl_amd_assembly_program

@CosmicFusion
Copy link

@CosmicFusion
hi, recently, I just find out the blender is a 3d designer app, as 3d max.
And because of my rx580 character , I do some dig for blender on gfx803 with ROCm. Unfortunately, even latest blender-3.3 said it can support gfx900, gfx803 always export a abnormal image.
Compiling OK, export image wont get right result.

Me and GloriousEggroll tried adding gfx803 to blender 3.4 , it either renders a white image or a red mess , this is blender issue not a hip one as I can compile hip code successfully on my RX 580

@rajhlinux
Copy link

rajhlinux commented Sep 17, 2022

CosmicFusion

so if you want to use your gfx803 gpu you need to :
sudo echo 'ROC_ENABLE_PRE_VEGA=1' >> /etc/environment
and no patch is needed.

Is this exclusive to Fedora only? Or can we use this on other distros?

Can be used on all distros , tested on PopOS using repo.radeon.com , and fedora using their packages , and my personal built from source ones

You think I can use OpenCL 1.2 on FreeBSD 13.1 for my RX-580 GPU? I need to run OpenCV or "Dlib" machine vision frameworks using my GPU for accelerated computations for face recognition. clinfo shows I am using clover with my gpu supporting only opencl 1.1 when it should be 2.0, using MESA drivers.

@redthing1
Copy link

Wow, I got it to work using https://github.com/rocm-arch/rocm-arch/blob/master/README.md and HSA_OVERRIDE_GFX_VERSION=10.3.0

@owariee
Copy link

owariee commented Dec 24, 2022

AFAIK, we have not removed any code intentionally. But maybe something changed in the stack and we don't validate gfx8 on ROCm, so it might not be working anymore. One thing from support point of view, each card has some duration of support. We can not continue supporting cards for more number of years as per business standards. As new cards coming into the market, we keep adding the new ones into the supported list and keep dropping the old ones after certain amount of time, which is the process.

Anyhow, I am not closing this ticket right now. Let me wait for some time. Thank you.

i dont seen this being a valid response from AMD as a consumer, i never, really, never, seen a driver for a series of cards dropping support for a feature that is marketed in the cards on purpose, so really, fix this ASAP. Community are handling your issues for you, on arch polaris works just fine with a patched package from AUR (https://archlinux.org/packages/community-testing/x86_64/rocm-opencl-runtime/)

@John-Gee
Copy link

AFAIK, we have not removed any code intentionally. But maybe something changed in the stack and we don't validate gfx8 on ROCm, so it might not be working anymore. One thing from support point of view, each card has some duration of support. We can not continue supporting cards for more number of years as per business standards. As new cards coming into the market, we keep adding the new ones into the supported list and keep dropping the old ones after certain amount of time, which is the process.
Anyhow, I am not closing this ticket right now. Let me wait for some time. Thank you.

i dont seen this being a valid response from AMD as a consumer, i never, really, never, seen a driver for a series of cards dropping support for a feature that is marketed in the cards on purpose, so really, fix this ASAP. Community are handling your issues for you, on arch polaris works just fine with a patched package from AUR (archlinux.org/packages/community-testing/x86_64/rocm-opencl-runtime)

You don't need to patch ROCm anymore, ROC_ENABLE_PRE_VEGA=1 should be enough.

@redthing1
Copy link

@CosmicFusion I attempted blender port as well. I have successfully run the 3.3 ROCm build. But when I used the same patches for 3.4 I got the same thing you said, some white and red mess.

@CosmicFusion
Copy link

CosmicFusion commented Dec 29, 2022

@CosmicFusion I attempted blender port as well. I have successfully run the 3.3 ROCm build. But when I used the same patches for 3.4 I got the same thing you said, some white and red mess.

wait do you mean you were able to successfully render on blender using 3.3?

@Mhowser
Copy link

Mhowser commented Jan 8, 2023

@CosmicFusion I attempted blender port as well. I have successfully run the 3.3 ROCm build. But when I used the same patches for 3.4 I got the same thing you said, some white and red mess.

wait do you mean you were able to successfully render on blender using 3.3?

@redthing1 what is the hardware that you used?

@CosmicFusion
Copy link

CosmicFusion commented Jan 8, 2023

@CosmicFusion I attempted blender port as well. I have successfully run the 3.3 ROCm build. But when I used the same patches for 3.4 I got the same thing you said, some white and red mess.

wait do you mean you were able to successfully render on blender using 3.3?

@redthing1 what is the hardware that you used?

Gigabyte RX 580 8GB with only the ROC_ENABLE_PRE_VEGA=1, for the record this still works in ROCm 5.4.1 for OpenCL, but i haven't tested HIP since 5.2.3 and blender 3.4 alpha or beta i don't remember, and that was without the HSA OVERRIDE env, I currently have exams so i can't test

EDIT : Ohh you are not talking to me lol

@redthing1
Copy link

@CosmicFusion I attempted blender port as well. I have successfully run the 3.3 ROCm build. But when I used the same patches for 3.4 I got the same thing you said, some white and red mess.

wait do you mean you were able to successfully render on blender using 3.3?

@redthing1 what is the hardware that you used?

@CosmicFusion I attempted blender port as well. I have successfully run the 3.3 ROCm build. But when I used the same patches for 3.4 I got the same thing you said, some white and red mess.

wait do you mean you were able to successfully render on blender using 3.3?

@redthing1 what is the hardware that you used?

Gigabyte RX 580 8GB with only the ROC_ENABLE_PRE_VEGA=1, for the record this still works in ROCm 5.4.1 for OpenCL, but i haven't tested HIP since 5.2.3 and blender 3.4 alpha or beta i don't remember, and that was without the HSA OVERRIDE env, I currently have exams so i can't test

EDIT : Ohh you are not talking to me lol

Well funny enough. He used the exact same hardware as me. Gigabyte RX 580 8GB with only the ROC_ENABLE_PRE_VEGA=1

@boxerab
Copy link
Author

boxerab commented Feb 9, 2023

Well funny enough. He used the exact same hardware as me. Gigabyte RX 580 8GB with only the ROC_ENABLE_PRE_VEGA=1

@redthing1 what Linux distribution are you using ? And how do you set ROC_ENABLE_PRE_VEGA=1 ? Do you have to set this variable before you install ?

@redthing1
Copy link

Well funny enough. He used the exact same hardware as me. Gigabyte RX 580 8GB with only the ROC_ENABLE_PRE_VEGA=1

@redthing1 what Linux distribution are you using ? And how do you set ROC_ENABLE_PRE_VEGA=1 ? Do you have to set this variable before you install ?

Yes

@boxerab
Copy link
Author

boxerab commented Feb 9, 2023

Well funny enough. He used the exact same hardware as me. Gigabyte RX 580 8GB with only the ROC_ENABLE_PRE_VEGA=1

@redthing1 what Linux distribution are you using ? And how do you set ROC_ENABLE_PRE_VEGA=1 ? Do you have to set this variable before you install ?

Yes

Great! I tried running install script from
https://docs.amd.com/bundle/ROCm-Installation-Guide-v5.4.3/page/How_to_Install_ROCm.html
but I didn't have the env variable set and it failed to find the cl device. I guess I could set it first and then try again.

@boxerab
Copy link
Author

boxerab commented Feb 10, 2023

Alright, I admit defeat. Tried to install 5.4.3 on Ubuntu 22 from install script with ROC_ENABLE_PRE_VEGA env variable set in /etc/environment. No cl device detected.

Fedora 37 works right out of the box, so I will stick with Fedora.

Unfortunately, the performance of my cl kernels is 50% of what I used to get with ROCm 3.x

@CosmicFusion
Copy link

Alright, I admit defeat. Tried to install 5.4.3 on Ubuntu 22 from install script with ROC_ENABLE_PRE_VEGA env variable set in /etc/environment. No cl device detected.

Fedora 37 works right out of the box, so I will stick with Fedora.

Unfortunately, the performance of my cl kernels is 50% of what I used to get with ROCm 3.x

You need to run this

sudo usermod -aG video $LOGNAME
sudo usermod -aG render $LOGNAME
reboot

@boxerab
Copy link
Author

boxerab commented Feb 10, 2023

You need to run this

sudo usermod -aG video $LOGNAME
sudo usermod -aG render $LOGNAME
reboot

Thanks, I will stick with Fedora, as I prefer it to clunky old Ubuntu. I am assuming that perf won't change between the two distributions.

@boxerab
Copy link
Author

boxerab commented Feb 13, 2023

As a matter of fact, running my kernels on Fedora 37 with latest packaged ROCm gives same performance as with 3.x ROCm.

I am quite happy with the Fedora 37 packages and their support for Polaris cards.
Going to close this.

@boxerab boxerab closed this as completed Feb 13, 2023
@Mhowser
Copy link

Mhowser commented Feb 13, 2023

So we must use Fedora if we want to use our cards for anything compute related? Can these packages be ported over to other distros?

@boxerab
Copy link
Author

boxerab commented Feb 13, 2023

So we must use Fedora if we want to use our cards for anything compute related? Can these packages be ported over to other distros?

Some here have got it working for officially supported Ubuntu 22 with AMD packages, unfortunately I only had luck with Fedora. Also I believe there is support on Arch. For other distros I'm not sure.

@tsl0922
Copy link

tsl0922 commented Apr 20, 2023

So we must use Fedora if we want to use our cards for anything compute related? Can these packages be ported over to other distros?

Some here have got it working for officially supported Ubuntu 22 with AMD packages, unfortunately I only had luck with Fedora. Also I believe there is support on Arch. For other distros I'm not sure.

I can confirm that rocm-5.4.3 works on Ubuntu 22.04.2 LTS / Python 3.10.6 for stable-diffusion-webui, using a manually compiled pytorch 1.13.1 with PYTORCH_ROCM_ARCH=gfx803.

https://github.com/tsl0922/pytorch-gfx803

@vittorio88
Copy link

@tsl0922 Hey, so it seem that with ROCM_ENABLE_PRE_VEGA enabled, RX580 and family works just fine?

Why is it that TORCH requires to be manually compiled?
Just to add: export PYTORCH_ROCM_ARCH=gfx803 ?

Why is this not the default for Torch, and do you know if/where this was changed in Torch code?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests