ROCm & Windows Support

[Looong01]

🚀 The feature, motivation and pitch

AMD has release ROCm windows support, as docs.amd.com shows:

Please add PyTorch support of Windows on AMD GPUs!

Alternatives

No response

Additional context

No response

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang

[hongxiayang]

Windows Support for PyTorch is currently not available. This includes WSL2 based workflows.
AMD is continuing to invest in Windows support including the PyTorch ecosystem.
For details or future updates, please check: https://rocm.docs.amd.com/en/latest/rocm.html#rocm-on-windows

[Cyberhan123]

Is there any progress?

[Looong01]

Nope, waiting for pytorch official’s support. Loong, send by Mail App<https://krs.microsoft.com/redirect?id=P5Ie3-7G> of Windows 11 Edition 发件人: ***@***.***> 发送时间: 2023年8月18日 15:04 收件人: ***@***.***> 抄送: ***@***.***>; ***@***.***> 主题: Re: [pytorch/pytorch] ROCm & Windows Support (Issue #106608) Is there any progress? ― Reply to this email directly, view it on GitHub<#106608 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQSYOXYHIH35Z2ATNZ4RY2LXV4HXLANCNFSM6AAAAAA3D5M4AY>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[gblssroman]

Waiting...

[maxime-fleury]

They MIOPS and other things to be implemented on windows AMD to implement pytorch @rtccreator

[estatic]

and now? :)

[1823616178]

Why does AMD work on Linux but not win

[sascharo]

What's the current state of this?

[pein0saga]

Why does AMD work on Linux but not win

It is said that AMD was focused on linux arc (ex:PS3, PS4) years ago so their tech documents on WINDOWS were very poor.

[kjhanjee]

Any updates on this?

I recently moved from cuda to rocm and I'm not liking the experience one bit on linux. Ubuntu just hangs completely if my code is about to go into OOM error even on an RX 7900 XTX. My rtx3070 did better than this on windows. Was easily able to scale using the shared gpu memory on windows for training. Now my pytorch transformer is crashing with even 320M parameters on fp32.

[sascharo]

I recently moved from cuda to rocm and I'm not liking the experience one bit on linux. Ubuntu just hangs completely if my code is about to go into OOM error even on an RX 7900 XTX. My rtx3070 did better than this on windows. Was easily able to scale using the shared gpu memory on windows for training. Now my pytorch transformer is crashing with even 320M parameters on fp32.

Which Ubuntu version? Native Linux or WSL2?

[kjhanjee]

I recently moved from cuda to rocm and I'm not liking the experience one bit on linux. Ubuntu just hangs completely if my code is about to go into OOM error even on an RX 7900 XTX. My rtx3070 did better than this on windows. Was easily able to scale using the shared gpu memory on windows for training. Now my pytorch transformer is crashing with even 320M parameters on fp32.

Which Ubuntu version? Native Linux or WSL2?

Native Linux

[sascharo]

Native Linux

I hope someone can help you figure it out. ROCm 6 will apparently be released this month; hopefully, Radeon support will be more mature.

I'm still hesitant to get a RX 7900 XTX. The experiences over the past 10+ years I had with AMD's promises when it came to their software stack taught me to be careful. 😏

[kjhanjee]

Native Linux

I hope someone can help you figure it out. ROCm 6 will apparently be released this month; hopefully, Radeon support will be more mature.

I'm still hesitant to get a RX 7900 XTX. The experiences over the past 10+ years I had with AMD's promises when it came to their software stack taught me to be careful. 😏

Yeah, I would say please hold on to the thought of getting AMD just yet though don't get Nvidia either. AMD needs to get their act together and the community needs to start pushing more for AMD until they become like Nvidia and then we will support Intel.

All said and done, Pytorch should also pitch in to get the libraries on all operating systems so that the general peeps like me can keep experimenting and figuring out where all things can be improved.

[Cyberhan123]

Obviously AMD is arrogant and ignore Windows users. Intel already supports Windows.I don't have any good feelings towards AMD now :)
ROCm/ROCm#666
ROCm/ROCm#2405

[kjhanjee]

Obviously AMD is arrogant and ignore Windows users. Intel already supports Windows.I don't have any good feelings towards AMD now :)
ROCm/ROCm#666
ROCm/ROCm#2405

I honestly hope to god somebody in the higher management in AMD sees your comment and gets the common market sentiment against them

[Cyberhan123]

@kjhanjee
I put a lot of effort into compatibility with AMD hip. I don't know how to use C/C++ and cmake at all, but now I know it after studying :), because in order to really use AMD gpus, I can only choose ggml. However, the performance of AMD drivers on Windows makes me feel frustrated.
I provide the complete rocm build process and github actions in this project:
https://github.com/saharNooby/rwkv.cpp
But the performance test failed to beat NVIDIA 3060 Ti 8 GB.
I wrote a github action specifically for building rocm, but after reading the documentation, I found that the funny thing is that architectures before gfx1032 do not support hip.
https://github.com/Cyberhan123/rocm-toolkit

[Cyberhan123]

Now I have completely surrendered to Nvidia. I plan to sell my 7900xtx soon. Because of Black Friday, GTX4060Ti 16G is enough. It is cheap, has large memory, and even has a complete CUDA and a complete ecosystem.

[kjhanjee]

@kjhanjee
I put a lot of effort into compatibility with AMD hip. I don't know how to use C/C++ and cmake at all, but now I know it after studying :), because in order to really use AMD gpus, I can only choose ggml. However, the performance of AMD drivers on Windows makes me feel frustrated.
I provide the complete rocm build process and github actions in this project:
https://github.com/saharNooby/rwkv.cpp
But the performance test failed to beat NVIDIA 3060 Ti 8 GB.
I wrote a github action specifically for building rocm, but after reading the documentation, I found that the funny thing is that architectures before gfx1032 do not support hip.
https://github.com/Cyberhan123/rocm-toolkit

Great thing man it will help out a lot of people. I was able to set up ROCm with the 7900xtx. And now I am working on deep learning using ubuntu. This is where it gets tricky, the os hangs completely or crashes gpu drivers if it is about to get into OOM.

[Cyberhan123]

Great thing man it will help out a lot of people. I was able to set up ROCm with the 7900xtx. And now I am working on deep learning using ubuntu. This is where it gets tricky, the os hangs completely or crashes gpu drivers if it is about to get into OOM.

This is not our problem, cheap and paper performance makes me no longer believe in AMD. It makes me waste too much time. To be honest, I suggest you try 4060Ti 16G. It allows you to play Apex and run/train ai at the same time.

[maxime-fleury]

For those like myself that have struggled to get AMD inference running locally on Windows, this may be your lucky day. Below is the git for the new transformers release.

https://github.com/huggingface/transformers/releases/tag/v4.36.0?utm_source=tldrai

[kjhanjee]

Great thing man it will help out a lot of people. I was able to set up ROCm with the 7900xtx. And now I am working on deep learning using ubuntu. This is where it gets tricky, the os hangs completely or crashes gpu drivers if it is about to get into OOM.

This is not our problem, cheap and paper performance makes me no longer believe in AMD. It makes me waste too much time. To be honest, I suggest you try 4060Ti 16G. It allows you to play Apex and run/train ai at the same time.

I hope you wouldn't. I know switching to the winning side might be great but for the longer run, and at least in the ai world, pushing for the underdog might just work out.

[Looong01]

@kjhanjee
I put a lot of effort into compatibility with AMD hip. I don't know how to use C/C++ and cmake at all, but now I know it after studying :), because in order to really use AMD gpus, I can only choose ggml. However, the performance of AMD drivers on Windows makes me feel frustrated.
I provide the complete rocm build process and github actions in this project:
https://github.com/saharNooby/rwkv.cpp
But the performance test failed to beat NVIDIA 3060 Ti 8 GB.
I wrote a github action specifically for building rocm, but after reading the documentation, I found that the funny thing is that architectures before gfx1032 do not support hip.
https://github.com/Cyberhan123/rocm-toolkit

Actually, u can use environment variable changing to make it become gfx1030

[Cyberhan123]

Actually, u can use environment variable changing to make it become gfx1030

@Looong01 L
But my friend’s gpu is 6700xt，I set the build options like this:

https://github.com/saharNooby/rwkv.cpp/blob/master/.github/workflows/build.yml#L189

-G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DRWKV_HIPBLAS=ON -DCMAKE_BUILD_TYPE=Release -DAMDGPU_TARGETS="gfx1100;gfx1102;gfx1030"

His gpu doesn't work. Maybe I set it up wrong. Can you give me some advice.

[Cyberhan123]

For those like myself that have struggled to get AMD inference running locally on Windows, this may be your lucky day. Below is the git for the new transformers release.

https://github.com/huggingface/transformers/releases/tag/v4.36.0?utm_source=tldrai

ohhh,Does this mean that pytorch supports hip?

[Looong01]

Actually, u can use environment variable changing to make it become gfx1030

@Looong01 L
But my friend’s gpu is 6700xt，I set the build options like this:

https://github.com/saharNooby/rwkv.cpp/blob/master/.github/workflows/build.yml#L189

-G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DRWKV_HIPBLAS=ON -DCMAKE_BUILD_TYPE=Release -DAMDGPU_TARGETS="gfx1100;gfx1102;gfx1030"

His gpu doesn't work. Maybe I set it up wrong. Can you give me some advice.

In ubuntu you can '''export HSA_OVERRIDE_GFX_VERSION=10.3.0'''.
In Windows, you can try the same thing

[Cyberhan123]

In ubuntu you can '''export HSA_OVERRIDE_GFX_VERSION=10.3.0'''. In Windows, you can try the same thing

Thx

[sascharo]

ROCm 6.0.0 is out, with only two Radeon cards supported under Linux, the RX 7900 XTX and the VII:
rocm.docs.amd.com/en/latest/
Are they going backwards when it comes to GPU support? I thought the last version 5 release also supported the RX 7900 XT.

[Cyberhan123]

Windows rocm can run stable diffusion now：https://github.com/leejet/stable-diffusion.cpp

[SakuranomiyaMaika]

Windows rocm can run stable diffusion now：https://github.com/leejet/stable-diffusion.cpp

how to use it?

[Comodore125]

Any news regarding pytorch? anyway I also dont understand why linux support be that lame. (less HW than with windows as I checked.)

[javag97]

Sucks. Going to run a Linux distribution to get this to work. if anyone has recommendations on distributions. I have an RX 7900 XTX probably looking at Ubuntu

[sascharo]

Sucks. Going to run a Linux distribution to get this to work. if anyone has recommendations on distributions. I have an RX 7900 XTX probably looking at Ubuntu

Ubuntu has the broadest support these days. I would give that a shot first.

[johnnynunez]

rocm6.1 will be compatible with windows, so now you can compile MiOpen and AMDMIGraphX for windows

[kjhanjee]

Sucks. Going to run a Linux distribution to get this to work. if anyone has recommendations on distributions. I have an RX 7900 XTX probably looking at Ubuntu

Ubuntu has the broadest support these days. I would give that a shot first.

I already tried that with Ubuntu 22.04 but had to give up due to some very basic memory leakages. My 7900xtx was getting ooms even with <1B scale models

[javag97]

I will try both Linux Mint/Kubuntu and see how that goes. I went ahead and grabbed an extra m.2 drive just to play around with linux.

[javag97]

I will try both Linux Mint/Kubuntu and see how that goes. I went ahead and grabbed an extra m.2 drive just to play around with linux.

I am going for Kubuntu 22.04 LTS and see how it goes

[javag97]

Getting pytorch to work with AMD GPU was not fun and was hacky at best. Hopefully this gets more feature mature in the future

[cccyberwolke]

Whats the status here? Anything that is stopping development on getting rocm to work with pytorch on Windows?
I would be willing to put in some work myself, other applications using rocm on Windows are working perfectly fine, getting pytorch to work is the only big thing missing i feel like

[supernovae]

+1 looking forward to ROCm 6/windows support with PyTorch 👍 - Any way we can help?

[welnaseth]

+1 I was trying to get pytorch on wsl w/rocm working before I realized wsl doesn't support and gpus correctly to use rocm. It would be great if the pieces have all fallen in place for native windows pytorch to support rocm.

[CodingPerNight]

+1 looking forward to ROCm 6/windows support with PyTorch 👍

[Gonzalo1987]

Something new?
Thanksss!

[jeus1609]

Excuse me, does anyone know if I can use ROCm in WSL2? for pytorch

[Bocchi-Chan2023]

still waiting...

[kjhanjee]

Excuse me, does anyone know if I can use ROCm in WSL2? for pytorch

Not as of now, AMD drivers aren't compatible with WSL2 to make GPU available properly to the linux kernel

[kjhanjee]

still waiting...

It might not happen anytime soon. The problem is on AMD's end where they have to bring MIOpen to windows first and then it will be possible to create a pytorch source that can be built out for ROCm and Windows

[sascharo]

still waiting...

I'm optimistic that issues like that will be solved by 2034.

[Bocchi-Chan2023]

still waiting...

I'm optimistic that issues like that will be solved by 2034.

I'm from 2069 and ROCm is still not available in Pytorch for Windows.
So I'm here to change the future