Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabling xla on tensorflow-rocm with Rocm 5.2.3? #1796

Closed
ffleader1 opened this issue Aug 23, 2022 · 7 comments
Closed

Enabling xla on tensorflow-rocm with Rocm 5.2.3? #1796

ffleader1 opened this issue Aug 23, 2022 · 7 comments

Comments

@ffleader1
Copy link

ffleader1 commented Aug 23, 2022

I have a gfx1030 card l, Rocm 5.2.3 and currently running into xla issues with tensorflow.
My GPU can do pytorch model fine, can do tensorflow fine, without xla. With xla however, I got this error: bitcode module not found at ./opencl.bc

Am I missing something during the install process (because I installed with rocm usecase).

How to get my device running tensorflow training with xla?
Thank you

@aoolmay
Copy link

aoolmay commented Aug 26, 2022

Can confirm, error is reproducible on 6800XT and 6950XT.
Fresh installation: ROCm 5.2.3 + tensorflow-rocm 2.9.2 + Linux 5.11.0-27 (as per installation instructions)
Last working setup, ROCm 5.1.3 + tensorflow 2.9.1

Loss of XLA capability results in ~7% performance loss on most tensorflow workloads.

@Mushoz
Copy link

Mushoz commented Dec 4, 2022

Same issue here. Any idea on when we can expect a fix?

@Mushoz
Copy link

Mushoz commented Dec 5, 2022

I would have expected this issue to get way more attention if it affected everyone. Maybe it doesn't? A question for you @aoolmay and @ffleader1 : What distro are you guys running? I am using Archlinux myself, which is technically unsupported. I am curious if this also affects people that are running supported distributions.

Lastly, my models currently refuse to run at all. How do I explicitly disable XLA so that it at least works again?

@macmv
Copy link

macmv commented Feb 28, 2023

I've had the same issue (also running arch with a 6800xt), and setting ROCM_PATH to /opt/rocm fixed it for me.

@Epliz
Copy link

Epliz commented Mar 25, 2023

hit the same issue with a MI100 GPU, setting ROCM_PATH also helped

@nartmada
Copy link
Collaborator

Hi @ffleader1, please check latest ROCm Documentation and ROCm 6.0.0 to see if your query has been resolved. If resolved, please close the ticket. Thanks.

@ffleader1
Copy link
Author

Hi @ffleader1, please check latest ROCm Documentation and ROCm 6.0.0 to see if your query has been resolved. If resolved, please close the ticket. Thanks.

I am no longer have access to my original PC
But guessing the complaints were not that much anymore, I guess it's fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants