New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MIOpen using wrong ISA for certain convolution operations #117
Comments
@zjing14 Please take a look. |
@daniellowell The Conv3x3AsmWrW is not supported on RX4/560 (i.e., Fiji/Polaris Arch). We need to disable the kernel on Fiji platform. Will create a PR. |
@89Mods We will fix this in the next release. In the mean time can you see if setting the environment variable: |
@daniellowell I tried setting the environment variable as you suggested, but while it does hide the errors, nothing works correctly anymore. I also tried downgrading back to MIOpen 2.0.0, but doing that now causes a Segfault in PyTorch, so I'm pretty much stuck with the broken 2.0.1 release. |
Resolved in 2.1.0 |
@89Mods please try the latest release. If your issue is not resolved please feel free to reopen this issue. |
I've just updated MIOpen to the latest version (package version 2.0.1.7405), which now apparently uses the wrong ISA for my GPU. I normally work with PyTorch for all my projects, however, this issue occurs while MIOpen is tuning the Perf. Db before even running my python code. As soon as it tries to do so, it repeatedly runs into errors similar to this:
This goes on for several pages, full log is here: https://pastebin.com/dughVHes
I'm running an RX 470, though the error also occurs on a RX 560.
The only way I've found to replicate this is to try to run a convolution with exactly 3 output channels and 128 input channels, otherwise, this error occurs seemingly at random, as most operations complete successfully and only some trigger the error (as seen in the full log).
The text was updated successfully, but these errors were encountered: