HCC STATUS_CHECK Error: HSA_STATUS_ERROR_INCOMPATIBLE_ARGUMENTS

[PhilipDeegan]

Saxpy example fails with message

### HCC STATUS_CHECK Error: HSA_STATUS_ERROR_INCOMPATIBLE_ARGUMENTS (0x100d) at file:/home/scchan/code/github/hcc-roc-1.4.x/hcc/lib/hsa/mcwamp_hsa.cpp line:2511

Thread 1 "hcc" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:58
58      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:58
#1  0x00007ffff6f3e40a in __GI_abort () at abort.c:89
#2  0x00007ffff679ed31 in Kalmar::HSADevice::BuildOfflineFinalizedProgramImpl(void*, int) () from /opt/rocm/hcc-lc/lib/libmcwamp_hsa.so
#3  0x00007ffff679b6b5 in Kalmar::HSADevice::BuildProgram(void*, void*, bool) () from /opt/rocm/hcc-lc/lib/libmcwamp_hsa.so
#4  0x0000000000404266 in Kalmar::KalmarBootstrap::KalmarBootstrap() ()
#5  0x0000000000403817 in __hcc_shared_library_init ()
#6  0x000000000040811d in __libc_csu_init ()
#7  0x00007ffff6f2a240 in __libc_start_main (main=0x405290 <main>, argc=1, argv=0x7fffffffe548, init=0x4080d0 <__libc_csu_init>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe538) at ../csu/libc-start.c:247
#8  0x0000000000407ffa in _start ()

[whchung]

Could you let me know your hardware configuration? By default HCC produces GCN ISA for Fiji ASIC with AMD:AMDGPU:8:0:3 ISA version string. The error message you saw are mostly related to incompatible GCN ISA version. You can use:

--amdgpu-target=AMD:AMDGPU:7:0:0 for Kaveri devices
--amdgpu-target=AMD:AMDGPU:7:0:1 for Hawaii devices
--amdgpu-target=AMD:AMDGPU:8:0:1 for Carrizo devices
--amdgpu-target=AMD:AMDGPU:8:0:3 for Fiji / Polaris devices

[PhilipDeegan]

That's probably it, when I build it hcc/etc from source I use "AMD:AMDGPU:7:0:1" and it was fine with 1.3.1, haven't been able to finish building 1.4 from source due to #196

[PhilipDeegan]

Linking with "--amdgpu-target=AMD:AMDGPU:7:0:1" has resolved this.

[pszi1ard]

I'm running into the same issues while trying to use a machine with a Fiji and a Hawaii GPU. I tried to compile passing either or both --amdgpu-target=AMD:AMDGPU:7:0:1 --amdgpu-target=AMD:AMDGPU:8:0:3, but I still get the HSA_STATUS_ERROR_INCOMPATIBLE_ARGUMENTS error.

[whchung]

@pszi1ard may I understand the output of hcc --version in your installation? If it shows clang-3.5, then it does not support one single binary with multiple ISAs. It's only supported in Clang 4.0-based HCC, which should be included in the upcoming ROCm 1.5.

In case you are not afraid of building the compiler from source, you can find a useful script at:
https://github.com/GPUOpen-ProfessionalCompute-Tools/HIP/releases/tag/preview_1.0.17066
which helps you build HCC, HIP for that particular developer preview release.

[pszi1ard]

Indeed, it's based on clang 3.5.

$ /opt/rocm/hcc/bin/hcc --version | head -n1
HCC clang version 3.5.0 (based on HCC 0.10.16501-81f0a2f-02246a0 LLVM 3.5.0svn)

However I still can't run any hcc program, even if I specify only one of the target. Is that because the binary ends up with code for only one target arch, but by default the code will execute on both detected devices? How can I restrict which devices can the runtime access? Does GPU_DEVICE_ORDINAL work?

[whchung]

@pszi1ard Given your system configuration ( Fiji + Hawaii ), I would recommend you to try build HIP & HCC from source, before we officially publish Clang 4.0 (or 5.0)-based HCC in the next ROCm release.

could you open up a new ticket and post relevant information there? we can continue the discussion over there.

[PhilipDeegan]

I have a source build here change line 67 to target different GPU by default.

Personally I'd never mix GPUs in a system. didn't even know it was possible.

[pszi1ard]

@pszi1ard Given your system configuration ( Fiji + Hawaii ), I would recommend you to try build HIP & HCC from source, before we officially publish Clang 4.0 (or 5.0)-based HCC in the next ROCm release.

I'm not sure I'll have the time, but I will report back if I do. Not sure what to open a ticket for, but here's an RFE for selecting devices using and env var: #249)/

I have a source build here change line 67 to target different GPU by default.

Thanks, I might try that!

Personally I'd never mix GPUs in a system. didn't even know it was possible.

Why? I have dev boxes with different gen devices exactly in order to facilitate testing. Not supporting such a setup is just counter-intuitive.

[scchan]

@pszi1ard You could also get a newer hcc installer package from the preview HIP release so you won't have to build hcc from source
https://github.com/GPUOpen-ProfessionalCompute-Tools/HIP/releases