Tests failing on GPU: Binary format for key='0', ident='' is not recognized

[iamholger]

Hi,

building sycl-blas works fine on my system but the tests only work on the host device (that is all tests fail on either
my nvidia or intel gpu).
I see the warning

/home/hschulz/SYCL/ComputeCpp-CE-2.1.0-x86_64-linux-gnu/bin/spirv-ll-tool: /lib64/libtinfo.so.5: no version information available (required by /home/hschulz/SYCL/ComputeCpp-CE-2.1.0-x86_64-linux-gnu/bin/spirv-ll-tool)

at every build step but according to codeplay's support pages this should not be an issue.

Testing e.g.
./test/unittest/blas3_gemm_test --device=nvidia:gpu
results in messages of the form

[----------] Global test environment set-up.
[----------] 72 tests from Gemm/GemmSmallBetaNonZeroLDMatchFloat
[ RUN      ] Gemm/GemmSmallBetaNonZeroLDMatchFloat.test/0
Device vendor: NVIDIA Corporation
Device name: GeForce GTX 960M
Device type: gpu
Error: [ComputeCpp:RT0100] Failed to build program (<Build log for program 0xf45840 (size: 65) error   : Binary format for key='0', ident='' is not recognized>)
Value mismatch at index 0: -0.814363; expected 101.708
../test/unittest/blas3/blas3_gemm_common.hpp:157: Failure

I am following your instructions to build with cmake using this command, the logfile is attached:
cmake.log
cmake -GNinja ../ -DComputeCpp_DIR=$PWD/../../ComputeCpp-CE-2.1.0-x86_64-linux-gnu/ -DCMAKE_INSTALL_PREFIX=$PWD/../local -DBLAS_ENABLE_TESTING=ON

My system is running Fedora30 and computecpp_info reports:

GLIBC version: 2.29
GLIBCXX: 20190605
This version of libstdc++ is not supported.

[...]

  Device is supported                     : UNTESTED - Untested OS
  Bitcode targets                         : ptx64 
  CL_DEVICE_NAME                          : GeForce GTX 960M
  CL_DEVICE_VENDOR                        : NVIDIA Corporation
  CL_DRIVER_VERSION                       : 440.64
  CL_DEVICE_TYPE                          : CL_DEVICE_TYPE_GPU 

  Device is supported                     : NO - No bitcode target found
  Bitcode targets                         : 
  CL_DEVICE_NAME                          : pthread-Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
  CL_DEVICE_VENDOR                        : GenuineIntel
  CL_DRIVER_VERSION                       : 1.3-pre
  CL_DEVICE_TYPE                          : CL_DEVICE_TYPE_CPU 

  Device is supported                     : UNTESTED - Untested OS
  Bitcode targets                         : spir32 
  CL_DEVICE_NAME                          : Intel(R) HD Graphics Skylake Halo GT2
  CL_DEVICE_VENDOR                        : Intel
  CL_DRIVER_VERSION                       : 1.3
  CL_DEVICE_TYPE                          : CL_DEVICE_TYPE_GPU 

I heard today that the glibc warning should not matter, too.
I am not sure why things are going wrong and would greatly appreciate any help.

[DuncanMcBain]

When using nvidia GPUs, you need to output PTX instead of the default SPIR. You can do this by adding -DCOMPUTECPP_BITCODE=ptx64 to your CMake. There's not really any way for CMake to determine which to output since that entirely depends on which devices are available at runtime, while this decision must be made at compile time.

[iamholger]

Ah gotcha! Thanks for the quick reply. The tests work now for me.

What I do see during compilation with ninja

make -GNinja ../ -DComputeCpp_DIR=$PWD/../../ComputeCpp-CE-2.1.0-x86_64-linux-gnu/ -DCMAKE_INSTALL_PREFIX=$PWD/../local -DBLAS_ENABLE_TESTING=ON  -DCOMPUTECPP_BITCODE=ptx64
ninja

is a ton of warnings of the sort

warning: [Computecpp:CC0035]: Intrinsic llvm.fmuladd.v4f32 has been generated in function _ZN4blas4GemmINS_10MatrixViewIfN2cl4sycl8accessorIfLi1ELNS3_6access4modeE2ELNS5_6targetE1ELNS5_11placeholderE1EEEiNS_9col_majorEEESB_Lb0ELb0ELb0ELi64ENS_4TileILi2ELi2ELi4ELi4ELi1ELi1ELi4ELi4EEELb0ELb1EfLb0ELi1ELi1ELi2ELi4ELi1EE13compute_panelILb1EZNSE_4evalE__ns1NS3_7nd_itemILi1EEEEUlRKiSJ_E_PU3AS1fSM_EE__ns1vSK_iiiiiSM_SM_SM_ which is illegal in SPIR and may result in a compilation failure [-Wsycl-undef-func]

Is that related to the more moderate compute capabilities of my nvidia gpu?
And maybe more generally, do you have a suggestion for reading material
for this sort of question?

Thanks!

[DuncanMcBain]

That warning is likely harmless in this case. Anything that happens at compile time has no knowledge of your hardware, so it's not that. I guess we output the warning because it might go wrong at runtime, in which case there will be a record of it in the compilation log (but likely won't). If you do have failures when trying to run these new binaries, please let us know here!

For reading material, the particular error you had does have a few results on Google, and some of those point to another Codeplay repository and hopefully also our support site. Our site has material about compiling for different devices, which should include Nvidia devices, so that might be what you're looking for!

[iamholger]

Thanks a lot, that's super clear now.