Spack builds with ROCM 6.0

[PaulMullowney]

I am trying to build with ROCM 6.0. I am seeing:

==> Error: NoHeadersError: Unable to locate rocsparse headers in /home/amd/rocm-6.0.0-12969/include

/home/users/pmullown/spack-manager/spack/var/spack/repos/builtin/packages/hypre/package.py:245, in configure_args:
242 rocm_inc = ""
243 for pkg in rocm_pkgs:
244 if "^" + pkg in spec:

245 rocm_inc += spec[pkg].headers.include_flags + " "

The issue, I think, is that header files are now in:
/home/amd/rocm-6.0.0-12969/include/rocsparse/rocsparse.h

With ROCM 5.4.3, the paths often look like:
/opt/rocm-5.4.3/rocsparse/include/rocsparse.h

I'm not sure how to fix the hypre package.py to make this work. Any help is appreciated!

[victorapm]

Hi Paul, is rocm 6.0 available already?

[PaulMullowney]

Not yet.

[victorapm]

I won't be able to test then, but it seems these includes should be updated:

hypre/src/utilities/_hypre_utilities.hpp

Lines 213 to 230 in 40ef37a

	using hypre_DeviceItem = void*;
	#include <hip/hip_runtime.h>
	#if defined(HYPRE_USING_ROCBLAS)
	#include <rocblas/rocblas.h>
	#endif
	#if defined(HYPRE_USING_ROCSPARSE)
	#include <rocsparse/rocsparse.h>
	#endif
	#if defined(HYPRE_USING_ROCSOLVER)
	#include <rocsolver/rocsolver.h>
	#endif
	#if defined(HYPRE_USING_ROCRAND)
	#include <rocrand/rocrand.h>
	#endif

[victorapm]

Oh wait, they are already in the new format you mentioned. Did you manage to build hypre outside spack?

[PaulMullowney]

no. I am struggling with:
hipcc --amdgpu-target=gfx942 -fPIC -O2 -x hip -std=c++14 -DHAVE_CONFIG_H -I.. -I./.. -I./../struct_mv -I. -I/home/amd/rocm-6.0.0-12969/include -I/opt/cray/pe/mpich/8.1.28/ofi/amd/5.0/include -c device_utils.c -o device_utils.obj

Warning: The --amdgpu-target option has been deprecated and will be removed in the future. Use --offload-arch instead.
clang: error: cannot find HIP runtime; provide its path via '--rocm-path', or pass '-nogpuinc' to build without HIP runtime
clang: error: cannot find HIP runtime; provide its path via '--rocm-path', or pass '-nogpuinc' to build without HIP runtime

[victorapm]

Is hip/hip_runtime.h located under /home/amd/rocm-6.0.0-12969/include?

[victorapm]

PS: we need to change --amdgpu-target to --offload-arch, but that seems a separate issue :)

[PaulMullowney]

[pmullown@x1000c2s1b1n0 ~]$ ls /home/amd/rocm-6.0.0-12969/include/hip/hip_runtime.h
/home/amd/rocm-6.0.0-12969/include/hip/hip_runtime.h

[victorapm]

Ok! Can you compile other HIP codes? How about a simple "hello_world"?

[PaulMullowney]

Yes. I can build and run code with this rocm.

[victorapm]

Good! That's all that I could think of. Maybe others have more ideas... Hope you manage to find the issue. Otherwise, I can take a look when this becomes public.

[PaulMullowney]

Fyi, I was able to get hypre to build. The issue was in how a HIP_PATH variable was getting set in spack. I'm now debugging run time failures on new hardware.

[victorapm]

Great! Thanks for the update!