Add CMake support for HIP via CHIP-SPV

[pvelesko]

Summary

Add CMake support for compiling LAMMPS for Intel GPUs via CHIP-SPV implementation of HIP.

Related Issue(s)

none

Author(s)

Paulius Velesko
PGLC for Argonne ALCF
pvelesko@pglc.io

Licensing

By submitting this pull request, I agree, that my contribution will be included in LAMMPS and redistributed under either the GNU General Public License version 2 (GPL v2) or the GNU Lesser General Public License version 2.1 (LGPL v2.1).

Backward Compatibility

Backwards compatible.

Implementation Notes

When using the HIP backend it is now mandatory to have HIP_PATH set in the environment. Previously, it would default to /opt/rocm/ but this default no longer makes sense now that there are multiple HIP implementations.

Post Submission Checklist

• The feature or features in this pull request is complete
• Licensing information is complete
• Corresponding author information is complete
• The source code follows the LAMMPS formatting guidelines
• Suitable new documentation files and/or updates to the existing docs are included
• The added/updated documentation is integrated and tested with the documentation build system
• The feature has been verified to work with the conventional build system
• The feature has been verified to work with the CMake based build system
• Suitable tests have been added to the unittest tree.
• A package specific README file has been included or updated
• One or more example input decks are included

Further Information, Files, and Links
https://github.com/CHIP-SPV/chip-spv

[akohlmey]

@pvelesko Thanks for your submission. Please note that we cannot merge this unless the documentation is updated accordingly. It would also be nice, if you could discuss the differences to using OpenCL. From the perspective of a scientist user with minimal experience in compiling software (i.e. a large part of the LAMMPS user community) using OpenCL may be preferred as its pre-requisites are minimal.

We would also appreciate it very much, if you could also add support for the conventional build system (should not be too difficult considering the small number of changes for CMake).

[pvelesko]

@akohlmey I updated the documentation and addressed the PR comments. I also modified the Makefile for GPU and it seems to compile the GPU stuff ok but I run into some issues with MPI calls unrecognized option ‘--showme:compile’

[akohlmey]

@akohlmey I updated the documentation and addressed the PR comments. I also modified the Makefile for GPU and it seems to compile the GPU stuff ok but I run into some issues with MPI calls unrecognized option ‘--showme:compile’

Can you provide some details about this? I.e. which commands did you run where and what was the output to the screen?

[pvelesko]

@akohlmey

pvelesko@jlselogin4:~/space/lammps/lib/gpu> make -f Makefile.hip
clang-14: error: unsupported option '--showme:compile'
/gpfs/jlse-fs0/users/pvelesko/install/HIP/clang14/chip-spv-1.2/bin/hipcc -Wno-deprecated-declarations -fopenmp -DMPI_GERYON -DUCL_NO_EXIT -fPIC -O3 -DUSE_HIP -D_SINGLE_DOUBLE  -DLAMMPS_SMALLBIG  -I./ -I/gpfs/jlse-fs0/users/pvelesko/install/HIP/clang14/chip-spv-1.2/../include -o obj/lal_zbl.o -c lal_zbl.cpp -I./obj

[akohlmey]

@akohlmey

pvelesko@jlselogin4:~/space/lammps/lib/gpu> make -f Makefile.hip
clang-14: error: unsupported option '--showme:compile'
/gpfs/jlse-fs0/users/pvelesko/install/HIP/clang14/chip-spv-1.2/bin/hipcc -Wno-deprecated-declarations -fopenmp -DMPI_GERYON -DUCL_NO_EXIT -fPIC -O3 -DUSE_HIP -D_SINGLE_DOUBLE  -DLAMMPS_SMALLBIG  -I./ -I/gpfs/jlse-fs0/users/pvelesko/install/HIP/clang14/chip-spv-1.2/../include -o obj/lal_zbl.o -c lal_zbl.cpp -I./obj

Please see these variable definitions in Makefile.hip:

MPI_COMP_OPTS = $(shell mpicxx --showme:compile)
MPI_LINK_OPTS = $(shell mpicxx --showme:link)

Apparently Makefile.hip assumes that people use OpenMPI while you seem to be using MPICH 😉

Looks like this needs a little "GNU make magic(tm)" to discriminate between those and have alternatives to retrieve the MPI settings from the MPI compiler wrappers.

[pvelesko]

Looks like this needs a little "GNU make magic(tm)"
Does this need to happen in this PR?

[akohlmey]

Looks like this needs a little "GNU make magic(tm)"
Does this need to happen in this PR?

Do you have some spare magic lying around that you could use to implement it? ;-)

Is there an urgency to have this merged right away? I can probably come up with some way (we've done similar things elsewhere, e.g. to detect/select the python executable) if I can find some time...

[pvelesko]

Is there an urgency to have this merged right away?
no urgency

@akohlmey I can cast lvl 1 attempt.

https://www.mpich.org/static/docs/v3.1/www1/mpicxx.html

We have -compile_info and -link_info for MPICH so now we just need to figure out if mpicxx is MPICH or OpenMPI.

mpicxx -v
mpicxx for MPICH version 4.1a1
Intel(R) oneAPI DPC++/C++ Compiler 2022.1.0 (2022.x.0.20220629)
Target: x86_64-unknown-linux-gnu

Looks like we can get it from -v but I don't have OpenMPI installed to check what the equivalent does.

[akohlmey]

@pvelesko I just tried to cast a spell on the HIP makefile based on our other hacks. It works for me on Fedora 36 with both OpenMPI and MPICH. Please check it out and let us know.

[pvelesko]

@akohlmey LGTM