Allow using the SYCL execution space on AMD GPUs #6321
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Intended for testing purposes only and hence requires the UNSUPPORTED_ARCH flag to be set (prefer the HIP execution space for these GPUs).
|
Can one of the admins verify this patch? |
crtrott
reviewed
Jul 31, 2023
| @@ -573,7 +573,7 @@ FUNCTION(CHECK_CUDA_ARCH ARCH FLAG) | |||
| ENDIF() | |||
| SET(CUDA_ARCH_ALREADY_SPECIFIED ${ARCH} PARENT_SCOPE) | |||
| IF (NOT KOKKOS_ENABLE_CUDA AND NOT KOKKOS_ENABLE_OPENMPTARGET AND NOT KOKKOS_ENABLE_SYCL AND NOT KOKKOS_ENABLE_OPENACC) | |||
| MESSAGE(WARNING "Given CUDA arch ${ARCH}, but Kokkos_ENABLE_CUDA, Kokkos_ENABLE_OPENACC, and Kokkos_ENABLE_OPENMPTARGET are OFF. Option will be ignored.") | |||
| MESSAGE(WARNING "Given CUDA arch ${ARCH}, but Kokkos_ENABLE_CUDA, Kokkos_ENABLE_SYCL, Kokkos_ENABLE_OPENACC, and Kokkos_ENABLE_OPENMPTARGET are OFF. Option will be ignored.") | |||
There was a problem hiding this comment.
I think we left this off here intentionally since its unsupported.
There was a problem hiding this comment.
On the other hand we print them above, so I guess could go either way.
There was a problem hiding this comment.
I think it makes sense to print SYCL in the warning as well.
masterleinad
approved these changes
Jul 31, 2023
crtrott
approved these changes
Jul 31, 2023
|
OK to test. |
|
Thanks for the feedback! One more thing: I noticed that one of the tests failed due to some trailing whitespaces I missed in cmake/kokkos_arch.cmake. I have fixed that in the last commit just now! |
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request adds the changes necessary to run the Kokkos SYCL execution space on AMD GPUs (if
Kokkos_ENABLE_UNSUPPORTED_ARCH=ON).Of course, I realize that this is not the intended execution space for AMD GPUs (hence I made sure it cannot be used without
Kokkos_ENABLE_UNSUPPORTED_ARCH). However, I needed these changes as I adapted the toolchain of our own simulation (which uses Kokkos) for SYCL and dpcpp but lacked access to an Intel GPU as yet - hence, I wanted to test it on an AMD MI100.The changes required to do this with Kokkos are quite minimal; one simply has to make sure the correct AMD architecture flags are passed to dpcpp (as outlined here). Note that the logic to similarly run the SYCL execution space on NVIDIA devices already existed in
cmake/kokkos_arch.cmake, I merely adapted it to also work for AMD GPUs.I tested it (using our simulation Octo-Tiger) with dpcpp/2023-03 on a MI100 and everything seems to work fine! I figured it might be useful to other people as well, even if just for testing/benchmark purposes, hence this PR!