Add TPL oneMKL GEMV support #1912
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
jczhang07
commented
Jul 25, 2023
lucbv
requested changes
Jul 25, 2023
There was a problem hiding this comment.
@jczhang07 thanks for helping with this, I have some comments regarding the execution space that needs to be addressed, the reinterpret_cast is fine as is, just wanted to offer a different option without use of the macro
| sycl::event event; \ | ||
| \ | ||
| Kokkos::Profiling::pushRegion(label); \ | ||
| const STDSCALAR* a = reinterpret_cast<const STDSCALAR*>(A.data()); \ |
There was a problem hiding this comment.
This is okay, but I would have looked into a compile time check like:
if constexpr (Kokkos::ArithTraits<SCALAR>::is_complex) {
const std::complex<Kokkos::ArithTraits<SCALAR>::mag_type>* a = reinterpret_cast<...>(A.data());
} else {
// no reinterpret cast, just use the pointers directly
}
There was a problem hiding this comment.
Tried but found this problem: a is undefined in "CODE USING a"
if constexpr (Kokkos::ArithTraits<SCALAR>::is_complex) {
const std::complex<Kokkos::ArithTraits<SCALAR>::mag_type>* a = reinterpret_cast<...>(A.data());
} else {
const SCALAR *a = A.data();
}
CODE USING a;
I have to duplicate "CODE USING a" to the if-else clauses to solve the problem. Thus I had another approach using kokkos_to_std_type_map
jczhang07
force-pushed
the
jczhang/2023-07-21/feature-tpl-mkl-gemv
branch
from
6c11c96
to
9fd938d
Compare
jczhang07
commented
Jul 25, 2023
| exec.sycl_queue(), trans, M, N, alpha, a, LDA, x, 1, beta, y, 1); \ | ||
| } else { \ | ||
| event = oneapi::mkl::blas::column_major::gemv( \ | ||
| exec.sycl_queue(), trans, M, N, alpha, a, LDA, x, 1, beta, y, 1); \ |
There was a problem hiding this comment.
@lucbv I am unsure of this one. The oneMKL gemv returned event, but I don't wait on that because I don't want to synchronize the GPU. But I saw you did that in SpMV. In other words, the event variable is unused here.
There was a problem hiding this comment.
I believe in the case of SpMV I need to wait because the handle would otherwise be destroyed before the release function is called on it.
The SpMV TPL path will be optimized a bit for the handle destruction to be done in a non-blocking way but I have not worked on that part yet...
There was a problem hiding this comment.
Pipeline was clean, but anyway I removed the use of event.
There was a problem hiding this comment.
Thanks : )
Let us retest and this should be good
lucbv
approved these changes
Jul 25, 2023
lucbv
added
enhancement
feature request
AT: PRE-TEST INSPECTED
kokkos-devops-admin
removed
the
AT: PRE-TEST INSPECTED
|
Status Flag 'Pre-Test Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED by label AT: PRE-TEST INSPECTED! Autotester is Removing Label; this inspection will remain valid until a new commit to source branch is performed. |
|
Status Flag 'Pull Request AutoTester' - Failure: Timed out waiting for job KokkosKernels_PullRequest_GNU1021 to start: Total Wait = 3603
|
jczhang07
force-pushed
the
jczhang/2023-07-21/feature-tpl-mkl-gemv
branch
from
9fd938d
to
e2162fd
Compare
|
Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
lucbv
approved these changes
Jul 27, 2023
kokkos-devops-admin
removed
the
AT: PRE-TEST INSPECTED
|
Status Flag 'Pre-Test Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED by label AT: PRE-TEST INSPECTED! Autotester is Removing Label; this inspection will remain valid until a new commit to source branch is performed. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_CUDA11_CUDA11_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC930_Light_Tpls_GCC930_Tpls_CLANG13CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GNU1021
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GNU1021_Light_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GNU1021
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL19_solo
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_CLANG1001_solo
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_A64FX_Tpls_ARMPL2110
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_A64FX_GCC1020
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_VEGA908_ROCM520
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_VEGA908_Tpls_ROCM520
Jenkins Parameters
Using Repos:
Pull Request Author: jczhang07 |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_CUDA11_CUDA11_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC930_Light_Tpls_GCC930_Tpls_CLANG13CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GNU1021
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GNU1021_Light_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GNU1021
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL19_solo
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_CLANG1001_solo
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_A64FX_Tpls_ARMPL2110
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_A64FX_GCC1020
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_VEGA908_ROCM520
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_VEGA908_Tpls_ROCM520
Jenkins Parameters
Console Output (last 100 lines) : KokkosKernels_PullRequest_CUDA11_CUDA11_LayoutRight # 765 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_GCC930_Light_Tpls_GCC930_Tpls_CLANG13CUDA10 # 356 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_GNU1021 # 29 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_GNU1021_Light_LayoutRight # 28 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_Tpls_GNU1021 # 28 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_Tpls_INTEL19_solo # 34 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_CLANG1001_solo # 27 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_A64FX_Tpls_ARMPL2110 # 539 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_A64FX_GCC1020 # 534 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_VEGA908_ROCM520 # 533 (click to expand)Console Output (last 100 lines) : KokkosKernels_PullRequest_VEGA908_Tpls_ROCM520 # 55 (click to expand) |
|
@jczhang07 the oneapi header needs to be guarded with then you can remove lines 798 and 890 |
jczhang07
force-pushed
the
jczhang/2023-07-21/feature-tpl-mkl-gemv
branch
from
e2162fd
to
171ef7d
Compare
lucbv
approved these changes
Jul 27, 2023
kokkos-devops-admin
removed
the
AT: PRE-TEST INSPECTED
|
Status Flag 'Pre-Test Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED by label AT: PRE-TEST INSPECTED! Autotester is Removing Label; this inspection will remain valid until a new commit to source branch is performed. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_CUDA11_CUDA11_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC930_Light_Tpls_GCC930_Tpls_CLANG13CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GNU1021
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GNU1021_Light_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GNU1021
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL19_solo
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_CLANG1001_solo
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_A64FX_Tpls_ARMPL2110
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_A64FX_GCC1020
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_VEGA908_ROCM520
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_VEGA908_Tpls_ROCM520
Jenkins Parameters
Using Repos:
Pull Request Author: jczhang07 |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED Pull Request Auto Testing has PASSED (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_CUDA11_CUDA11_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC930_Light_Tpls_GCC930_Tpls_CLANG13CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GNU1021
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GNU1021_Light_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GNU1021
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL19_solo
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_CLANG1001_solo
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_A64FX_Tpls_ARMPL2110
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_A64FX_GCC1020
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_VEGA908_ROCM520
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_VEGA908_Tpls_ROCM520
Jenkins Parameters
|
|
Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ lucbv ]! |
|
Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - This Repo does not support Automerge |
ndellingwood
added a commit
to ndellingwood/kokkos-kernels
that referenced
this pull request
Nov 2, 2023
…NENTS Workaround for kokkos#2023 Cross-reference kokkos#1912
ndellingwood
added a commit
to ndellingwood/kokkos-kernels
that referenced
this pull request
Nov 2, 2023
…NENTS Workaround for kokkos#2023 Cross-reference kokkos#1912
ndellingwood
added a commit
to ndellingwood/kokkos-kernels
that referenced
this pull request
Nov 2, 2023
…NENTS Workaround for kokkos#2023 Cross-reference kokkos#1912
ndellingwood
added a commit
to ndellingwood/kokkos-kernels
that referenced
this pull request
Nov 2, 2023
…NENTS Workaround for kokkos#2023 Cross-reference kokkos#1912
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.