Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenMPTarget: fixes and workarounds to work with "Release" build type. #3748

Merged
merged 6 commits into from
Feb 3, 2021

Conversation

rgayatri23
Copy link
Contributor

@rgayatri23 rgayatri23 commented Jan 25, 2021

Fixes

  1. Changed the testing from "Debug" build type to "RelWithDebInfo" build type
  2. team_size for scratch tests in incremental tests are set to multiples of 32 for the OPENMPTARGET backend to avoid hanging in the Release builds.
  3. omp_target_memcpy calls are protected to be called only when the copy-size > 0 .
  4. Added a nowait clause to the TeamVectorRange implementation. Avoids hanging in Release builds and is also accurate for the corresponding Kokkos constructs.
  5. Following unit tests fail in the Release build *reducers_size_t:*reducers_double:*reducers_struct . WIP to fix them.

…d of Debug mode. Fixing bugs and implementing workarounds to work in the Release mode.
@@ -217,41 +217,47 @@ template <class ExecutionSpace>
struct DeepCopy<Kokkos::Experimental::OpenMPTargetSpace,
Kokkos::Experimental::OpenMPTargetSpace, ExecutionSpace> {
DeepCopy(void* dst, const void* src, size_t n) {
omp_target_memcpy(dst, const_cast<void*>(src), n, 0, 0,
omp_get_default_device(), omp_get_default_device());
if (n)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Readability

Suggested change
if (n)
if (n > 0)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like you to add a comment in the code where you elaborate on why that call to omp_target_memcpy is protected. The specification does not explicitly calls out whether it is legal to pass null pointers with a zero length (see https://www.openmp.org/spec-html/5.1/openmpsu172.html). Did you check whether this was the issue?

Also, any good reason to ignore the return value here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the specification does not say anything specific about passing zero length to omp_target_memcpy. I guess the Release build just protects against unnecessary copies.
I added a return value to the memcpy calls and am checking for its success.

core/src/OpenMPTarget/Kokkos_OpenMPTarget_Parallel.hpp Outdated Show resolved Hide resolved
…o builds of the OPENMPTARGET backend. Added a return value for the omp_target_memcpy.
@masterleinad
Copy link
Contributor

The failing SYCL build is fixed by #3741.

@dalg24 dalg24 requested a review from crtrott January 26, 2021 19:52
Comment on lines +123 to +127
#ifdef KOKKOS_ENABLE_OPENMPTARGET
test.run(1, 32, 9);
test.run(2, 64, 22);
test.run(14, 128, 321);
#else
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for changing this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the Release and RelWithDebInfo builds, the scratch implementation fails unless the team_size is a multiple of 32.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind adding a FIXME with this comment (also in the other place)?

Comment on lines 110 to +113
#ifdef KOKKOS_ENABLE_OPENMPTARGET
test.run(14, 277, 1);
test.run(1, 32, 4);
test.run(4, 64, 10);
test.run(14, 128, 20);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, the team_size needs to be a multiple of 32 for Release builds.

@rgayatri23 rgayatri23 added Compiler Issue An issue that Kokkos cannot / should not fix; Kokkos must communicate to relevant vendor and removed [WIP] labels Jan 26, 2021
…elated to Release and RelWithDebInfo builds.
Copy link
Contributor

@masterleinad masterleinad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@masterleinad
Copy link
Contributor

Retest this please.

// FIXME_OPENMPTARGET - The size of data in array_reduce has to be a power of
// 2 for OPENMPTARGET backend in Release and RelWithDebInfo builds.
#ifdef KOKKOS_ENABLE_OPENMPTARGET
TestReducers<array_reduce<float, 4>, TEST_EXECSPACE>::test_sum(1031);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this guy is now twice here isn't it?

crtrott
crtrott previously requested changes Feb 2, 2021
Copy link
Member

@crtrott crtrott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one of the test lines is duplicated see my comment.

Otherwise fine.

@rgayatri23
Copy link
Contributor Author

one of the test lines is duplicated see my comment.

Otherwise fine.

Fixed it.

@dalg24 dalg24 dismissed crtrott’s stale review February 3, 2021 15:15

Changes requested have been applied

@dalg24 dalg24 merged commit 02ec7b8 into kokkos:develop Feb 3, 2021
@rgayatri23 rgayatri23 deleted the OpenMPTarget_fixes branch March 24, 2023 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Compiler Issue An issue that Kokkos cannot / should not fix; Kokkos must communicate to relevant vendor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants