Try using FindOpenMP instead of figuring out flags manually #4105

masterleinad · 2021-06-18T15:30:18Z

Trying to address #3256 by relying on CMake's FindOpenMP.

masterleinad · 2021-06-18T18:56:39Z

Retest this please.

masterleinad · 2021-06-21T13:38:58Z

Retest this please.

masterleinad · 2021-06-21T22:17:47Z

This pull currently doesn't add any linker flags but passes our regular CI. It would be good to check this on a somewhat wider ranger of configurations.

ndellingwood · 2021-07-16T18:36:24Z

I configured and built this PR with intel/17.0.4 and intel/19.0.3 each with cmake/3.16.0 and cmake/3.19.3, there were no issues in any of these cases

masterleinad · 2021-07-16T19:00:04Z

I think the interesting cases to test with are CLANG_CRAY and CLANG_MSVC.

ndellingwood · 2021-07-16T20:29:45Z

I'm not able to test clang_msvc, here is from my attempt to test with clang_cray that failed to configure (I don't have much experience with that env, so I could be doing something wrong):

Load cray computing env and cmake/3.16.2

module load cce/11.0.2 cmake/3.16.2

clang++ --version
Cray clang version 11.0.2  (7cc96bf2bf88b71c173230d893018a3f38fbbb09)
Target: x86_64-unknown-linux-gnu
Thread model: posix

Configure
Kokkos SHA 4a29d0a

cmake -DCMAKE_CXX_COMPILER=clang++ -DKokkos_ARCH_HSW=ON -DKokkos_ENABLE_OPENMP=ON -DKokkos_ENABLE_TESTS=ON $KOKKOS_PATH

Configure output:

ndellin@voltrino:~/kokkos/testing/Voltrino-HSW-CCE> cmake -DCMAKE_CXX_COMPILER=clang++ -DKokkos_ARCH_HSW=ON -DKokkos_ENABLE_OPENMP=ON -DKokkos_ENABLE_TESTS=ON $KOKKOS_PATH 2>&1 | tee config.out
-- Setting default Kokkos CXX standard to 14
-- The CXX compiler identification is Clang 11.0.0
-- Check for working CXX compiler: /opt/cray/pe/cce/11.0.2/cce-clang/x86_64/bin/clang++
-- Check for working CXX compiler: /opt/cray/pe/cce/11.0.2/cce-clang/x86_64/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Setting build type to 'RelWithDebInfo' as none was specified.
-- Setting policy CMP0074 to use <Package>_ROOT variables
-- The project name is: Kokkos
CMake Warning at cmake/kokkos_check_env.cmake:10 (MESSAGE):
  CRAYPE_LINK_TYPE is not set.  Linking is likely to fail unless this is set
  to 'dynamic'
Call Stack (most recent call first):
  CMakeLists.txt:162 (INCLUDE)


CMake Error at /projects/Voltrino/hpcsoft/cle7.0/common/cmake/3.16.2/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
  Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES)
Call Stack (most recent call first):
  /projects/Voltrino/hpcsoft/cle7.0/common/cmake/3.16.2/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE)
  /projects/Voltrino/hpcsoft/cle7.0/common/cmake/3.16.2/share/cmake-3.16/Modules/FindOpenMP.cmake:511 (find_package_handle_standard_args)
  cmake/kokkos_enable_devices.cmake:47 (FIND_PACKAGE)
  cmake/kokkos_tribits.cmake:229 (INCLUDE)
  CMakeLists.txt:170 (KOKKOS_SETUP_BUILD_ENVIRONMENT)

Is there anything obvious that I may be doing wrong on my end?

masterleinad · 2021-07-16T21:09:21Z

I almost forgot that I can check myself for clang_cray. On spock, I get

-- Setting default Kokkos CXX standard to 14
-- The CXX compiler identification is Clang 11.0.0
-- Check for working CXX compiler: /opt/cray/pe/craype/2.7.6/bin/CC
-- Check for working CXX compiler: /opt/cray/pe/craype/2.7.6/bin/CC - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Setting build type to 'RelWithDebInfo' as none was specified.
-- Setting policy CMP0074 to use <Package>_ROOT variables
-- The project name is: Kokkos
CMake Warning at cmake/kokkos_check_env.cmake:10 (MESSAGE):
  CRAYPE_LINK_TYPE is not set.  Linking is likely to fail unless this is set
  to 'dynamic'
Call Stack (most recent call first):
  CMakeLists.txt:162 (INCLUDE)


-- Found OpenMP_CXX: -fopenmp (found version "5.0") 
-- Found OpenMP: TRUE (found version "5.0")  
-- Using -std=gnu++14 for C++14 extensions as feature
-- Built-in Execution Spaces:
--     Device Parallel: NoTypeDefined
--     Host Parallel: Kokkos::OpenMP
--       Host Serial: NONE
-- 
-- Architectures:
-- Found TPLLIBDL: /usr/lib64/libdl.so  
-- Kokkos Devices: OPENMP, Kokkos Backends: OPENMP
-- Configuring done
-- Generating done
-- Build files have been written to: /ccs/home/darndt/kokkos/build_cray

so that seems to work.
@ndellingwood Can you use OpenMP successfully on your system with develop instead of this pull request?

ndellingwood · 2021-07-16T21:27:41Z

@masterleinad I'm able to get the develop branch to configure,

ndellin@voltrino:~/kokkos/testing/Voltrino-HSW-CCE> cmake -DCMAKE_CXX_COMPILER=clang++ -DKokkos_ARCH_HSW=ON -DKokkos_ENABLE_OPENMP=ON -DKokkos_ENABLE_TESTS=ON $KOKKOS_PATH
-- Setting default Kokkos CXX standard to 14
-- The CXX compiler identification is Clang 11.0.0
-- Check for working CXX compiler: /opt/cray/pe/cce/11.0.2/cce-clang/x86_64/bin/clang++
-- Check for working CXX compiler: /opt/cray/pe/cce/11.0.2/cce-clang/x86_64/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Setting build type to 'RelWithDebInfo' as none was specified.
-- Setting policy CMP0074 to use <Package>_ROOT variables
-- The project name is: Kokkos
CMake Warning at cmake/kokkos_check_env.cmake:10 (MESSAGE):
  CRAYPE_LINK_TYPE is not set.  Linking is likely to fail unless this is set
  to 'dynamic'
Call Stack (most recent call first):
  CMakeLists.txt:162 (INCLUDE)


-- Using -std=gnu++14 for C++14 extensions as feature
-- Built-in Execution Spaces:
--     Device Parallel: NoTypeDefined
--     Host Parallel: Kokkos::OpenMP
--       Host Serial: NONE
-- 
-- Architectures:
--  HSW
-- Found TPLLIBDL: /usr/lib64/libdl.so  
-- Sources TestOpenMP.cpp
-- Kokkos Devices: OPENMP, Kokkos Backends: OPENMP
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ndellin/kokkos/testing/Voltrino-HSW-CCE

However I am running into build errors in this case:

ld.lld: error: unable to find library -lcraymp
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)

crtrott · 2021-08-03T22:54:51Z

DId you test CUDA + OpenMP?

crtrott · 2021-08-04T15:22:24Z

I did some testing with ICPX for simple OpenMP run, OpenMPTarget, and CUDA+OpenMP even with nvc++ and it all works. So this appears to be good on that front. Still we need to resolve the cray stuff.

…pull/4105, kokkos/kokkos/pull/4278, kokkos/kokkos/pull/4281

crtrott · 2021-12-16T17:29:18Z

@masterleinad is that still WIP? Or does this work now?

masterleinad · 2021-12-16T18:31:18Z

@masterleinad is that still WIP? Or does this work now?

I didn't change anything but I would think that the error in #4105 (comment) is more an issue with the machine being misconfigured. I would be interesting to hear if the problem persists.

masterleinad · 2022-07-18T18:54:03Z

I updated this pull request to just use

IF (Kokkos_ENABLE_OPENMP)
  find_package(OpenMP REQUIRED)
  KOKKOS_EXPORT_CMAKE_TPL(OpenMP REQUIRED)
  target_link_libraries(kokkoscore PUBLIC OpenMP::OpenMP_CXX)
ENDIF()

which is the same approach we use for desul. This would also force Trilinos to use the same path instead of

KOKKOS_LINK_TPL(kokkoscore PUBLIC IMPORTED_NAME OpenMP::OpenMP_CXX OPENMP)

which doesn't do anything for Trilinos.

junghans · 2022-08-08T21:16:23Z

I guess that would also help in #5184

masterleinad · 2022-08-09T15:26:13Z

One disadvantage of this approach would be that using different compilers for Kokkos and downstream applications causes issues when using OpenMP since the compiler flags for the found target might be different. On the other hand, I wouldn't expect that to be very stable anyway.

PhilMiller · 2022-08-10T17:59:38Z

One disadvantage of this approach would be that using different compilers for Kokkos and downstream applications causes issues when using OpenMP since the compiler flags for the found target might be different. On the other hand, I wouldn't expect that to be very stable anyway.

Was that ever something we really supported? Is it something we want to explicitly disclaim from 4.0 onward?

nmm0

Looks good to me

aminiussi · 2023-02-20T10:48:02Z

Hi, correct me if I'm wrong, but this fix is not in 3.7.1 right ?
Thanks

masterleinad · 2023-02-20T13:19:20Z

Hi, correct me if I'm wrong, but this fix is not in 3.7.1 right ?

Yes, this feature is only merged to the 4.0.00 release branch.

aminiussi · 2023-02-20T14:39:22Z

Thanks. Is there a roadmap page somewhere ?

masterleinad · 2023-02-20T15:58:12Z

Thanks. Is there a roadmap page somewhere ?

We normally only do release notes (#5439, https://github.com/kokkos/kokkos/blob/master/CHANGELOG.md).

masterleinad added the [WIP] label Jun 18, 2021

masterleinad mentioned this pull request Jun 23, 2021

Add XLClang as compiler #4120

Merged

masterleinad added this to In progress in Developer: Daniel Arndt Jul 7, 2021

masterleinad moved this from In progress to Awaiting Feedback in Developer: Daniel Arndt Jul 8, 2021

crtrott added this to In progress in Kokkos Release 3.5 Jul 14, 2021

ajpowelsnl removed the [WIP] label Jul 15, 2021

masterleinad moved this from In progress to Awaiting Feedback in Kokkos Release 3.5 Jul 20, 2021

ajpowelsnl added the InDevelop Enhancement, fix, etc. has been merged into the develop branch; label Jul 20, 2021

masterleinad mentioned this pull request Aug 3, 2021

AppleClang+Openmp not correctly detected #3256

Closed

crtrott removed the InDevelop Enhancement, fix, etc. has been merged into the develop branch; label Aug 3, 2021

masterleinad mentioned this pull request Aug 27, 2021

Better OpenMP detection with AppleClang #3257

Closed

Chrismarsh added a commit to Chrismarsh/conan-trilinos that referenced this pull request Sep 1, 2021

Add trilinos with Kokkos with following PRs merged in: kokkos/kokkos/…

da0d24e

…pull/4105, kokkos/kokkos/pull/4278, kokkos/kokkos/pull/4281

masterleinad mentioned this pull request Sep 24, 2021

Improve sort arborx/ArborX#145

Closed

masterleinad marked this pull request as ready for review December 14, 2021 21:05

crtrott moved this from Awaiting Feedback to In progress in Kokkos Release 3.5 Dec 16, 2021

crtrott removed this from In progress in Kokkos Release 3.5 Dec 16, 2021

crtrott added this to In progress in Kokkos Release 3.6 via automation Dec 16, 2021

masterleinad force-pushed the openmp_detection branch from 4a29d0a to 1f233f4 Compare December 16, 2021 18:41

masterleinad added 4 commits July 18, 2022 14:43

Try using FindOpenMP instead of figuring out flags manually

9bb4005

Use LIB_NAMES instead

426d982

Don't use link flags

3c64b3a

Link with OpenMP

040419a

masterleinad force-pushed the openmp_detection branch from 1f233f4 to 040419a Compare July 18, 2022 18:47

junghans approved these changes Aug 8, 2022

View reviewed changes

masterleinad added 2 commits August 9, 2022 10:04

Merge remote-tracking branch 'upstream/develop' into openmp_detection

c45e698

Don't test using a different compiler with OpenMP support

db66946

masterleinad force-pushed the openmp_detection branch from 0ff97bf to db66946 Compare August 9, 2022 15:24

Merge remote-tracking branch 'upstream/develop' into openmp_detection

8e274f6

PhilMiller modified the milestones: Tentative 3.7 Release, Tentative 4.0 Release Aug 17, 2022

nmm0 approved these changes Aug 17, 2022

View reviewed changes

crtrott merged commit 188fac1 into kokkos:develop Aug 17, 2022

PhilMiller assigned crtrott Aug 17, 2022

masterleinad mentioned this pull request Sep 7, 2022

CHANGELOG: 4.0 #5439

Closed

crtrott mentioned this pull request Oct 5, 2022

Trilinos nightly failure, OpenMP backend: simpleBuildAgainstTrilinos app builds against install fails with "Target "MyApp" links to target "OpenMP::OpenMP_CXX" but the target was not found." #5514

Closed

masterleinad mentioned this pull request Oct 6, 2022

Fix using OpenMP in code downstream from Trilinos #5526

Closed

masterleinad mentioned this pull request Oct 13, 2022

Trilinos: Pass OpenMP flags instead of linking with the OpenMP target #5532

Merged

dalg24 mentioned this pull request Oct 18, 2022

[3.7.01] CI: test flang #5558

Closed

This was referenced Dec 12, 2022

Building on a Cray system with OpenMP enabled fails #2785

Closed

Makefile.kokkos fails to detect CCE10 with OpenMP as device #3183

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Try using FindOpenMP instead of figuring out flags manually #4105

Try using FindOpenMP instead of figuring out flags manually #4105

masterleinad commented Jun 18, 2021 •

edited

masterleinad commented Jun 18, 2021

masterleinad commented Jun 21, 2021

masterleinad commented Jun 21, 2021

ndellingwood commented Jul 16, 2021

masterleinad commented Jul 16, 2021

ndellingwood commented Jul 16, 2021 •

edited

masterleinad commented Jul 16, 2021

ndellingwood commented Jul 16, 2021

crtrott commented Aug 3, 2021

crtrott commented Aug 4, 2021

crtrott commented Dec 16, 2021

masterleinad commented Dec 16, 2021

masterleinad commented Jul 18, 2022

junghans commented Aug 8, 2022

masterleinad commented Aug 9, 2022

PhilMiller commented Aug 10, 2022

nmm0 left a comment

aminiussi commented Feb 20, 2023

masterleinad commented Feb 20, 2023

aminiussi commented Feb 20, 2023

masterleinad commented Feb 20, 2023

Try using FindOpenMP instead of figuring out flags manually #4105

Try using FindOpenMP instead of figuring out flags manually #4105

Conversation

masterleinad commented Jun 18, 2021 • edited

masterleinad commented Jun 18, 2021

masterleinad commented Jun 21, 2021

masterleinad commented Jun 21, 2021

ndellingwood commented Jul 16, 2021

masterleinad commented Jul 16, 2021

ndellingwood commented Jul 16, 2021 • edited

masterleinad commented Jul 16, 2021

ndellingwood commented Jul 16, 2021

crtrott commented Aug 3, 2021

crtrott commented Aug 4, 2021

crtrott commented Dec 16, 2021

masterleinad commented Dec 16, 2021

masterleinad commented Jul 18, 2022

junghans commented Aug 8, 2022

masterleinad commented Aug 9, 2022

PhilMiller commented Aug 10, 2022

nmm0 left a comment

Choose a reason for hiding this comment

aminiussi commented Feb 20, 2023

masterleinad commented Feb 20, 2023

aminiussi commented Feb 20, 2023

masterleinad commented Feb 20, 2023

masterleinad commented Jun 18, 2021 •

edited

ndellingwood commented Jul 16, 2021 •

edited