Set Kokkos_DEVICES and Kokkos_ARCH variables in exported CMake configuration

[dalg24]

On a related note, we need to make sure downstream code that do find_package(Kokkos) will be able to deduce the list of devices that where enabled at the time Kokkos was built.
c.f. arborx/ArborX#73 (comment)

Originally posted by @dalg24 in #2175 (comment)

[dalg24]

Current version of #2104 would break Cabana and ArborX. Both projects have some logic to instantiate code or enable examples/tests that rely on knowing at configuration time what devices were enabled at the time Kokkos was built.
See:
https://github.com/ECP-copa/Cabana/blob/b27a54c8c2df2f8b46d93f15d3e6992ee05041d3/CMakeLists.txt#L47-L49
and
https://github.com/arborx/ArborX/blob/857bcb6e06a984ca2363174ed7c989cdfb90f0e8/cmake/FindKokkos.cmake#L81-L82

CC: @junghans

[dalg24]

Adding a reference to the helper function I had proposed at the user meeting (see also #2064) which let the user code assert that a backend has been enabled.

kokkos/core/cmake/kokkos-config.cmake.in

Lines 60 to 92 in 839229d

	# This function makes sure that kokkos was built with the requested backends
	# and target architectures and generates a fatal error if it was not.
	#
	# kokkos_check_requested(
	# [DEVICES <devices>...] # Set of backends (e.g. "OpenMP" and/or "Cuda")
	# [ARCH <archs>...] # Target architectures (e.g. "Power9" and/or "Volta70")
	# )
	function(kokkos_check_requested)
	cmake_parse_arguments(_KOKKOS_REQUESTED "" "" "DEVICES;ARCH" ${ARGN})
	set(_KOKKOS_REQUESTED_ARGS)
	foreach(_X DEVICES ARCH)
	if(_KOKKOS_REQUESTED_${_X})
	list(APPEND _KOKKOS_REQUESTED_ARGS ${_X})
	endif()
	endforeach()
	set(_KOKKOS_CHECK_REQUESTED_SUCCESS TRUE)
	foreach(_X ${_KOKKOS_REQUESTED_ARGS})
	foreach(_requested ${_KOKKOS_REQUESTED_${_X}})
	foreach(_provided ${KOKKOS_${_X}})
	if(_requested STREQUAL _provided)
	string(REPLACE ";" " " ${_requested} "${_KOKKOS_REQUESTED_${_X}}")
	endif()
	endforeach()
	endforeach()
	find_package_handle_standard_args("KOKKOS_${_X}" DEFAULT_MSG ${_KOKKOS_REQUESTED_${_X}})
	if(NOT KOKKOS_${_X}_FOUND)
	set(_KOKKOS_CHECK_REQUESTED_SUCCESS FALSE)
	endif()
	endforeach()
	if(NOT _KOKKOS_CHECK_REQUESTED_SUCCESS)
	message(FATAL_ERROR "Kokkos does NOT provide all backends and/or architectures requested")
	endif()
	endfunction()

[junghans]

I agree, exporting devices would be very useful.

[ipdemes]

We need this for FLeCSI too

[junghans]

@crtrott

[junghans]

Another option would be to set the these as properties in kokkos-config.cmake, e.g.:

set_target_properties(Kokkos::kokkos PROPERTIES ARCH "@KOKKOS_ARCH@")

then we could process them in Cabana and FleCSI with:

get_target_property(KOKKOS_ARCH Kokkos::kokkos ARCH)

and everything is transparently attached to the Kokkos::kokkos target.

[ipdemes]

@crtrott : any update on this?

[dalg24]

@jjwilke has added Kokkos_ARCH and Kokkos_DEVICES in the latest revision of #2104.

[ipdemes]

@dalg24 : Great! Thank you