Test Legion use case

[masterleinad]

Addressing #5144 (comment). I don't feel strongly about deleting the other test otr not.

[masterleinad]

The actual Legion code can be found at https://gitlab.com/StanfordLegion/legion/-/blob/master/runtime/realm/kokkos_interop.cc#L168-288.

[masterleinad]

I ended up reordering execution spaces so that host spaces are initialized first since I didn't find a good way to test

kokkos/core/src/Cuda/Kokkos_Cuda_Instance.cpp

Lines 360 to 367 in d19aab9

	#ifndef KOKKOS_IMPL_TURN_OFF_CUDA_HOST_INIT_CHECK
	if (!HostSpace::execution_space::impl_is_initialized()) {
	const std::string msg(
	"Cuda::initialize ERROR : HostSpace::execution_space is not "
	"initialized");
	throw_runtime_exception(msg);
	}
	#endif

(since it's in a *.cc file and I didn't want to add that define to the library target).

[masterleinad]

Only the SYCL build is timing out in Kokkos_Serial1 which looks spurious.

[Rombur]

Suggested change

	EXPECT_FALSE(Kokkos::Experimental::HIP::impl_is_initialized());
	EXPECT_FALSE(Kokkos::HIP::impl_is_initialized());

[Rombur]

Suggested change

	Kokkos::Experimental::HIP::impl_initialize(kokkos_init_settings);
	Kokkos::HIP::impl_initialize(kokkos_init_settings);

[Rombur]

Suggested change

	EXPECT_TRUE(Kokkos::Experimental::HIP::impl_is_initialized());
	EXPECT_TRUE(Kokkos::HIP::impl_is_initialized());

[Rombur]

Suggested change

	Kokkos::Experimental::HIP::impl_finalize();
	Kokkos::HIP::impl_finalize();

[Rombur]

Suggested change

	EXPECT_FALSE(Kokkos::Experimental::HIP::impl_is_initialized());
	EXPECT_FALSE(Kokkos::HIP::impl_is_initialized());

[masterleinad]

What I wanted to achieve with this pull request (mostly) was that the way we promised Legion to initialize Kokkos works, i.e.,
the sequence of

Kokkos::Impl::pre_initialize()
EXECUTION_SPACE::impl_initialize();
Kokkos::Impl::post_initialize();
EXECUTION_SPACE::impl_finalize();

is valid. On the way, I discovered that every backend implements EXECUTION_SPACE::impl_is_initialized() and so I used it.
If we think that this is not a useful test, I'm happy to bury it.

[crtrott]

Maybe we should just use the aliases i.e. call:

if(!is_same_v<Serial, DefaultHostExecutionSpace>)
  Serial::impl_initialize();
if(!is_same_v<DefaultExecutionSpace, DefaultHostExecutionSpace>)
  DefaultHostExecutionSpace::impl_initialize();
DefaultExecutionSpace::impl_initialize(...);

Same for finalize. Then add some actual test in between.

View<int*> data("d",1000);
deep_copy(d,1);
int result;
Kokkos::parallel_reduce("TestRed", d.extent(0), KOKKOS_LAMBDA(int i, int& sum) {
  sum += d(i);
},result);
EXPECT_EQ(result, d.extent(0));

Don't use the impl_is_initialized, just check that Kokkos::is_initialized is correct.

[dalg24]

Would approve with the changes suggested in masterleinad#9

[dalg24]

I just realize the compile test is not being removed any more. Is it intentional? I am fine with the removal.