Reductions into device side view

[crtrott]

This doesn't work right now, but we should make this work. Would also allow asynchronous reductions.

Test case:

#include<Kokkos_Core.hpp>

int main(int argc, char* argv[]) {
  Kokkos::initialize(argc,argv);
  {
     int N = (argc>1) ? atoi(argv[1]) : 10000;
     int M = (argc>2) ? atoi(argv[2]) : 10000;
     int R = (argc>3) ? atoi(argv[3]) : 10;

     Kokkos::View<int64_t> result("Result");
     int64_t reducer_result, view_result, scalar_result;

     Kokkos::Timer timer;

     // Test Reducer (this segfaults for CUDA)
     Kokkos::parallel_reduce(N, KOKKOS_LAMBDA (const int& i, int64_t& lsum) {
       lsum += 1;
     },Kokkos::Sum<int64_t,Kokkos::DefaultExecutionSpace>(result));
     double time = timer.seconds();
     // Check whether it was asyncronous
     timer.reset();
     Kokkos::fence();
     double time_fence = timer.seconds();
     Kokkos::deep_copy(reducer_result,result);
     timer.reset();

     // Test View (this segfaults for CUDA)
     Kokkos::parallel_reduce(N, KOKKOS_LAMBDA (const int& i, int64_t& lsum) {
       lsum += 1;
     },result);
     double time = timer.seconds();
     // Check whether it was asyncronous
     timer.reset();
     Kokkos::fence();
     double time_fence = timer.seconds();
     Kokkos::deep_copy(view_result,result);
     timer.reset();

     // Test Scalar
     Kokkos::parallel_reduce(N, KOKKOS_LAMBDA (const int& i, int64_t& lsum) {
       lsum += 1;
     },scalar_result);
     double time2 = timer.seconds();

     // Check whether it was asyncronous
     timer.reset();
     Kokkos::fence();
     double time_fence2 = timer.seconds();


     printf("%e %e %e %e || %li %li\n",time,time_fence,time2,time_fence2,d_result,s_result);
  }
  Kokkos::finalize();
}

[crtrott]

Both the View based and the Reducer based reductions segfault.

[dhollman]

This feels like it should be marked "defect" and not "enhancement"; I'm not sure I understand why this doesn't/isn't supposed to work now.

Also, the fence() feels wasteful here — we're fencing on all of memory to get a single scalar? Is this something we could/should do better with some sort of thin wrapper to streams?

For instance, something like

auto stream_reducer = Kokkos::Sum<int64_t,Kokkos::DefaultStreamExecutionSpace>();
Kokkos::parallel_reduce(N, KOKKOS_LAMBDA (const int& i, int64_t& lsum) {
  lsum += 1;
},stream_reducer);
// wraps cudaStreamAddCallback:
stream_reducer.when_ready([&](int64_t& result) { reducer_result = result; });
// wraps cudaStreamSynchronize:
Kokkos::fence(stream);

[crtrott]

No the fence is not meant for memory. The fence is meant for timing purposes to figure out whether the call was asynchronous.

[crtrott]

I marked it enhancement because we

• never explicitly promised this works
• it never worked
• now we will make this work

[crtrott]

Pull request #1825