Clean up benchmarks/gups, add permuted index mode #6378

cwpearson · 2023-08-23T19:40:55Z

Use the default execution space / memory space
Use c++ stdlib where possible
Enable when Kokkos_ENABLE_BENCHMARKS is set
- Convert from Makefile -> CMake

Sample executions

$ ./Kokkos_gups

-------------------------------------------------------------
Kokkos GUPS Benchmark
-------------------------------------------------------------
Kokkos::Cuda::initialize WARNING: running kernels compiled for compute capability 8.0 on device with compute capability 8.6 , this will likely reduce potential performance.
Reports fastest timing per kernel
Creating Views...
Memory Sizes:
- Elements:             33554432 (    268.4355 MB)
- Indices:                  8192 (      0.0655 MB)
 - Atomics:                   No
Benchmark kernels will be performed for 10 iterations.
-------------------------------------------------------------
Initializing Views...
Starting benchmarking...
-------------------------------------------------------------
GUP/s Random:                0.193954
-------------------------------------------------------------

$ ./Kokkos_gups --atomics

-------------------------------------------------------------
Kokkos GUPS Benchmark
-------------------------------------------------------------
Kokkos::Cuda::initialize WARNING: running kernels compiled for compute capability 8.0 on device with compute capability 8.6 , this will likely reduce potential performance.
Reports fastest timing per kernel
Creating Views...
Memory Sizes:
- Elements:             33554432 (    268.4355 MB)
- Indices:                  8192 (      0.0655 MB)
 - Atomics:                  Yes
Benchmark kernels will be performed for 10 iterations.
-------------------------------------------------------------
Initializing Views...
Starting benchmarking...
-------------------------------------------------------------
GUP/s Random:                0.188255
-------------------------------------------------------------

$ ./Kokkos_gups --pattern-permutation

-------------------------------------------------------------
Kokkos GUPS Benchmark
-------------------------------------------------------------
Kokkos::Cuda::initialize WARNING: running kernels compiled for compute capability 8.0 on device with compute capability 8.6 , this will likely reduce potential performance.
Reports fastest timing per kernel
Creating Views...
Memory Sizes:
- Elements:             33554432 (    268.4355 MB)
- Indices:                  8192 (      0.0655 MB)
 - Atomics:                   No
Benchmark kernels will be performed for 10 iterations.
-------------------------------------------------------------
Initializing Views...
Starting benchmarking...
-------------------------------------------------------------
GUP/s Random:                0.141398
-------------------------------------------------------------

$ ./Kokkos_gups --pattern-permutation --atomics

-------------------------------------------------------------
Kokkos GUPS Benchmark
-------------------------------------------------------------
Kokkos::Cuda::initialize WARNING: running kernels compiled for compute capability 8.0 on device with compute capability 8.6 , this will likely reduce potential performance.
Reports fastest timing per kernel
Creating Views...
Memory Sizes:
- Elements:             33554432 (    268.4355 MB)
- Indices:                  8192 (      0.0655 MB)
 - Atomics:                  Yes
Benchmark kernels will be performed for 10 iterations.
-------------------------------------------------------------
Initializing Views...
Starting benchmarking...
-------------------------------------------------------------
GUP/s Random:                0.139726
-------------------------------------------------------------

@simongdg

benchmarks/gups/CMakeLists.txt

crtrott · 2023-08-28T17:36:00Z

benchmarks/gups/gups.cpp

+  int64_t data          = 33554432;
+  int64_t repeats       = 10;
+  bool useAtomics       = false;
+  AccessPattern pattern = AccessPattern::random;


should we either make the default atomics == false pattern = permute, or atomics = true and pattern = random?

That would avoid race conditions in the default mode?

The RandomAccess benchmark in the HPC Challenge tolerates some amount of errors, so it might not be surprising to people if there are race conditions in our default GUPS execution. I'm happy to change it though if anyone has a strong opinion.

cmake/kokkos_tribits.cmake

dalg24 · 2023-08-30T17:20:23Z

cmake/kokkos_tribits.cmake

@@ -51,6 +51,7 @@ MACRO(KOKKOS_PROCESS_SUBPACKAGES)
  ADD_SUBDIRECTORY(simd)
  if (NOT KOKKOS_HAS_TRILINOS)
    ADD_SUBDIRECTORY(example)
+    ADD_SUBDIRECTORY(benchmarks)


Fine but just noting that we can refactor that in the future.
Instead of defining these KOKKOS_ADD_*_DIRECTORIES macros we could just guard the adding the directories here.

benchmarks/gups/gups.cpp

cwpearson · 2023-09-13T20:02:00Z

$ ./Kokkos_gups 
-------------------------------------------------------------
Kokkos GUPS Benchmark
-------------------------------------------------------------
Kokkos::Cuda::initialize WARNING: running kernels compiled for compute capability 8.0 on device with compute capability 8.6 , this will likely reduce potential performance.
Reports fastest timing per kernel
Creating Views...
Memory Sizes:
- Elements:             33554432 (    268.4355 MB)
- Indices:                  8192 (      0.0328 MB)
 - Atomics:                   No
Benchmark kernels will be performed for 10 iterations.
-------------------------------------------------------------
Initializing Data...
Starting benchmarking...
-------------------------------------------------------------
GUP/s Random:                0.482265
-------------------------------------------------------------

$ ./Kokkos_gups --atomics
-------------------------------------------------------------
Kokkos GUPS Benchmark
-------------------------------------------------------------
Kokkos::Cuda::initialize WARNING: running kernels compiled for compute capability 8.0 on device with compute capability 8.6 , this will likely reduce potential performance.
Reports fastest timing per kernel
Creating Views...
Memory Sizes:
- Elements:             33554432 (    268.4355 MB)
- Indices:                  8192 (      0.0328 MB)
 - Atomics:                  Yes
Benchmark kernels will be performed for 10 iterations.
-------------------------------------------------------------
Initializing Data...
Starting benchmarking...
-------------------------------------------------------------
GUP/s Random:                0.460090
-------------------------------------------------------------

$ ./Kokkos_gups --pattern-permutation
-------------------------------------------------------------
Kokkos GUPS Benchmark
-------------------------------------------------------------
Kokkos::Cuda::initialize WARNING: running kernels compiled for compute capability 8.0 on device with compute capability 8.6 , this will likely reduce potential performance.
Reports fastest timing per kernel
Creating Views...
Memory Sizes:
- Elements:             33554432 (    268.4355 MB)
- Indices:                  8192 (      0.0328 MB)
 - Atomics:                   No
Benchmark kernels will be performed for 10 iterations.
-------------------------------------------------------------
Initializing Data...
Starting benchmarking...
-------------------------------------------------------------
GUP/s Random:                0.237500
-------------------------------------------------------------

$ ./Kokkos_gups --pattern-permutation --atomics
-------------------------------------------------------------
Kokkos GUPS Benchmark
-------------------------------------------------------------
Kokkos::Cuda::initialize WARNING: running kernels compiled for compute capability 8.0 on device with compute capability 8.6 , this will likely reduce potential performance.
Reports fastest timing per kernel
Creating Views...
Memory Sizes:
- Elements:             33554432 (    268.4355 MB)
- Indices:                  8192 (      0.0328 MB)
 - Atomics:                  Yes
Benchmark kernels will be performed for 10 iterations.
-------------------------------------------------------------
Initializing Data...
Starting benchmarking...
-------------------------------------------------------------
GUP/s Random:                0.260215
-------------------------------------------------------------

benchmarks/gups/gups.cpp

* Use stdlib for time and random numbers * Use Kokkos default spaces rather than explicit spaces * add a permutation mode

cwpearson · 2023-09-15T20:19:49Z

I made the optional changes and squashed into 2 commits

masterleinad

Fine with me.

crtrott · 2023-09-26T13:35:45Z

Retest this please.

crtrott · 2023-09-26T15:31:18Z

Failed tests due to disk full.

cwpearson requested a review from crtrott August 23, 2023 19:40

cwpearson mentioned this pull request Aug 23, 2023

Add crtrott's launch_latency benchmark #6379

Merged

dalg24 reviewed Aug 23, 2023

View reviewed changes

benchmarks/gups/CMakeLists.txt Outdated Show resolved Hide resolved

crtrott reviewed Aug 28, 2023

View reviewed changes

cwpearson force-pushed the feature/gups-permute-mode branch from 9d58323 to 75993cf Compare August 30, 2023 15:50

dalg24 reviewed Aug 30, 2023

View reviewed changes

cwpearson force-pushed the feature/gups-permute-mode branch from 75993cf to ef6591b Compare September 13, 2023 16:32

cwpearson requested a review from dalg24 September 14, 2023 19:59

dalg24 approved these changes Sep 14, 2023

View reviewed changes

benchmarks/gups/gups.cpp Outdated Show resolved Hide resolved

benchmarks/gups/gups.cpp Outdated Show resolved Hide resolved

benchmarks/gups/gups.cpp Outdated Show resolved Hide resolved

cwpearson added 2 commits September 15, 2023 14:19

Clean up benchmarks/gups

002cce0

* Use stdlib for time and random numbers * Use Kokkos default spaces rather than explicit spaces * add a permutation mode

benchmark/gups: use CMake

e743017

cwpearson force-pushed the feature/gups-permute-mode branch from f741c50 to e743017 Compare September 15, 2023 20:19

cwpearson requested review from crtrott and dalg24 September 15, 2023 20:21

masterleinad approved these changes Sep 15, 2023

View reviewed changes

dalg24 mentioned this pull request Sep 15, 2023

Nightly failures: mathematical_functions_remainder_function exceeding tolerance in Trilinos builds #6451

Closed

crtrott approved these changes Sep 26, 2023

View reviewed changes

crtrott merged commit 9250328 into kokkos:develop Sep 26, 2023
27 of 28 checks passed

cwpearson mentioned this pull request Sep 26, 2023

CHANGELOG: 4.2.0 #6197

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clean up benchmarks/gups, add permuted index mode #6378

Clean up benchmarks/gups, add permuted index mode #6378

cwpearson commented Aug 23, 2023

crtrott Aug 28, 2023

cwpearson Aug 30, 2023

dalg24 Aug 30, 2023

cwpearson commented Sep 13, 2023

cwpearson commented Sep 15, 2023

masterleinad left a comment

crtrott commented Sep 26, 2023

crtrott commented Sep 26, 2023

Clean up benchmarks/gups, add permuted index mode #6378

Clean up benchmarks/gups, add permuted index mode #6378

Conversation

cwpearson commented Aug 23, 2023

crtrott Aug 28, 2023

Choose a reason for hiding this comment

cwpearson Aug 30, 2023

Choose a reason for hiding this comment

dalg24 Aug 30, 2023

Choose a reason for hiding this comment

cwpearson commented Sep 13, 2023

cwpearson commented Sep 15, 2023

masterleinad left a comment

Choose a reason for hiding this comment

crtrott commented Sep 26, 2023

crtrott commented Sep 26, 2023