Running Spatter

Last updated: March 30th, 2019

While we eventually plan to script these selections, it is useful to understand how Spatter allocates buffer space to know the max size you can use on a specific system.

Let's say we want to do a gather operation with 500 MB:

Now, let's test the case of transferring 500 GB of data: (8388608000 * 8 / (10^9)) = 67GB data moved. Total memory required would be 67 GB for the Index buffer, 67*2GB for the Source buffer, and 67 GB for the Target buffer, which is about 268 GB.

Note: Wrapping is not turned on by default but can be selected for larger scatter/gather operations. When wrapping, it uses different different elements when going over the source buffer again, meaning the second pass over the indices may have worse alignment than the first.

./spatter -s [sparsity] -l [num_doubles_moved] -w [wrap must be <= sparsity]
#To test with sparsity 1/2 on Power9 using OMP
`./spatter -s 2 -l 8388608000 -w 2`
Warning: No backend specified, guessing OpenMP
Warning: Kernel unspecified, guess GATHER
Warning: Kernel file unspecified, guessing kernels/kernels_vector.cl
backend kernel op time source_size target_size idx_size bytes_moved usable_bandwidth actual_bandwidth omp_threads vector_len block_dim
OPENMP GATHER COPY 23.080705 134217728000 67108864000 67108864000 134217728000 5815.148492 8722.722738 128 1 1 0
OPENMP GATHER COPY 5.232357 134217728000 67108864000 67108864000 134217728000 25651.485157 38477.227736 128 1 1 0