Think about memory coalescing

[hahnjo]

Once there is a sufficiently advanced prototype that allows realistic profiling, it might be worth thinking about the device memory layout and memory coalescing:

• https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#device-memory-accesses
• https://docs.nvidia.com/cuda/cuda-c-best-practices-guide/index.html#coalesced-access-to-global-memory

Right now, BlockData stores an array of structures. When accessing the same field of all tracks, this results in strided memory accesses and the hardware may not be able to do much about it. Instead, data accessed for all tracks simultaneously could be stored in a structure of arrays, if memory bandwidth is an issue for one of the kernels and memory coalescing is measured to improve performance.

[agheata]

The new container SparseArray is the evolved version, it will allow both stridden access (based on collected indices of tracks alive or other arbitrary criteria) and coalescing (compacting tracks into a new container). Both operations have overheads, to be seen which one is more efficient in which conditions for the simulation workflow. The alternative of coalescing only the needed data into specific SOA is an evolution to be considered.

[hahnjo]

The alternative of coalescing only the needed data into specific SOA is an evolution to be considered.

Yes, this is what the hardware can optimize for and what I was referring to.