Parallel Particle Init

[ax3l]

We need to improve the particle initialization: to allow having a first few boxes initially, initialize in parallel & redistribute.

impactx/src/particles/ImpactXParticleContainer.cpp

Line 103 in 4b7fa17

// Redistribute(); // TODO

Currently, we will be limited to initializing as many particles as a single GPU can handle (even if we run on multiple GPUs).

[ax3l]

initially ping-ing @atmyers and @RemiLehe to discuss this in the coming meeting

[atmyers]

Getting back to this. Currently, we can't use the ParticleReduce functions in AMReX prior to calling Redistribute. The reason is that we always add the particles to grid 0, tile 0 of level 0 to the particle container, whether or not that's a valid destination for a given process. However, it wouldn't be hard to modify the reduction functions AMReX to simply loop over all tiles, whether they are "valid" or not. I think this is maybe the cleanest way to do this, since then we can just call AddNParticles, then the existing "compute min and max particle positions" function in ImpactX, then call Redistribute. What do you think @ax3l ?

[ax3l]

Discussed on Slack and implemented in AMReX via AMReX-Codes/amrex#2695 🎉