Typeless communicator

[hkaiser]

• This allows to get rid of the macros for all_reduce, all_to_all, broadcast, gather, and scatter

Relies on #4676 (only the last commit is significant)

[msimberg]

Can't comment on the implementation, but less boilerplate is definitely good. Thanks!

[hkaiser]

Can't comment on the implementation, but less boilerplate is definitely good. Thanks!

@msimberg the idea is to replace the std::vector<T> in the communicator with a util::any allowing to defer specifying the actual type of the data to collect to the point where the first participant of a particular collective operation contacts the communicator.

[biddisco]

The use of scatter_to and scatter_from / gather_here + gather_there is a truly awful API. It forces the user to place if locality== checks around calls and makes the code look terrible. Much better would be a gather/scatter call with the check for locality done inside the function itself so that user code was symmetrical wrt locality.

[hkaiser]

The use of scatter_to and scatter_from / gather_here + gather_there is a truly awful API. It forces the user to place if locality== checks around calls and makes the code look terrible. Much better would be a gather/scatter call with the check for locality done inside the function itself so that user code was symmetrical wrt locality.

@biddisco For scatter this is done on purpose. I considered a single API, however that would require all participating sites to supply a (possibly default constructed value) that would be discarded in the end as just one site is producing the data for all of them. Please share your ideas of a possible concrete API here.

For gather, please note that #4676 already adds a common API that is doing exactly what you are suggesting.

[hkaiser]

For gather, please note that #4676 already adds a common API that is doing exactly what you are suggesting.

I'm not so sure anymore if introducing a common API is really a good idea. The API introduced by #4676 looks like:

template <typename T> 
hpx::future<std::vector<std::decay_t<T>>> gather(char const* basename, 
    T&& local_result, std::size_t num_sites = std::size_t(-1), 
    std::size_t generation = std::size_t(-1), 
    std::size_t this_site = std::size_t(-1), std::size_t root_site = 0); 

which is essentially equivalent to an all_gather as all participating sites receive the overall result. This is not what we want, I think - but let's discuss this.

If no further opinions are voiced here, I'll go back to #4676 and remove the common API for gather before it's merged.

[biddisco]

OK. I understand, if the interface allow passing by value, then having a default construction of something that isn't used is pointless. I would hope that most of the time, things are being passed via pointers/references rather than copies of std::vectors and suchlike. If there is also a symmetric gather/scatter function that allows the (dare I say it) MPI like API, then that's fine.

[hkaiser]

OK. I understand, if the interface allow passing by value, then having a default construction of something that isn't used is pointless. I would hope that most of the time, things are being passed via pointers/references rather than copies of std::vectors and suchlike.

The problem would go away only if we allowed for pointers in the API, which I would like to avoid.

[hkaiser]

This is now good to go as well.