hpx::bind scatter/balanced give wrong pu masks

[biddisco]

on a dual socket node with 18 cores per socket and 2 hyperthreads, there are 72 threads

--hpx::bind=balanced --hpx:threads=4

should give a pu mask

000000000000000000000000000000000000000000000101000000000000000000000000000000000000000000000101

but it actually gives

000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001010101

and in fact

--hpx::bind=scatter --hpx:threads=4

should give the same pu mask, but with a different thread numbering for each pu (0213 or 0123 depending on your choice).

[hkaiser]

--bind=balanced distributes threads across cores in a balanced way, not across sockets. From what I can see from the bitmasks - all is correct. This is not a bug.

[biddisco]

https://stellar-group.github.io/hpx/docs/html/hpx/manual/init/commandline/details.html
Shows the distribution - but it seems I misunderstood that these were cores only and not sockets/etc.

We need a distribution mode that allows pus to be assigned across numa-domains in a balanced or scattered way. It is not clear to me that the balanced/scatter modes are of any use in their current form. Can we simply redefine them or should we create new modes that give the distribution I would like?

[sithhell]

I suggest new modes with a -numa suffix. The current modes were introduced for CPUs with more than 2 hyperthreads.

[biddisco]

This was addressed by adding a numa balanced bind mode in #2900