Topic/topo subgrouping

[vspetrov]

What

Adds basic subgrouping functionality to UCC. Ranks participating in a team can now be partitioned into groups (1) belonging to same node, (2) socket, (3) nodeleaders, etc.

Why ?

These groupings are used to build hierarchical collective schedules

How ?

1. Each rank encodes local process information (currently host_hash, socket_id, pid) into ctx address header.
2. This info is exchanged at ucc_context_create (with OOB) as part of address_exchange flow.
3. CTX level topo datastructure is initialized: it stores the global view of processes data (global with respect to context)
4. When team is created the ucc_team_topo_t is initialized from global ucc_topo_t -> this gives the team level subgrouping information.
5. Subgroups (SBGPS) are created "locally" (w/o communication) and on-demand.
6. The whole topo flow is only enabled if any CL/TL reports that it needs it.

[manjugv]

UCC WG July 21st:

1. Add performance number do understand the overhead of subgroup creation.
2. Create issue for sharing topology info with other processes on the node.
3. Create issue for API change to input hwloc topology information.

[vspetrov]

32 nodes, ppn 32 (max 1024 ranks)
ucc_context_create breakdown in usec: so it is ~0.2 sec. "bound socket id detection" takes 0.013 sec in worst case (looks noisy, probably due to reading from FS). ctx_topo data structure init is nothing in terms of time (though scales linearly the absolute number is way lower). Note, sock id detection can be optimized if we pass that info from runtime via additional lib params.

nnodes	nranks	total	sock id detection	ctx topo init
2	64	198757	13375.094	6.633
4	128	204162	13349.99	10.773
8	256	208647	10379.154	20.85
16	512	214565	9993.638	48.438
32	1024	220230	2311.789	119.392

this is ucc_team creation breakdown in usec. There is still 1 allgather that takes most of the time (i also discuss it in #274 how we should eliminate it). Again the cost of team_topo_init and actual subgroups creation (pure local math operations w/o communication) is nothing.

nnodes	nranks	total	team_topo_init	sbgp_node	sbgp_socket	sbgp_node_leaders
2	64	8611.22	0.157	0.309	0.27	0.423
4	128	8977.55	0.091	0.555	0.269	0.561
8	256	10650.6	0.131	0.834	0.299	1.042
16	512	12868.4	0.147	1.653	0.301	2.317
32	1024	15373.4	0.176	4.027	0.435	3.53

Final note, keep in mind that "topo" structs and sbgps are allocated only when TL/CL requires them. For this evaluation i specifically forced those flows (although cl/basic + tl/ucp does not ask for topo).

[vspetrov]

@manjugv @bureddy @Sergei-Lebedev i added perf numbers and fixed minor linter issues. plz review.

[bureddy]

move to math.h?

[vspetrov]

done