-
Notifications
You must be signed in to change notification settings - Fork 6
CODES
CODES ANL page: http://press3.mcs.anl.gov/codes/developer-access/
ROSS GitHub repo: https://github.com/carothersc/ROSS
Network Workloads (NW): A simulated MPI Process
Logical Process (LP): a Terminal or Router
Terminal: a simulated burst buffer, compute node, or MPI process
Router: a simulated network device that connects terminals to other Routers
Kernel Process (KP): a group of Logical Processes, the default is 16
PE: physical MPI process
Event Rate: number of forward events processed per second
Efficiency: number of forward events / number of backward (rollback) events
The current approach to partitioning in CODES is a linear breakup of LPs. LPs are grouped with some number of NWs followed by an equal number of terminals and a router. KPs are built over several of these groups. Several of these KPs are given to each PE. The partition of these KPs is a linear partition based on IDs.
We will be partitioning the graph resulting from the application mapped to the network topology. This graph is defined at run time in CODES and thus, an in-memory approach will be taken to compute a static partitioning of their graph (we assume that CODES does not support migration). The construction of the CODES partition is done in the function int codes_mapping_init(void) in codes/src/util/codes_mapping.c.
The original idea is to use an application trace to define work associated to LPs and create weights off of this. Knowing this would require loading trace files (1s-10s of GB) as well as knowing the routing protocol (adaptive routing is based on congestion). If we can read and interpret the files in parallel the trace files wouldn't be too difficult. If we are dealing with adaptive routing we may need to use dynamic load balancing which would require migrating LPs/KPs.
- Q) Does CODES support migrating LPs? A) No.
-
How are KPs defined? If we assign LPs to MPI processes (the partition), will KPs be defined within each process or is something else done that will result in problems (i.e., a KP spread across multiple processes)?
-
How can we measure the imbalance of a partition in CODES? Would the # of (forward) events processed by each MPI process make sense? If so, we would define the imbalance as the
maximum(process events)/average(process events)? Here the thinking is that we want to eliminate nodes that process a lot of events. We may also want to look at the number of rollbacks a partition results in; i.e., a bad partition has lots of rollback.