Repository navigation
Programming Model
| Term | Meaning |
|---|---|
| Target | A machine (a node in pushseg's configuration) |
| Application | A process on a target |
| Worker | A thread of an application |
| Rank | A worker's global number, 0 … N−1
|
A layout lists the applications and their thread counts as target:application:threads:
0:0:4,1:0:4 target 0, application 0 → ranks 0-3
target 1, application 0 → ranks 4-7
The algorithm only sees "I am rank 5 of 8". Whether rank 6 is a thread of the same process, another process on the same machine, or a process on another machine is decided by the layout and the configuration, not by the code.
Every application creates one data segment, with an area per local rank. Buffers are allocated in the same order everywhere, so a buffer has the same offset in every rank's area. A rank can therefore name any place in any other rank's memory without asking first.
rank 0 area rank 1 area rank 2 area ...
offset 0 [ field ] [ field ] [ field ]
offset F [ flags ] [ flags ] [ flags ]
offset G [ gather buf ] [ gather buf ] [ gather buf ]
| Operation | Meaning |
|---|---|
put(rank, buffer, offset, data, bytes) |
write into another rank's buffer |
signal(rank, flag, index, value) |
set a 64-bit flag in another rank's memory, ordered after the preceding puts |
wait(flag, index, value) |
wait until one's own flag has reached value
|
local<T>(buffer) |
a pointer into one's own area: reads are local |
The pattern is always the same: put the data, signal, and the receiver waits for the signal and then reads its own memory. Flags carry increasing values (a step or snapshot number), so they never need resetting.
Within one application a put is a memory copy. Between applications it is a pushseg write, over TCP or shared memory.
A group is a set of ranks; its first member is the root. Collective offers:
| Collective | Effect |
|---|---|
barrier |
nobody leaves until everybody has arrived |
broadcast |
the root's data to every member |
scatter / gather
|
one part per member, from or to the root |
allreduce |
combine one value per member; everyone gets the result |
allreduce combines values at the root in member order, so a floating-point sum is the same on every run and every deployment.
Groups can overlap and run at the same time: a simulation group and an analysis group, pairs of workers, everyone.
Real parallel programs are rarely parallel from start to end. A typical program is a sequence of phases separated by barriers: distribute, compute, combine, compute again, collect. Groups are fixed for the whole run; there is no dynamic load balancing. The Demonstrator is built this way.
gridlane
Repositories