Skip to content

Portability

Oguzhan KAYNAK edited this page Nov 14, 2026 · 1 revision

Portability

A deployed system may have N targets talking over TCP. During development you may have one machine. gridlane lets you run the same binaries in both settings; only the configuration file changes.

Demonstrated

The same job, 8 workers, run two ways:

Deployment A Deployment B
Layout 2 targets × 1 application × 4 workers 1 target × 2 applications × 4 workers
--layout 0:0:4,1:0:4 0:0:4,0:1:4
Between applications TCP POSIX shared memory
Result agrees with the Python reference agrees with the Python reference

The final fields of A and B are equal byte for byte. scripts/portability_demo.sh runs both and compares them, and CTest runs it on every build.

flowchart LR
  subgraph A["A: two targets"]
    direction TB
    a0["target 0<br/>app 0: ranks 0-3"]
    a1["target 1<br/>app 0: ranks 4-7"]
    a0 <-->|TCP| a1
  end
  subgraph B["B: one target"]
    direction TB
    b0["app 0: ranks 0-3"]
    b1["app 1: ranks 4-7"]
    b0 <-->|POSIX shm| b1
  end
Loading

What changes: the configuration

pushseg reads one configuration file shared by all processes. Deployment A, over TCP:

[apps]
coordinator = 1
worker = 2

[node.0]
tcp = 10.0.0.10:10000
[node.1]
tcp = 10.0.0.11:10000

[links]
default = tcp

Deployment B, on one machine over shared memory:

[apps]
coordinator = 1
worker = 2

[node.0]
shm =

[links]
default = shm

[backend.shm]
name_prefix = /gridlane

The backend is chosen per pair of nodes, so a system can mix them: shared memory inside a machine, TCP between machines, a device API where a device exists.

What does not change

  • The algorithm: it sees ranks, buffers and flags.
  • The control messages: the same generated codecs, the same pairring channel.
  • The binaries: gridlane_worker and gridlane_coordinator are not rebuilt.

Why the results are equal

Each cell of the computation goes through the same operations in the same order whatever the split, and sums are combined in member order. The transport only moves bytes, it never changes them. So any deployment of the same job gives the same bits; the distributed run is also bit-identical to a single-threaded run.

Clone this wiki locally