Skip to content

Layered Model

Oguzhan KAYNAK edited this page Nov 14, 2026 · 1 revision

The layered model

Each layer is a separate project with its own contract, its own tests and its own extension point. No layer references another: pushseg knows nothing of pairring, pairring knows nothing of pushseg or wireforge, wireforge knows nothing of either. Only gridlane knows them all, and it joins them with adapters of a few dozen lines.

flowchart TB
  subgraph L4["gridlane"]
    A1["PushsegRegion<br/>(pushseg → pairring)"]
    A2["PairringTransport<br/>(pairring → wireforge)"]
    F["Fabric, Collective<br/>(data plane on pushseg)"]
  end
  WF["wireforge runtime"] 
  PR["pairring"]
  PS["pushseg"]
  A2 -. implements wireforge's transport .-> WF
  A2 -. uses .-> PR
  A1 -. implements pairring's RemoteRegion .-> PR
  A1 -. uses .-> PS
  F -. uses .-> PS
Loading

Because the joints are adapters, any layer can be replaced: pairring could run over an RDMA library instead of pushseg; wireforge could be replaced by another serializer; pushseg could gain a backend for a PCIe device.

Transports (bottom)

Whatever moves bytes: TCP sockets, POSIX shared memory, a vendor's PCIe or network library. gridlane's tests use only TCP and POSIX shared memory.

pushseg: segments

Write far, read local. A process owns segments and reads them through plain pointers. Other processes open one-way connections to a segment and write into it. There are no remote reads, by design: a remote read costs a round trip on every transport.

What the upper layers rely on is the memory contract, the same on every backend:

Operation Guarantee
write bulk bytes, no ordering among writes
store<T> (1, 2, 4, 8 bytes, aligned) never torn: the reader sees the old or the new value
store, notify, write_notify ordering points: everything written before is visible no later than the point

So "write the data, then store a flag" works everywhere: when the reader sees the flag, the data is there.

Why it is used here: it gives the data plane a shared-memory-like model over any transport. Details: pushseg README.

pairring: message channel

A point-to-point channel built from two rings, each in the receiver's memory, written only by the sender. It needs from the layer below only five functions (write, optional writev, store64, notify, size), which pushseg's contract satisfies directly.

  • Whole messages: send returns true or false, never a partial message; a full ring returns false.
  • Every record carries a CRC-32C; a corrupt record never reaches the application.
  • An epoch handshake: either side can restart and reconnect without reading stale data.
  • Doorbells are hints: one wakeup delivers everything that arrived.

Why it is used here: the control plane needs reliable, ordered, whole messages on top of remote-writable memory. Details: pairring protocol.

wireforge: messages

Messages are defined once in a model (model/project.yaml): fields, ranges, versions, which program sends what. wireforge generates the encoders, decoders, dispatchers and an interface control document. The generated code is committed in generated/, so a reader can see what was generated and regenerate it after changing the model.

Why it is used here: version queries, heartbeats and job parameters must be typed, versioned and range-checked; hand-written structs would not be. Details: wireforge guide.

gridlane: composition

  • Adapters (include/gridlane/glue/): a pushseg connection as a pairring RemoteRegion; a pairring channel as a wireforge transport.
  • Data plane runtime: a layout of ranks, symmetric memory, put / signal / wait, collectives. See Programming Model.
  • Demonstrator: see Demonstrator.

Two planes

flowchart LR
  subgraph control["control plane: small, typed, versioned"]
    direction LR
    m["version query · heartbeat · job · progress"] --> w1["wireforge"] --> c1["pairring"] --> s1["pushseg"]
  end
  subgraph data["data plane: raw, bulk"]
    direction LR
    d["fields · halo rows · snapshots · results"] --> s2["pushseg"]
  end
Loading

Bulk data never goes through the message layer. It is written straight into the memory where the receiver will read it, so it is never encoded, copied into a message, or decoded.

Clone this wiki locally