Protocol
Copyright (c) 2014, Max Mertens. All rights reserved.
SphereSim uses nanomsg for decentralized communication
between servers and clients.
nanomsg is "a socket library that provides several common communication
patterns" and is used to send messages over TCP.
A message used in SphereSim consists of the following message parts:
- Simulation ID: 4 bytes
- Sender ID: 4 bytes
- Command ID: 2 bytes
- Data ID: 2 bytes
- Data length: 2 bytes
- Data: any number of bytes (0 to 65535) specified in 5.
A message must contain at least a simulation ID, a sender ID and a command ID. It may additionally contain any number (0 to n) of data parameters, each consisting of the data ID and length and the data itself.
The data of each message part is stored as binary data in little-endian format.
Each byte value may range from 0 to 255.
SphereSim uses the PUBSUB scalability protocol provided by nanomsg:
A publish (PUB) socket sends messages to one or more connected peers.
A subscribe (SUB) socket receives these messages from one or more connected
peers.
SphereSim servers bind to one PUB and one SUB socket.
Clients connect one PUB and one SUB socket to the server's respective
counterparts.
A typical connection between 1 server and 2 clients looks like this:
Servers use one extra PUB socket to communicate with other servers.
Be aware that each server has to reach all clients and vice-versa, and each
server has to reach all other servers.
A typical connection between 2 servers and 2 clients looks like this:
A server has to follow these rules:
- Generate a random even 32 bit number (i.e., with least significant bit cleared) used as its sender ID.
- Send continuous heartbeats with simulation ID
0and its sender ID about every second.
A client has to follow these rules:
- Generate a random odd 32 bit number (i.e., with least significant bit set) used as its sender ID and simulation ID.
- Send continuous (normal or announcement) heartbeats with its simulation and sender ID about every second.
The least significant bit must be set or cleared so that servers and clients are
distinguishable by their ID and to avoid conflicts with server heartbeats using
simulation ID 0.
Servers and clients are represented by finite-state machines (FSM).
The following FSM diagrams use these types of structural elements:
Virtual states are no real states, but rather intermediary steps in the process.
The following diagram shows the FSM of a server:
The following diagram shows the FSM of a client:
These are the steps during connection establishment following from above diagrams:
- The servers each generate a sender ID and send heartbeats (see above).
- The client connects its sockets to the respective server sockets.
- The client waits for server heartbeats.
- The client generates a sender and simulation ID (see above).
- The client starts to send announcement heartbeats (see above) and waits for server responses.
- The servers receive the client announcement heartbeat.
- If the simulation ID is already registered, the servers send negative acknowledge to the client.
- Else the servers register the simulation ID and send acknowledge to the client.
- If the client receives negative acknowledge or does not receive any response in a particular amount of time, it retries connection from step 4.
- Else the connection is established: The client switches to normal heartbeats and sends initial simulation data.
- If a server does not receive any hearbeat signal from one of its clients in a particular amount of time, it de-registers the client and deletes related data.
Feel free to adopt this transfer protocol to your software.
SphereSim and this protocol are released under the BSD 3-Clause License.
Please refer to LICENSE file for details.