-
Notifications
You must be signed in to change notification settings - Fork 0
The Twine Protocol
See this blog post.
It's netstrings on steroids.
Starting from simplicity of netstrings, how can it be enhanced to give meaning to the payload. Twine Protocol is basically just a header that tells the receiver how long the data is (so it can be ready completely) and what to do with it (a destination, more on that below).
Twine is completely neutral on payload. It's a pile of bytes as far as it's concerned. It's up to the consumer of the payload to know what to do with it. It could be JSON string, or a single byte representing a boolean.
While there is a concep of client and server in Twine, it is only because one has to dial, and other has to listen for the initial connection.
Once the connection is established, either side can initiate sending messages equally.
In the current implementation (likely to change soon) the destination for a message is encoded as two identifiers: service and command.
To receive a message, a service registers itself as a handler for that service id and dispatches incoming messages based on the command id.
I won't elaborate because I plan on changing this.
A "message ID" identifies a series of up to three related communications between the sender and the receiver. These are:
- Initial send: a currently unused ID is chosen for the initial send from A to B.
- Reply OR ok/error:
- B can send a "reply" (that may contain a payload) to A using the same message ID.
- B can simply acknowledge receipt of initial send with an OK signal
- B can send an error message if it encountered an error processing the received data.
- If B sent a "reply", then A must send OK or error in response.
The message is considered "closed" once one side sends OK or error.
A new message id is put in use by the sending (A) side. This means the message id is "open" on the send side.
Upon receipt, the receiving (B) side registers this message id as "open".
When a reply is sent (either "reply", "ok", or "error") from B, the B side marks that message id as closed, but keeps it around so it can direct ok or error messages to the right place. *
Upon receipt of reply or ok or error the A side marks this message as closed. *
(*) Actually not sure this is correct. Message id should remain open on A side until OK/Error is sent after a reply. On the B side, it should be considered closed but still accept an OK/Error. Once that is received, flush it out for good.
An "open" message_id can behave like a channel by using the open message_id as a reference.
Either side can send a new message using the open message_id as a reference.
The side expecting such messages must listen for them by referencing the open message_id.
Example:
- SideA sends
message_id: 100, service: "DB", command: "Select", payload: "SELECT * FROM table" - SideA listens for messages that reference 100
- SideB receives 100, passes to DB,
- SideB DB issues query to DB, and gets rows back from DB
- SideB sends message_id: 101, command: "row", payload: "row 1 data"
- SideB sends message_id: 102, command: "row", payload: "row 2 data"
- SideB sends message_id: 103, command: "row", payload: "row 3 data"
- ...
- SideB sends message_id: 100, command: "OK"
- SideA gets "OK" on original message and can stop listenting and cleans up knowing there are no more rows coming (* need to clarify, because some row data could be in-flight? or we need to enforce message delivery hierarchy?)
Describe typical code used to send messages, etc...