rpc_doc.go

// Package rpc implements zrepl daemon-to-daemon RPC protocol
// on top of a transport provided by package transport.
// The zrepl documentation refers to the client as the
// `active side` and to the server as the `passive side`.
//
// # Design Considerations
//
// zrepl has non-standard requirements to remote procedure calls (RPC):
// whereas the coordination of replication (the planning phase) mostly
// consists of regular unary calls, the actual data transfer requires
// a high-throughput, low-overhead solution.
//
// Specifically, the requirements for data transfer is to perform
// a copy of an io.Reader over the wire, such that an io.EOF of the original
// reader corresponds to an io.EOF on the receiving side.
// If any other error occurs while reading from the original io.Reader
// on the sender, the receiver should receive the contents of that error
// in some form (usually as a trailer message)
// A common implementation technique for above data transfer is chunking,
// for example in HTTP:
// https://tools.ietf.org/html/rfc2616#section-3.6.1
//
// With regards to regular RPCs, gRPC is a popular implementation based
// on protocol buffers and thus code generation.
// gRPC also supports bi-directional streaming RPC, and it is possible
// to implement chunked transfer through the use of streams.
//
// For zrepl however, both HTTP and manually implemented chunked transfer
// using gRPC were found to have significant CPU overhead at transfer
// speeds to be expected even with hobbyist users.
//
// However, it is nice to piggyback on the code generation provided
// by protobuf / gRPC, in particular since the majority of call types
// are regular unary RPCs for which the higher overhead of gRPC is acceptable.
//
// Hence, this package attempts to combine the best of both worlds:
//
// # GRPC for Coordination and Dataconn for Bulk Data Transfer
//
// This package's Client uses its transport.Connecter to maintain
// separate control and data connections to the Server.
// The control connection is used by an instance of pdu.ReplicationClient
// whose 'regular' unary RPC calls are re-exported.
// The data connection is used by an instance of dataconn.Client and
// is used for bulk data transfer, namely `Send` and `Receive`.
//
// The following ASCII diagram gives an overview of how the individual
// building blocks are glued together:
//
//	                 +------------+
//	                 | rpc.Client |
//	                 +------------+
//	                   |         |
//	          +--------+         +------------+
//	          |                               |
//	+---------v-----------+          +--------v------+
//	|pdu.ReplicationClient|          |dataconn.Client|
//	+---------------------+          +--------v------+
//	          | label:            label:      |
//	          | zrepl_control      zrepl_data |
//	          +--------+         +------------+
//	                   |         |
//	                +--v---------v---+
//	                |  transportmux  |
//	                +-------+--------+
//	                        | uses
//	                +-------v--------+
//	                |versionhandshake|
//	                +-------+--------+
//	                        | uses
//	                 +------v------+
//	                 |  transport  |
//	                 +------+------+
//	                        |
//	                     NETWORK
//	                        |
//	                 +------+------+
//	                 |  transport  |
//	                 +------^------+
//	                        | uses
//	                +-------+--------+
//	                |versionhandshake|
//	                +-------^--------+
//	                        | uses
//	                +-------+--------+
//	                |  transportmux  |
//	                +--^--------^----+
//	                   |        |
//	          +--------+        --------------+       ---
//	          |                               |        |
//	          | label:            label:      |        |
//	          | zrepl_control      zrepl_data |        |
//	    +-----+----+              +-----------+---+    |
//	    |netadaptor|              |dataconn.Server|    | rpc.Server
//	    |     +    |              +------+--------+    |
//	    |grpcclient|                     |             |
//	    |identity  |                     |             |
//	    +-----+----+                     |             |
//	          |                          |             |
//	+---------v-----------+              |             |
//	|pdu.ReplicationServer|              |             |
//	+---------+-----------+              |             |
//	          |                          |            ---
//	          +----------+  +------------+
//	                     |  |
//	                 +---v--v-----+
//	                 |  Handler   |
//	                 +------------+
//	        (usually endpoint.{Sender,Receiver})
package rpc

// edit trick for the ASCII art above:
// - remove the comments //
// - vim: set virtualedit+=all
// - vim: set ft=text