Rollup and new ViewChange documentation

Added some documentation to how the new rollup works.

byzcoin/README.md

@@ -78,33 +78,28 @@ Items 5 and 6 are the 'real' ByzCoin improvements as described in the
 
 ## Transaction collection and View Change
 
-Transactions can be submitted by end-users to any conode in the roster for
-the Skipchain that is holding the ByzCoin.
-
-At the beginning of each block creation, the leader launches a protocol to
-contact all the followers in parallel and to request the outstanding
-transactions they have. Once a follower answers this request, they are
-counting on the leader to faithfully attempt to include their transaction.
-There is no retry mechanism.
-
-With the collected transactions now in the leader, it runs them in order
-to find out how many it can fit into 1/2 of a block interval. It then sends
-the proposed block to the followers for them to validate. If there are transactions
-remaining to be run, they will be prepended to the next collected set of
-transactions when the next block interval expires.
-
-A "view change" (change of leader) is needed when the leader stops performing
-its duties correctly. Followers notice the need for a new leader if the leader
-stops sending heartbeat messages within some time window or detect a malicious
-behaviour (not implemented yet).
-
-The design is similar to the view-change protocol in PBFT (OSDI99). We keep the
-view-change message that followers send when they detect an anomaly. But we
-replace the new-view message with the ftcosi protocol and block creation. The
-result of ftcosi is an aggregate signature of all the nodes that agree to
-perform the view-change. The signature is included in the block which nodes
-accept if the aggregate signature is correct. This technique enables nodes to
-synchronise and replay blocks to compute the most up-to-date leader.
+Since `VersionRollup`, the transaction collection and view change request
+ have been changed. The leader does not request new transactions anymore,
+ rather the nodes send new transactions to the leader. The leader puts the
+ transactions in a queue and creates new blocks with as many transactions as
+ are found in the queue, respecting the maximum size of the block. This makes
+ the system more responsive if there are few transactions submitted to the
+ chain.
+
+If a node cannot send a transaction to the leader, it asks all other nodes to
+ send the transaction to the leader themselves. Every node that couldn't send
+ the transaction to the leader will start a view change request. This will
+ only detect stopped leaders, but not leaders who censor certain
+ transactions.
+
+The design of the view-change is similar to the view-change protocol in PBFT
+ (OSDI99). We keep the view-change message that followers send when they
+ detect an anomaly. But we replace the new-view message with the ftcosi
+ protocol and block creation. The result of ftcosi is an aggregate signature
+ of all the nodes that agree to perform the view-change. The signature is
+ included in the block which nodes accept if the aggregate signature is
+ correct. This technique enables nodes to synchronise and replay blocks to
+ compute the most up-to-date leader.
 
 # Structure Definitions
 

byzcoin/proto.go

@@ -71,7 +71,13 @@ type CreateGenesisBlock struct {
 	Roster onet.Roster
 	// GenesisDarc defines who is allowed to write to this skipchain.
 	GenesisDarc darc.Darc
-	// BlockInterval in int64.
+	// BlockInterval in int64 as nanoseconds since the Unix Epoch.
+	// Before VersionRollup,
+	// this was the time the leader waited between a signed block and asking
+	// for new transactions.
+	// With VersionRollup,
+	// the BlockInterval is only used to calculate the maximum protocol
+	// timeouts and the time-window of acceptance of a new block.
 	BlockInterval time.Duration
 	// Maximum block size. Zero (or not present in protobuf) means use the default, 4 megs.
 	// optional

byzcoin/service.go

@@ -369,8 +369,14 @@ func (s *Service) prepareTxResponse(req *AddTxRequest, tx *TxResult) (*AddTxResp
 
 // AddTransaction requests to apply a new transaction to the ledger. Note
 // that unlike other service APIs, it is *not* enough to only check for the
-// error value to find out if an error has occured. The caller must also check
+// error value to find out if an error has occurred. The caller must also check
 // AddTxResponse.Error even if the error return value is nil.
+// Since VersionRollup, the leader failure detection has been moved in here.
+// It is still only a stop-detection, and does not handle a censoring leader.
+// If a ClientTransaction cannot be sent to the leader,
+// it is sent to all other nodes.
+// Every node that cannot send it to the leader will request a viewChange.
+// If enough nodes fail to send it to the leader, a new leader will be elected.
 func (s *Service) AddTransaction(req *AddTxRequest) (*AddTxResponse, error) {
 	s.closedMutex.Lock()
 	if s.closed {
@@ -2168,8 +2174,11 @@ func (s *Service) verifySkipBlock(newID []byte, newSB *skipchain.SkipBlock) bool
 		}
 	}
 
+	// The window here is calculated so that it can include some clock skew,
+	// the time for the signature, and processing time of the transactions.
 	window := 4 * config.BlockInterval
 	if window < minTimestampWindow {
+		// Set a minimum window when BlockInterval is very small during testing.
 		window = minTimestampWindow
 	}
 	t1 := time.Now().Add(window)

byzcoin/tx_pipeline.go

@@ -16,6 +16,9 @@ var maxTxHashes = 1000
 
 // txPipeline gathers new ClientTransactions and VersionUpdate requests,
 // and queues them up to be proposed as new blocks.
+// With VersionRollup and newer,
+// the nodes send the ClientTransactions directly to the leader,
+// which queues them up, and proposes them to the nodes for signing.
 type txPipeline struct {
 	ctxChan     chan ClientTransaction
 	needUpgrade chan Version