From 8d71db2e7c3ac1898e5a0cc57762c62c6c2b32e7 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Mon, 7 Dec 2020 14:29:22 +0100 Subject: [PATCH 01/65] add I'm Online Heartbeat section, reference relevant sections --- host-spec/c04-networking.tm | 172 +++++++++++++++++++++++++++++++++++- 1 file changed, 170 insertions(+), 2 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 95dca51c8..854eafd31 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -1,8 +1,8 @@ - + -> +> <\body> @@ -119,6 +119,24 @@ sends blocks to connected peers. + + + The I'm Online heartbeat is a crucial part of the Polkadot validation + process, as it signals the active participation of validators and confirms + their reachability. The Polkadot network punishes unreachable validators + which have been elected to an authority by slashing their bonded funds. + This is achieved by requiring validators to issue an I'm Online heartbeat, + which comes in the form of a signed extrinsic, on the start of every Era. + + The Polkadot Runtime fully manages the creation and the timing of that + signed extrinsic, but it's the responsiblity of the Host to gossip that + extrinsic to the rest of the network. When the Runtime decides to create + the heartbeat, it will call the + Host API as described in section . + + The process of gossiping extrinsics is defined in section + . + @@ -629,3 +647,153 @@ \; + +<\initial> + <\collection> + + + > + + + + +<\references> + <\collection> + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + + + +<\auxiliary> + <\collection> + <\associate|toc> + |math-font-series||font-size||4Networking> + |.>>>>|> + + + |math-font-series||1Introduction> + |.>>>>|> + + + |1.1External Documentation + |.>>>>|> + > + + |1.2Discovery mechanism + |.>>>>|> + > + + |1.3Connection establishment + |.>>>>|> + > + + |1.4Substreams + |.>>>>|> + > + + |1.5I'm Online Heartbeat + |.>>>>|> + > + + |math-font-series||2Network + Messages> |.>>>>|> + + + |2.1API Package + |.>>>>|> + > + + |2.1.1BlockRequest + |.>>>>|> + > + + |2.1.2BlockResponse + |.>>>>|> + > + + |2.2Light Package + |.>>>>|> + > + + |2.2.1Request + |.>>>>|> + > + + |2.2.2Response + |.>>>>|> + > + + |2.2.3RemoteCallRequest + |.>>>>|> + > + + |2.2.4RemoteCallResponse + |.>>>>|> + > + + |2.2.5RemoteReadRequest + |.>>>>|> + > + + |2.2.6RemoteReadResponse + |.>>>>|> + > + + |2.2.7RemoteReadChildRequest + |.>>>>|> + > + + |2.2.8RemoteHeaderRequest + |.>>>>|> + > + + |2.2.9RemoteHeaderResponse + |.>>>>|> + > + + |2.2.10RemoteChangesRequest + |.>>>>|> + > + + |2.2.11RemoteChangesResponse + |.>>>>|> + > + + |2.3Finality Package + |.>>>>|> + > + + |2.3.1FinalityProofRequest + |.>>>>|> + > + + |2.3.2FinalityProofResponse + |.>>>>|> + > + + + \ No newline at end of file From a3c4bfa8811efc2bcd30d1e7c99b5d02950ce268 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Mon, 7 Dec 2020 17:41:09 +0100 Subject: [PATCH 02/65] add section about node identities --- host-spec/c04-networking.tm | 73 +++++++++++++++++++++++-------------- 1 file changed, 46 insertions(+), 27 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 854eafd31..a2b94bfec 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -36,19 +36,35 @@ - + - The Polkadot Host uses varies mechanism to find peers within the network, + Each Polkadot Host node maintains a ED25519 key pair which is used to + identify the node. The public key is shared with the rest of the network. + This allows nodes to establish secure communication channels. Nodes are + discovered as described in the Discovery Mechanism Section + (), where each node can be identified + by their corresponding (). + + Each node must have its own unique ED25519 key pair. Using the same pair + among two or more nodes is interpreted as bad behavior. + + <\definition> + The Polkadot node's , formally + referred to as >, is derived from the ED25519 public key + and is structured as defined in the libp2p specification + (). + + + + + The Polkadot Host uses various mechanisms to find peers within the network, to establish and maintain a list of peers and to share that list with other peers from the network. - The Polkadot Host uses various mechanism for peer dicovery. - <\itemize> Bootstrap nodes - hard-coded node identities and addresses provided - by network configuration itself. Those addresses are selected an updated - by the developers of the Polkadot Host. Node addresses should be selected - based on a reputation metric, such as reliability and uptime. + by the genesis state specification as described in Appendix + . mDNS - performs a broadcast to the local network. Nodes that might be listing can respond the the broadcast. @@ -132,7 +148,7 @@ signed extrinsic, but it's the responsiblity of the Host to gossip that extrinsic to the rest of the network. When the Runtime decides to create the heartbeat, it will call the - Host API as described in section . + Host API as described in Section . The process of gossiping extrinsics is defined in section . @@ -660,31 +676,34 @@ <\references> <\collection> > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > > > > > > - > - > + > + > + > + > From 7b4a08d5033a660b2354b5b531578e980bc31ff3 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Mon, 7 Dec 2020 20:02:00 +0100 Subject: [PATCH 03/65] scrap deprecated networking section --- host-spec/c04-networking.tm | 46 +++--- host-spec/c04a-networking.tm | 285 ----------------------------------- 2 files changed, 26 insertions(+), 305 deletions(-) delete mode 100644 host-spec/c04a-networking.tm diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index a2b94bfec..3dedb4f29 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -67,7 +67,8 @@ . mDNS - performs a broadcast to the local network. Nodes that might - be listing can respond the the broadcast. + be listing can respond the the broadcast. The libp2p mDNS specification + defines this process in more detail (). Kademlia requests - Kademlia supports requests, where nodes respond with their list of available peers. @@ -97,6 +98,10 @@ After a base-layer protocol is established, the Polkadot Host will apply the Noise protocol. + + + + After the node establishes a connection with a peer, the use of @@ -676,32 +681,33 @@ <\references> <\collection> > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > > > > > > > - > + > > > diff --git a/host-spec/c04a-networking.tm b/host-spec/c04a-networking.tm deleted file mode 100644 index aedcf60cf..000000000 --- a/host-spec/c04a-networking.tm +++ /dev/null @@ -1,285 +0,0 @@ - - - - - - -<\body> - > - - <\warning> - Polkadot network protocol is work-in-progress. The API specification and - usage may change in future. - - - This chapter offers a high-level description of the network protocol based - on . Polkadot network protocol - relies on . Specifically, the following libp2p modules are being - used in the Polkadot Networking protocol: - - <\itemize> - - - - - - - - - (kademlia) - - - - - - - For more detailed specification of these modules and the Peer-to-Peer layer - see libp2p specification document . - - - - Similar to other decentralized networks, each Polkadot Host node possesses - a network private key and a network public key representing an ED25519 key - pair . - - - - <\definition> - , formally noted by > is derived - from the node's public key as follows: - - >> \ and uniquely identifies a - node on the network. - - - Because the > is derived from the node's public key, - running two or more instances of Polkadot network using the same network - key is contrary to the Polkadot protocol. - - All network communications between nodes on the network use encryption - derived from both sides' keys. - - - - - - \ In order for a Polkadot node to join a peer-to-peer network, it has to - know a list of Polkadot nodes that already take part in the network. This - process of building such a list is referred to as . Each - element of this list is a pair consisting of the peer's node identities and - their addresses.\ - - - - \ Polkadot discovery is done through the following mechanisms: - - <\itemize> - : These are hard-coded node identities and - addresses passed alongside with the network configuration. - - , performing a UDP broadcast on the local network. Nodes - that listen may respond with their identity as described in the mDNS - section of . (Note: mDNS can be disabled - in the network configuration.) - - . Once connected to a peer node, a - Polkadot node can perform a random Kademlia `FIND_NODE` requests for the - nodes to respond by propagating their view of the - network. - - - - - A Polkadot node can establish a connection with nodes in its peer list. All - the connections must always use encryption and multiplexing. While some - nodes' addresses (eg. addresses using `/quic`) already imply the encryption - and/or multiplexing to use, for others the \Pmultistream-select\Q protocol - is used in order to negotiate an encryption layer and/or a multiplexing - layer. - - The following transport protocol is supported by a Polkadot node: - - <\itemize> - for addresses of the form `/ip4/1.2.3.4/tcp/5`. Once - the TCP connection is open, an encryption and a multiplexing layers are - negotiated on top. - - for addresses of the form `/ip4/1.2.3.4/tcp/5/ws`. - A TC/IP connection is open and the WebSockets protocol is negotiated on - top. Communications then happen inside WebSockets data frames. Encryption - and multiplexing are additionally negotiated again inside this channel. - - \ DNS for addresses of the form `/dns4/example.com/tcp/5` or - `/dns4/example.com/tcp/5/ws`. A node's address can contain a domain name. - - - - - \ The following encryption protocols from libp2p are supported by Polkadot - protocol: - - : A TLS-1.2-like protocol but without certificates - . Support for secio will likely to be - deprecated in the future. - - : Noise is a framework for crypto protocols based on - the Diffie-Hellman key agreement . Support for - noise is experimental and details may change in the future. - - \ - - The following multiplexing protocols are supported: - - <\itemize> - : Support for mplex will be deprecated in the future. - - . - - - - - Once a connection has been established between two nodes and is able to use - multiplexing, substreams can be opened. When a substream is open, the - protocol is used to negotiate which protocol to use - on that given substream.\ - - - - A Polkadot Host node should open several substreams. In particular, it - should periodically open ephemeral substreams in order to: - - <\itemize> - ping the remote peer and check whether the connection is still - alive. Failure for the remote peer to reply leads to a disconnection. - This uses the libp2p protocol specified in - . - - ask information from the remote. This is the protocol - specified in . - - send Kademlia random walk queries. Each Kademlia query is done in a - new separate substreams. This uses the libp2p protocol - specified in . - - - - - For the purposes of communicating Polkadot messages, the dailer of the - connection opens a unique substream. Optionally, the node can keep a unique - substream alive for this purpose. The name of the protocol negotiated is - based on the passed as part of the network configuration. - This protocol ID should be unique for each chain and prevents nodes from - different chains to connect to each other. - - The structure of SCALE encoded messages sent over the unique Polkadot - communication substream is described in Appendix - . - - Once the substream is open, the first step is an exchange of a - message from both sides described in Section . - - Communications within this substream include: - - <\itemize> - Syncing. Blocks are announced and requested from other nodes. - - Gossiping. Used by various subprotocols such as GRANDPA. - - Polkadot Network Specialization: . - - - \; - - <\with|par-mode|right> - - - - \; - - -<\initial> - <\collection> - - - - - - -<\references> - <\collection> - > - > - > - > - > - > - > - > - > - > - > - > - > - - - -<\auxiliary> - <\collection> - <\associate|bib> - parity_technologies_substrate_2019 - - protocol_labs_libp2p_2019 - - liusvaara_edwards-curve_2017 - - protocol_labs_libp2p_2019 - - protocol_labs_libp2p_2019 - - perrin_noise_2018 - - protocol_labs_libp2p_2019 - - protocol_labs_libp2p_2019 - - protocol_labs_libp2p_2019 - - <\associate|toc> - |math-font-series||1Network - Protocol> |.>>>>|> - - - 1.1Node Identities and Addresses - |.>>>>|> - - - 1.2Discovery Mechanisms - |.>>>>|> - - - 1.3Transport Protocol |.>>>>|> - - - |1.3.1Encryption - |.>>>>|> - > - - |1.3.2Multiplexing - |.>>>>|> - > - - 1.4Substreams |.>>>>|> - - - |1.4.1Periodic Ephemeral - Substreams |.>>>>|> - > - - |1.4.2Polkadot Communication - Substream |.>>>>|> - > - - - From 5596818852aed346657b1861d64191455f9c221d Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Mon, 7 Dec 2020 22:32:52 +0100 Subject: [PATCH 04/65] brief note about protocol id --- host-spec/c04-networking.tm | 114 ++++++++++++++++++++---------------- 1 file changed, 65 insertions(+), 49 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 3dedb4f29..feaaa71fd 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -70,10 +70,17 @@ be listing can respond the the broadcast. The libp2p mDNS specification defines this process in more detail (). - Kademlia requests - Kademlia supports - requests, where nodes respond with their list of available peers. + Kademlia requests - Kademlia is a distributed hash table for + decentralized networks and supports requests, where + nodes respond with their list of available peers. + + + Kademlia nodes only communicate with other nodes using the same protocol + identifier. The Polkadot network is identified by + (respectively for Kusama). + The Polkadot Host can establish a connection with any peer it knows the @@ -131,7 +138,7 @@ allows the Polkadot Host to perform information about blocks. - a request and response protocol that - allows a light client to perform information about the state. + allows a light client to perform request information about the state. - a notification protocol which sends transactions to connected peers. @@ -681,33 +688,34 @@ <\references> <\collection> > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > > > - > - > - > - > + > + > + > + > > > @@ -728,97 +736,105 @@ |.>>>>|> > - |1.2Discovery mechanism + |1.2Node Identities |.>>>>|> > - |1.3Connection establishment + |1.3Discovery mechanism |.>>>>|> > - |1.4Substreams + |1.4Connection establishment |.>>>>|> > - |1.5I'm Online Heartbeat + |1.5Encryption |.>>>>|> > + |1.6Substreams + |.>>>>|> + > + + |1.7I'm Online Heartbeat + |.>>>>|> + > + |math-font-series||2Network Messages> |.>>>>|> - + |2.1API Package |.>>>>|> - > + > |2.1.1BlockRequest |.>>>>|> - > + > |2.1.2BlockResponse |.>>>>|> - > + > |2.2Light Package |.>>>>|> - > + > |2.2.1Request |.>>>>|> - > + > |2.2.2Response |.>>>>|> - > + > |2.2.3RemoteCallRequest |.>>>>|> - > + > |2.2.4RemoteCallResponse |.>>>>|> - > + > |2.2.5RemoteReadRequest |.>>>>|> - > + > |2.2.6RemoteReadResponse |.>>>>|> - > + > |2.2.7RemoteReadChildRequest |.>>>>|> - > + > |2.2.8RemoteHeaderRequest |.>>>>|> - > + > |2.2.9RemoteHeaderResponse |.>>>>|> - > + > |2.2.10RemoteChangesRequest |.>>>>|> - > + > |2.2.11RemoteChangesResponse |.>>>>|> - > + > |2.3Finality Package |.>>>>|> - > + > |2.3.1FinalityProofRequest |.>>>>|> - > + > |2.3.2FinalityProofResponse |.>>>>|> - > + > \ No newline at end of file From 09c567fd46f323ac8b6a6a248d6ca1ede27420b4 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Wed, 9 Dec 2020 19:50:23 +0100 Subject: [PATCH 05/65] define BlockAnnounce and BlockAnnounceHandshake --- host-spec/c04-networking.tm | 168 ++++++++++++++++++++++++++---------- 1 file changed, 124 insertions(+), 44 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index feaaa71fd..f51d4d227 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -147,6 +147,75 @@ sends blocks to connected peers. + + + The Polkadot Host must actively communicate with the network in order to + participate in the validation process. This section describes the expected + behaviors of the node. + + + + When the node creates a new block, it must be announced to the network. + Other nodes within the network will track this announcement and can request + information about this block. This process is described in Section + . + + <\definition> + The initializes a substream to a remote + peer. Once established, all messages created by + the node are sent to that substream. + + The is a SCALE encoded structure of the + following format: + + <\eqnarray*> + >||,h,h|)>>>>> + + + where: + + <\eqnarray*> + |||>>||>>||>>>>>>>|>||>>|>||>>|>||>>>> + + + \; + + + <\definition> + The BlockAnnounce message is sent to the specified substream and + indicates to remote peers the that node has either created or received a + new block. + + The message is a SCALE encoded structure of the + following format: + + <\eqnarray*> + ||,ib|)>>>>> + + + where: + + <\eqnarray*> + >||>>||||>>||>>>>>>>>> + + + + + + The Polkadot node can receive a handful of messages from its peers, by + which the Polkadot node is expected to act accordingly. + + \; + The I'm Online heartbeat is a crucial part of the Polkadot validation @@ -679,8 +748,8 @@ <\initial> <\collection> - - > + + @@ -689,27 +758,30 @@ <\collection> > > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > + > + > + > > > > @@ -744,97 +816,105 @@ |.>>>>|> > - |1.4Connection establishment + |1.3.1Protocol Identifier |.>>>>|> > - |1.5Encryption + |1.4Connection establishment |.>>>>|> > - |1.6Substreams + |1.5Encryption |.>>>>|> > - |1.7I'm Online Heartbeat + |1.6Substreams |.>>>>|> > + |1.7Behavior for incoming + messages |.>>>>|> + > + + |1.8I'm Online Heartbeat + |.>>>>|> + > + |math-font-series||2Network Messages> |.>>>>|> - + |2.1API Package |.>>>>|> - > + > |2.1.1BlockRequest |.>>>>|> - > + > |2.1.2BlockResponse |.>>>>|> - > + > |2.2Light Package |.>>>>|> - > + > |2.2.1Request |.>>>>|> - > + > |2.2.2Response |.>>>>|> - > + > |2.2.3RemoteCallRequest |.>>>>|> - > + > |2.2.4RemoteCallResponse |.>>>>|> - > + > |2.2.5RemoteReadRequest |.>>>>|> - > + > |2.2.6RemoteReadResponse |.>>>>|> - > + > |2.2.7RemoteReadChildRequest |.>>>>|> - > + > |2.2.8RemoteHeaderRequest |.>>>>|> - > + > |2.2.9RemoteHeaderResponse |.>>>>|> - > + > |2.2.10RemoteChangesRequest |.>>>>|> - > + > |2.2.11RemoteChangesResponse |.>>>>|> - > + > |2.3Finality Package |.>>>>|> - > + > |2.3.1FinalityProofRequest |.>>>>|> - > + > |2.3.2FinalityProofResponse |.>>>>|> - > + > \ No newline at end of file From e06fed067fe90b56735ad6974413e62f121db368 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Fri, 11 Dec 2020 16:35:50 +0100 Subject: [PATCH 06/65] define Protobuf BlockRequesttype --- host-spec/c04-networking.tm | 149 +++++++++++++++++++++++++----------- 1 file changed, 103 insertions(+), 46 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index f51d4d227..2de1b06f0 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -183,14 +183,12 @@ block hash according to the node>>>|>||>>>> - - \; <\definition> - The BlockAnnounce message is sent to the specified substream and - indicates to remote peers the that node has either created or received a - new block. + The message is sent to the specified substream + and indicates to remote peers the that node has either created or + received a new block. The message is a SCALE encoded structure of the following format: @@ -209,6 +207,53 @@ + + + Block requests can be used to retrieve a range of blocks from peers. The + BlockRequest message is a Protobuf serialized structure of the following + format: + + <\big-table|>|>|>|>>||||>>>||||>>>||||>>>||||>||||>>>>>>> + \; + + + where + + <\itemize-dot> + > indictes all the fields should be included in the + request. It's big endian encoded bitmask which applies all desired fields + with bitwise OR operations. For example, the > value for + and is . + + <\big-table|>|>>||>||>||>||>||>>>>> + \; + + + > is a varying data type of the following values: + + <\big-table|>|>|>>|||>|||>>>>> + \; + + + > is either the block hash or block number depending + on the value of >. An implementation defined maximum is + used when unspecified. + + > is the number of blocks to be returned. An + implementation defined maximum is used when unspecified. + + + \; + The Polkadot node can receive a handful of messages from its peers, by @@ -759,29 +804,33 @@ > > > - > - > + > + > > - > - > - > - > - > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > + > + > + > + > + > + > + > > > > @@ -832,89 +881,97 @@ |.>>>>|> > - |1.7Behavior for incoming + |1.7Behavior for ougoing messages |.>>>>|> > - |1.8I'm Online Heartbeat + |1.7.1Announcing blocks |.>>>>|> > + |1.8Behavior for incoming + messages |.>>>>|> + > + + |1.9I'm Online Heartbeat + |.>>>>|> + > + |math-font-series||2Network Messages> |.>>>>|> - + |2.1API Package |.>>>>|> - > + > |2.1.1BlockRequest |.>>>>|> - > + > |2.1.2BlockResponse |.>>>>|> - > + > |2.2Light Package |.>>>>|> - > + > |2.2.1Request |.>>>>|> - > + > |2.2.2Response |.>>>>|> - > + > |2.2.3RemoteCallRequest |.>>>>|> - > + > |2.2.4RemoteCallResponse |.>>>>|> - > + > |2.2.5RemoteReadRequest |.>>>>|> - > + > |2.2.6RemoteReadResponse |.>>>>|> - > + > |2.2.7RemoteReadChildRequest |.>>>>|> - > + > |2.2.8RemoteHeaderRequest |.>>>>|> - > + > |2.2.9RemoteHeaderResponse |.>>>>|> - > + > |2.2.10RemoteChangesRequest |.>>>>|> - > + > |2.2.11RemoteChangesResponse |.>>>>|> - > + > |2.3Finality Package |.>>>>|> - > + > |2.3.1FinalityProofRequest |.>>>>|> - > + > |2.3.2FinalityProofResponse |.>>>>|> - > + > \ No newline at end of file From c93943fa36393e54b1cf3b27a93915dd7e8b517c Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Fri, 11 Dec 2020 16:38:14 +0100 Subject: [PATCH 07/65] add footer to protobuf tables --- host-spec/c04-networking.tm | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 2de1b06f0..98c70c982 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -219,7 +219,7 @@ at this block (optional)>|>>>||||>||||>>>>>>> - \; + Protobuf message. where @@ -234,14 +234,14 @@ 0001>>||>||>||>||>>>>> - \; + Bits of block data to be requested. > is a varying data type of the following values: <\big-table|>|>|>>|||>|||>>>>> - \; + Block hash or number to start from. > is either the block hash or block number depending From 70b5a087e696cf42cc75daa9223f579fef6ada28 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Fri, 11 Dec 2020 17:08:11 +0100 Subject: [PATCH 08/65] define BlockResponse and BlockData protobuf structures --- host-spec/c04-networking.tm | 219 +++++++++++++++++++++++------------- 1 file changed, 140 insertions(+), 79 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 98c70c982..b4909badc 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -209,48 +209,80 @@ - Block requests can be used to retrieve a range of blocks from peers. The - BlockRequest message is a Protobuf serialized structure of the following - format: - - <\big-table|>|>|>|>>||||>>>||||>>>||||>>>||||>||||>>>>>>> - Protobuf message. - - - where - - <\itemize-dot> - > indictes all the fields should be included in the - request. It's big endian encoded bitmask which applies all desired fields - with bitwise OR operations. For example, the > value for - and is . - - <\big-table|>|>>||>||>||>||>||>>>>> - Bits of block data to be requested. + Block requests can be used to retrieve a range of blocks from peers. + + <\definition> + The message is a Protobuf serialized structure of + the following format: + + <\big-table|>|>|>|>>||||>>>||||>>>||||>>>||||>||||>>>>>>> + Protobuf message. - > is a varying data type of the following values: + where - <\big-table|>|>|>>|||>|||>>>>> - Block hash or number to start from. - + <\itemize-dot> + > indictes all the fields that should be included + in the request. It's big endian encoded bitmask which applies all + desired fields with bitwise OR operations. For example, the + > value to request and + is (17). + + <\big-table|>|>>||>||>||>||>||>>>>> + Bits of block data to be requested. + + + > is a varying data type of the following values: + + <\big-table|>|>|>>|||>|||>>>>> + Protobuf message indicating the block to start from. + - > is either the block hash or block number depending - on the value of >. An implementation defined maximum is - used when unspecified. + > is either the block hash or block number + depending on the value of >. An implementation defined + maximum is used when unspecified. + + > is the number of blocks to be returned. An + implementation defined maximum is used when unspecified. + + + + <\definition> + The message is received after sending a + message to a peer. The message is a Protobuf + serialized structure of the following format: + + <\big-table||||||>|>|>>|||>|||>>>>> + Protobuf message. + - > is the number of blocks to be returned. An - implementation defined maximum is used when unspecified. - + where is a Protobuf structure containing the + requested blocks. Do note that the optional values are either present or + absent depending on the requested fields (bitmask value). The structure + has the following format: + + <\big-table|||||||||||||||||||||||||>|>|>|>>||||>||||>||||>||||>||||>||||>||||>||||>||||>>>>> + \; + + \; @@ -808,29 +840,31 @@ > > > - > - > - > - > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > > > > @@ -844,6 +878,29 @@ <\auxiliary> <\collection> + <\associate|table> + |1>|> + |language||BlockRequest> + Protobuf message. + |> + + |2>|> + Bits of block data to be requested. + |> + + |3>|> + Protobuf message indicating the block to start from. + |> + + |4>|> + |language||BlockResponse> + Protobuf message. + |> + + |5>|> + \; + |> + <\associate|toc> |math-font-series||font-size||4Networking> |.>>>>|> @@ -889,89 +946,93 @@ |.>>>>|> > + |1.7.2Requesting blocks + |.>>>>|> + > + |1.8Behavior for incoming messages |.>>>>|> - > + > |1.9I'm Online Heartbeat |.>>>>|> - > + > |math-font-series||2Network Messages> |.>>>>|> - + |2.1API Package |.>>>>|> - > + > |2.1.1BlockRequest |.>>>>|> - > + > |2.1.2BlockResponse |.>>>>|> - > + > |2.2Light Package |.>>>>|> - > + > |2.2.1Request |.>>>>|> - > + > |2.2.2Response |.>>>>|> - > + > |2.2.3RemoteCallRequest |.>>>>|> - > + > |2.2.4RemoteCallResponse |.>>>>|> - > + > |2.2.5RemoteReadRequest |.>>>>|> - > + > |2.2.6RemoteReadResponse |.>>>>|> - > + > |2.2.7RemoteReadChildRequest |.>>>>|> - > + > |2.2.8RemoteHeaderRequest |.>>>>|> - > + > |2.2.9RemoteHeaderResponse |.>>>>|> - > + > |2.2.10RemoteChangesRequest |.>>>>|> - > + > |2.2.11RemoteChangesResponse |.>>>>|> - > + > |2.3Finality Package |.>>>>|> - > + > |2.3.1FinalityProofRequest |.>>>>|> - > + > |2.3.2FinalityProofResponse |.>>>>|> - > + > \ No newline at end of file From b5da2fbc2d9dc3af8e4490d4ffc4a82b961a5ead Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Fri, 11 Dec 2020 17:14:24 +0100 Subject: [PATCH 09/65] add Direction protobuf structure --- host-spec/c04-networking.tm | 59 ++++++++++++++++++++++--------------- 1 file changed, 36 insertions(+), 23 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index b4909badc..4be244cd5 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -240,7 +240,8 @@ Bits of block data to be requested. - > is a varying data type of the following values: + > is a Protobuf structure indicating a varying + data type of the following values: <\big-table|>|>|>>|||>|||>>>>> @@ -251,6 +252,17 @@ depending on the value of >. An implementation defined maximum is used when unspecified. + is a Protobuf structure indicating the + sequence direction of the requested blocks. The structure is a varying + data type of the following format: + + <\big-table||||||||||>|>>||>||>||>||>>>>> + Protobuf structure. + + > is the number of blocks to be returned. An implementation defined maximum is used when unspecified. @@ -280,7 +292,7 @@ (optional)>|>||||>||||>>>>> - \; + Protobuf structure. @@ -842,29 +854,30 @@ > > > - > - > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > > > From f1dc1103c6140d56378d56c6e96497a1622d7360 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Fri, 11 Dec 2020 17:17:11 +0100 Subject: [PATCH 10/65] remove deprecated protobuf s ubsections --- host-spec/c04-networking.tm | 170 +++--------------------------------- 1 file changed, 14 insertions(+), 156 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 4be244cd5..ed8688a54 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -296,8 +296,6 @@ - \; - The Polkadot node can receive a handful of messages from its peers, by @@ -325,146 +323,6 @@ - - - ProtoBuf details: - - <\itemize> - syntax: proto3 - - package: api.v1 - - - - - Request block data from a peer. - - <\verbatim> - \; - - message BlockRequest { - - \ \ \ \ // Bits of block data to request. - - \ \ \ \ uint32 fields = 1; - - \ \ \ \ // Start from this block. - - \ \ \ \ oneof from_block { - - \ \ \ \ \ \ \ \ // Start with given hash. - - \ \ \ \ \ \ \ \ bytes hash = 2; - - \ \ \ \ \ \ \ \ // Start with given block number. - - \ \ \ \ \ \ \ \ bytes number = 3; - - \ \ \ \ } - - \ \ \ \ // End at this block. An implementation defined - - \ \ \ \ // maximum is used when unspecified. - - \ \ \ \ bytes to_block = 4; // optional - - \ \ \ \ // Sequence direction. - - \ \ \ \ Direction direction = 5; - - \ \ \ \ // Maximum number of blocks to return. An implementation - - \ \ \ \ // defined maximum is used when unspecified. - - \ \ \ \ uint32 max_blocks = 6; // optional - - } - - \; - - // Block enumeration direction - - enum Direction { - - \ \ \ \ // Enumerate in ascending order - - \ \ \ \ // (from child to parent). - - \ \ \ \ Ascending = 0; - - \ \ \ \ // Enumerate in descending order - - \ \ \ \ // (from parent to canonical child). - - \ \ \ \ Descending = 1; - - } - - \; - - - - - Response to Block Request. - - <\verbatim> - \; - - message BlockResponse { - - \ \ \ \ // Block data for the requested sequence. - - \ \ \ \ repeated BlockData blocks = 1; - - } - - \; - - // Block data sent in the response. - - message BlockData { - - \ \ \ \ // Block header hash. - - \ \ \ \ bytes hash = 1; - - \ \ \ \ // Block header if requested. - - \ \ \ \ bytes header = 2; // optional - - \ \ \ \ // Block body if requested. - - \ \ \ \ repeated bytes body = 3; // optional - - \ \ \ \ // Block receipt if requested. - - \ \ \ \ bytes receipt = 4; // optional - - \ \ \ \ // Block message queue if requested. - - \ \ \ \ bytes message_queue = 5; // optional - - \ \ \ \ // Justification if requested. - - \ \ \ \ bytes justification = 6; // optional - - \ \ \ \ // True if justification should be treated as present but - - \ \ \ \ // empty. This hack is unfortunately necessary because - - \ \ \ \ // shortcomings in the protobuf format otherwise doesn't - - \ \ \ \ // make it possible to differentiate between a lack of - - \ \ \ \ // justification and an empty justification. - - \ \ \ \ bool is_empty_justification = 7; // optional, false if absent - - } - - \; - - ProtoBuf details: @@ -862,21 +720,21 @@ > > > - > - > - > - > - > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > > > > From 7ef2a5abc56ea1da6f02c553bb01e4b9865c6033 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Fri, 11 Dec 2020 17:20:01 +0100 Subject: [PATCH 11/65] remove light and finality protobuf messages --- host-spec/c04-networking.tm | 370 ------------------------------------ 1 file changed, 370 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index ed8688a54..436c14a14 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -320,376 +320,6 @@ The process of gossiping extrinsics is defined in section . - - - - - - ProtoBuf details: - - <\itemize> - syntax: proto3 - - package: api.v1.light - - - - - Enumerate all possible light client request messages. - - <\verbatim> - \; - - message Request { - - \ \ \ \ oneof request { - - \ \ \ \ \ \ \ \ RemoteCallRequest remote_call_request = 1; - - \ \ \ \ \ \ \ \ RemoteReadRequest remote_read_request = 2; - - \ \ \ \ \ \ \ \ RemoteHeaderRequest remote_header_request = 3; - - \ \ \ \ \ \ \ \ RemoteReadChildRequest remote_read_child_request = 4; - - \ \ \ \ \ \ \ \ RemoteChangesRequest remote_changes_request = 5; - - \ \ \ \ } - - } - - \; - - - - - Enumerate all possible light client response messages. - - <\verbatim> - \; - - message Response { - - \ \ \ \ oneof response { - - \ \ \ \ \ \ \ \ RemoteCallResponse remote_call_response = 1; - - \ \ \ \ \ \ \ \ RemoteReadResponse remote_read_response = 2; - - \ \ \ \ \ \ \ \ RemoteHeaderResponse remote_header_response = 3; - - \ \ \ \ \ \ \ \ RemoteChangesResponse remote_changes_response = 4; - - \ \ \ \ } - - } - - \; - - - - - Remote call request. - - <\verbatim> - \; - - message RemoteCallRequest { - - \ \ \ \ // Block at which to perform call. - - \ \ \ \ bytes block = 2; - - \ \ \ \ // Method name. - - \ \ \ \ string method = 3; - - \ \ \ \ // Call data. - - \ \ \ \ bytes data = 4; - - } - - \; - - - - - Remote call response. - - <\verbatim> - \; - - message RemoteCallResponse { - - \ \ \ \ // Execution proof. - - \ \ \ \ bytes proof = 2; - - } - - \; - - - - - Remote storage read request. - - <\verbatim> - \; - - message RemoteReadRequest { - - \ \ \ \ // Block at which to perform call. - - \ \ \ \ bytes block = 2; - - \ \ \ \ // Storage keys. - - \ \ \ \ repeated bytes keys = 3; - - } - - \; - - - - - Remote read response. - - <\verbatim> - \; - - message RemoteReadResponse { - - \ \ \ \ // Read proof. - - \ \ \ \ bytes proof = 2; - - } - - \; - - - - - Remote storage read child request. - - <\verbatim> - \; - - message RemoteReadChildRequest { - - \ \ \ \ // Block at which to perform call. - - \ \ \ \ bytes block = 2; - - \ \ \ \ // Child Storage key, this is relative - - \ \ \ \ // to the child type storage location. - - \ \ \ \ bytes storage_key = 3; - - \ \ \ \ // Storage keys. - - \ \ \ \ repeated bytes keys = 6; - - } - - \; - - - - - Remote header request. - - <\verbatim> - \; - - message RemoteHeaderRequest { - - \ \ \ \ // Block number to request header for. - - \ \ \ \ bytes block = 2; - - } - - \; - - - - - Remote header response. - - <\verbatim> - \; - - message RemoteHeaderResponse { - - \ \ \ \ // Header. None if proof generation has failed - - \ \ \ \ // (e.g. header is unknown). - - \ \ \ \ bytes header = 2; // optional - - \ \ \ \ // Header proof. - - \ \ \ \ bytes proof = 3; - - } - - \; - - - - - Remote changes request. - - <\verbatim> - \; - - message RemoteChangesRequest { - - \ \ \ \ // Hash of the first block of the range (including first) - - \ \ \ \ // where changes are requested. - - \ \ \ \ bytes first = 2; - - \ \ \ \ // Hash of the last block of the range (including last) - - \ \ \ \ // where changes are requested. - - \ \ \ \ bytes last = 3; - - \ \ \ \ // Hash of the first block for which the requester has - - \ \ \ \ // the changes trie root. All other - - \ \ \ \ // affected roots must be proved. - - \ \ \ \ bytes min = 4; - - \ \ \ \ // Hash of the last block that we can use when - - \ \ \ \ // querying changes. - - \ \ \ \ bytes max = 5; - - \ \ \ \ // Storage child node key which changes are requested. - - \ \ \ \ bytes storage_key = 6; // optional - - \ \ \ \ // Storage key which changes are requested. - - \ \ \ \ bytes key = 7; - - } - - \; - - - - - Remote changes response. - - <\verbatim> - \; - - message RemoteChangesResponse { - - \ \ \ \ // Proof has been generated using block with this number - - \ \ \ \ // as a max block. Should be less than or equal to the - - \ \ \ \ // RemoteChangesRequest::max block number. - - \ \ \ \ bytes max = 2; - - \ \ \ \ // Changes proof. - - \ \ \ \ repeated bytes proof = 3; - - \ \ \ \ // Changes tries roots missing on the requester' node. - - \ \ \ \ repeated Pair roots = 4; - - \ \ \ \ // Missing changes tries roots proof. - - \ \ \ \ bytes roots_proof = 5; - - } - - \; - - // A pair of arbitrary bytes. - - message Pair { - - \ \ \ \ // The first element of the pair. - - \ \ \ \ bytes fst = 1; - - \ \ \ \ // The second element of the pair. - - \ \ \ \ bytes snd = 2; - - } - - \; - - - - - ProtoBuf details: - - <\itemize> - syntax: proto3 - - package: api.v1.finality - - - - - Request a finality proof from a peer. - - <\verbatim> - \; - - message FinalityProofRequest { - - \ \ \ \ // SCALE-encoded hash of the block to request. - - \ \ \ \ bytes block_hash = 1; - - \ \ \ \ // Opaque chain-specific additional request data. - - \ \ \ \ bytes request = 2; - - } - - \; - - - - - Response to a finality proof request. - - <\verbatim> - \; - - message FinalityProofResponse { - - \ \ \ \ // Opaque chain-specific finality proof. - - \ \ \ \ // Empty if no such proof exists. - - \ \ \ \ bytes proof = 1; // optional - - } - - \; - <\initial> From 465004aede3fa0df6331dc6e70ed24735df1309d Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Fri, 11 Dec 2020 19:27:10 +0100 Subject: [PATCH 12/65] link to relevant block header sections --- host-spec/c04-networking.tm | 27 ++++++++++----------------- 1 file changed, 10 insertions(+), 17 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 436c14a14..283723f5d 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -147,7 +147,7 @@ sends blocks to connected peers. - + The Polkadot Host must actively communicate with the network in order to participate in the validation process. This section describes the expected @@ -215,7 +215,7 @@ The message is a Protobuf serialized structure of the following format: - <\big-table|>|>|>|>>||||||||>|>|>|>>||||>>>||||>>>||||>>>|||> value to request and is (17). - <\big-table|>|>>|||||||>|>>||>||>||>||>||>>>>> @@ -243,7 +243,7 @@ > is a Protobuf structure indicating a varying data type of the following values: - <\big-table|>|>|>>||||||||>|>|>>|||>|||>>>>> Protobuf message indicating the block to start from. @@ -284,25 +284,18 @@ has the following format: <\big-table|||||||||||||||||||||||||>|>|>|>>||||>||||>||||>||||>||||>>||||>>||||>>||||>||||>||||>||||>||||>||||>>>>> + (optional)>|>>||||>||||>>>>> Protobuf structure. - - - The Polkadot node can receive a handful of messages from its peers, by - which the Polkadot node is expected to act accordingly. - - \; - The I'm Online heartbeat is a crucial part of the Polkadot validation From c949896b82c15d87759acd999691a339dbf8a84c Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Mon, 14 Dec 2020 18:11:46 +0100 Subject: [PATCH 13/65] document both types of substream protocols --- host-spec/c04-networking.tm | 146 +++++++++++------------------------- 1 file changed, 43 insertions(+), 103 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 283723f5d..c13fb52f0 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -83,6 +83,27 @@ + Polkadot nodes connect to peers by establishing a TCP connection. Once + established, the node initiates a handshake with the remote peer's + encryption layer. An additional layer, know as the multiplexing layer, + allows a connection to be split into substreams, either by the local or + remote node. + + The Polkadot node supports two kind of substream protocols: + + <\itemize-dot> + . After the protocol is + negotiated, the opening node sends a single message containing a request. + The remote node then sends a response, after which the substream is then + immediately closed. + + . After the protocol is negotiated, + the opening node sends a single hand shake message. The remote node can + then either accept or reject the substream. After the substream has been + accepted, the opening node can send an unbound number of individual + messages. + + The Polkadot Host can establish a connection with any peer it knows the address. uses the protocol in order to establish an encryption and multiplexing layer. The Polkadot @@ -296,6 +317,12 @@ + + + The Polkadot Host must send certain events to its peers, which is referred + to as \Pgossiping\Q. This includes extrinsics, Grandpa votes, + justifications and equivocations. + The I'm Online heartbeat is a crucial part of the Polkadot validation @@ -339,26 +366,9 @@ > > > - > - > - > - > - > - > - > - > - > + > + > > - > - > - > - > - > - > - > - > - > - > > > > @@ -387,13 +397,19 @@ |> |4>|> - |language||BlockResponse> - Protobuf message. + |language||Direction> + Protobuf structure. |> |5>|> - \; + |language||BlockResponse> + Protobuf message. |> + + |6>|> + |math-font-series||BlockData> + Protobuf structure. + |> <\associate|toc> |math-font-series||font-size||4Networking> @@ -432,8 +448,8 @@ |.>>>>|> > - |1.7Behavior for ougoing - messages |.>>>>|> + |1.7Network Messages + |.>>>>|> > |1.7.1Announcing blocks @@ -444,89 +460,13 @@ |.>>>>|> > - |1.8Behavior for incoming - messages |.>>>>|> - > - - |1.9I'm Online Heartbeat + |1.8Gossiping |.>>>>|> > - |math-font-series||2Network - Messages> |.>>>>|> - - - |2.1API Package - |.>>>>|> - > - - |2.1.1BlockRequest - |.>>>>|> - > - - |2.1.2BlockResponse - |.>>>>|> - > - - |2.2Light Package - |.>>>>|> - > - - |2.2.1Request - |.>>>>|> - > - - |2.2.2Response - |.>>>>|> - > - - |2.2.3RemoteCallRequest - |.>>>>|> - > - - |2.2.4RemoteCallResponse - |.>>>>|> - > - - |2.2.5RemoteReadRequest - |.>>>>|> - > - - |2.2.6RemoteReadResponse - |.>>>>|> - > - - |2.2.7RemoteReadChildRequest - |.>>>>|> - > - - |2.2.8RemoteHeaderRequest - |.>>>>|> - > - - |2.2.9RemoteHeaderResponse - |.>>>>|> - > - - |2.2.10RemoteChangesRequest - |.>>>>|> - > - - |2.2.11RemoteChangesResponse - |.>>>>|> - > - - |2.3Finality Package - |.>>>>|> - > - - |2.3.1FinalityProofRequest - |.>>>>|> - > - - |2.3.2FinalityProofResponse + |1.9I'm Online Heartbeat |.>>>>|> - > + > \ No newline at end of file From 813e4a7bc78ef84bd369f6b6054a97574f41317d Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Mon, 14 Dec 2020 22:23:54 +0100 Subject: [PATCH 14/65] add basic section about noise protocol --- host-spec/c04-networking.tm | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index c13fb52f0..cf12f5fa3 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -126,9 +126,26 @@ After a base-layer protocol is established, the Polkadot Host will apply the Noise protocol. - + - + The Noise protocol is a framework for bulding encryption protocols. lipp2p + utilizes that protocol for establishing encrypted communication channels. + + Polkadot nodes use the XX handshake pattern + () to establish a connection + between peers. Three steps are required to successfully complete the + handshake process: + + <\enumerate-numeric> + The initiator generates a keypair and sends the public key to the + responder. + + The responder generates its own keypair and sends its public key + back to the initiator. + + Both the initiator and the responder can derive a shared secret and + use it to encrypt all further communication. + From fc9d8c33320cae0a596fab8081456ffa93337047 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Mon, 14 Dec 2020 23:35:15 +0100 Subject: [PATCH 15/65] add additional protocols to the substream section --- host-spec/c04-networking.tm | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index cf12f5fa3..4616cd6ef 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -128,8 +128,9 @@ - The Noise protocol is a framework for bulding encryption protocols. lipp2p - utilizes that protocol for establishing encrypted communication channels. + The Noise protocol is a framework for bulding encryption protocols. + utilizes that protocol for establishing encrypted + communication channels. Polkadot nodes use the XX handshake pattern () to establish a connection @@ -154,10 +155,15 @@ the negotiation of , where each protocol servers a specific utility. - The Polkadot Host adoptes the following, standardized - application-specific protocols: + The Polkadot Host adoptes the following substreams: <\itemize> + - Open a substream for the Noise protocol to + establish a encryption layer. + + - Open a substream for handshakes to + negotiate a new protocol. + - Open a substream to a peer and initialize a ping to verify if a connection is till alive. If the peer does not respond, the connection is dropped. @@ -169,8 +175,6 @@ requests. - Additional, non-standardized protocols: - <\itemize> - a request and response protocol that allows the Polkadot Host to perform information about blocks. @@ -340,6 +344,8 @@ to as \Pgossiping\Q. This includes extrinsics, Grandpa votes, justifications and equivocations. + + The I'm Online heartbeat is a crucial part of the Polkadot validation From ccbb4a024ca79d0c99264bd7c589fd2f45750509 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Mon, 14 Dec 2020 23:41:48 +0100 Subject: [PATCH 16/65] link to mplex spec --- host-spec/c04-networking.tm | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 4616cd6ef..191332513 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -33,6 +33,8 @@ + + @@ -85,9 +87,9 @@ Polkadot nodes connect to peers by establishing a TCP connection. Once established, the node initiates a handshake with the remote peer's - encryption layer. An additional layer, know as the multiplexing layer, - allows a connection to be split into substreams, either by the local or - remote node. + encryption layer. An additional layer, know as the multiplexing layer + (), allows a + connection to be split into substreams, either by the local or remote node. The Polkadot node supports two kind of substream protocols: From 505a4d620537c4766aa4a95fbe4e0a2ef6808b87 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Mon, 14 Dec 2020 23:47:30 +0100 Subject: [PATCH 17/65] adjust introdution to mplex --- host-spec/c04-networking.tm | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 191332513..0c01fc438 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -87,9 +87,9 @@ Polkadot nodes connect to peers by establishing a TCP connection. Once established, the node initiates a handshake with the remote peer's - encryption layer. An additional layer, know as the multiplexing layer - (), allows a - connection to be split into substreams, either by the local or remote node. + encryption layer. An additional layer, know as the multiplexing layer, + allows a connection to be split into substreams, either by the local or + remote node. The Polkadot node supports two kind of substream protocols: @@ -107,9 +107,7 @@ The Polkadot Host can establish a connection with any peer it knows the - address. uses the protocol - in order to establish an encryption and multiplexing layer. The Polkadot - Host supports multiple base-layer protocols: + address. The Polkadot Host supports multiple base-layer protocols: <\itemize> TCP/IP - addresses in the form of @@ -153,9 +151,11 @@ After the node establishes a connection with a peer, the use of - multiplexing allows the Polkadot Host to open substreams. Substreams allow - the negotiation of , - where each protocol servers a specific utility. + multiplexing allows the Polkadot Host to open substreams. + uses the protocol () + to mange substream and to allow the negotiation of + , where each + protocol servers a specific utility. The Polkadot Host adoptes the following substreams: @@ -201,8 +201,8 @@ When the node creates a new block, it must be announced to the network. Other nodes within the network will track this announcement and can request - information about this block. This process is described in Section - . + information about this block. The mechanism for tracking announements and + requesting the required data is implementation specific. <\definition> The initializes a substream to a remote @@ -272,8 +272,8 @@ <\itemize-dot> > indictes all the fields that should be included - in the request. It's big endian encoded bitmask which applies all - desired fields with bitwise OR operations. For example, the + in the request. It's encoded bitmask which applies + all desired fields with bitwise OR operations. For example, the > value to request and is (17). From 95749778632a3ee58209b8b07e8a6cf28aeba49a Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 17:18:35 +0100 Subject: [PATCH 18/65] expand section of the Noise protocol overview --- host-spec/c04-networking.tm | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 0c01fc438..59d507f9a 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -142,12 +142,22 @@ responder. The responder generates its own keypair and sends its public key - back to the initiator. - - Both the initiator and the responder can derive a shared secret and - use it to encrypt all further communication. + back to the initiator. After that, the responder derives a shared secret + and uses it to encrypt all further communication. The responder now sends + its static Noise public key (which is non-persistant and generated on + every node startup), its public key and a signature of + the static Noise public key signed with the public key. + + The initiator derives a shared secret and uses it to encrypt all + further communication. It also sends its static Noise public key, + public key and a signature to the responder. + After these three steps, both the initiator and responder derive a new + sharedsecret using the static and session-defined Noise keys, + which is used to encrypt all further communication. The Noise specification + describes this process in detail. + After the node establishes a connection with a peer, the use of @@ -391,8 +401,6 @@ > > > - > - > > > > @@ -465,7 +473,7 @@ |.>>>>|> > - |1.5Encryption + |1.5Noise Protocol |.>>>>|> > From 1f3ea75fdf309b967eb1cbf477c051c832b6e74d Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 17:51:24 +0100 Subject: [PATCH 19/65] spec offchain_submit_transaction --- host-spec/ae-hostapi.tm | 345 +++++++++++++++++++++------------------- 1 file changed, 177 insertions(+), 168 deletions(-) diff --git a/host-spec/ae-hostapi.tm b/host-spec/ae-hostapi.tm index fabeb3e5c..edf3c1716 100644 --- a/host-spec/ae-hostapi.tm +++ b/host-spec/ae-hostapi.tm @@ -1304,34 +1304,10 @@ : a pointer-size as defined in Definition indicating the data to be hashed. - : a regular pointer to the buffer containing the + : a reglar pointer to the buffer containing the 256-bit hash result. - > - - Conducts a 512-bit Keccak hash. - - - - <\verbatim> - (func $ext_hashing_keccak_512_version_1 - - \ \ (param $data i64) (return i32)) - - - \; - - : - - <\itemize-dot> - : a pointer-size as defined in Definition - indicating the data to be hashed. - - : a regular pointer to the buffer containing the - 512-bit hash result. - - > Conducts a 256-bit Sha2 hash. @@ -2334,21 +2310,59 @@ indicating the log message. + + + Interface that provides functions for offchain functionality. + + + + Determines whether the local node is a running as a validator. Even if this + function returns , it does not mean that any keys are + configured and that the validators is registered in the chain. The + mechanism for determining this condition is implementation specific. + + + + + \; - <\with|par-mode|right> - - \ + : + + <\itemize-dot> + : if the node is is running as a + validator or if otherwise. + + + + + Submits an encoded transaction to the pool. + + + + + + \; + + : + + <\itemize-dot> + : a SCALE encoded as defined in + Definition yielding an empty value on both + success or failure. The cause of a failure is implementation specific. + + + \; <\initial> <\collection> - + - > + @@ -2358,74 +2372,77 @@ > > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > + > + > + > + > + > > > > @@ -2516,7 +2533,7 @@ > > > - > + > > > > @@ -2554,11 +2571,11 @@ |A.3>|> Table of possible HTTP error types - |> + |> |A.4>|> Log Levels for the logging interface - |> + |> <\associate|toc> |math-font-series||Appendix @@ -2893,7 +2910,7 @@ |.>>>>|> > - |A.4.2|language||ext_hashing_keccak_512> + |A.4.2|language||ext_hashing_sha2_256> |.>>>>|> > @@ -2901,7 +2918,7 @@ |.>>>>|> > - |A.4.3|language||ext_hashing_sha2_256> + |A.4.3|language||ext_hashing_blake2_128> |.>>>>|> > @@ -2909,7 +2926,7 @@ |.>>>>|> > - |A.4.4|language||ext_hashing_blake2_128> + |A.4.4|language||ext_hashing_blake2_256> |.>>>>|> > @@ -2917,7 +2934,7 @@ |.>>>>|> > - |A.4.5|language||ext_hashing_blake2_256> + |A.4.5|language||ext_hashing_twox_64> |.>>>>|> > @@ -2925,7 +2942,7 @@ |.>>>>|> > - |A.4.6|language||ext_hashing_twox_64> + |A.4.6|language||ext_hashing_twox_128> |.>>>>|> > @@ -2933,7 +2950,7 @@ |.>>>>|> > - |A.4.7|language||ext_hashing_twox_128> + |A.4.7|language||ext_hashing_twox_256> |.>>>>|> > @@ -2941,244 +2958,236 @@ |.>>>>|> > - |A.4.8|language||ext_hashing_twox_256> - |.>>>>|> - > - - |A.4.8.1Version 1 - Prototype - |.>>>>|> - > - A.5Offchain |.>>>>|> - + |A.5.1|language||ext_offchain_is_validator> |.>>>>|> - > + > |A.5.1.1Version 1 - Prototype |.>>>>|> - > + > |A.5.2|language||ext_offchain_submit_transaction> |.>>>>|> - > + > |A.5.2.1Version 1 - Prototype |.>>>>|> - > + > |A.5.3|language||ext_offchain_network_state> |.>>>>|> - > + > |A.5.3.1Version 1 - Prototype |.>>>>|> - > + > |A.5.4|language||ext_offchain_timestamp> |.>>>>|> - > + > |A.5.4.1Version 1 - Prototype |.>>>>|> - > + > |A.5.5|language||ext_offchain_sleep_until> |.>>>>|> - > + > |A.5.5.1Version 1 - Prototype |.>>>>|> - > + > |A.5.6|language||ext_offchain_random_seed> |.>>>>|> - > + > |A.5.6.1Version 1 - Prototype |.>>>>|> - > + > |A.5.7|language||ext_offchain_local_storage_set> |.>>>>|> - > + > |A.5.7.1Version 1 - Prototype |.>>>>|> - > + > |A.5.8|language||ext_offchain_local_storage_compare_and_set> |.>>>>|> - > + > |A.5.8.1Version 1 - Prototype |.>>>>|> - > + > |A.5.9|language||ext_offchain_local_storage_get> |.>>>>|> - > + > |A.5.9.1Version 1 - Prototype |.>>>>|> - > + > |A.5.10|language||ext_offchain_http_request_start> |.>>>>|> - > + > |A.5.10.1Version 1 - Prototype |.>>>>|> - > + > |A.5.11|language||ext_offchain_http_request_add_header> |.>>>>|> - > + > |A.5.11.1Version 1 - Prototype |.>>>>|> - > + > |A.5.12|language||ext_offchain_http_request_write_body> |.>>>>|> - > + > |A.5.12.1Version 1 - Prototype |.>>>>|> - > + > |A.5.13|language||ext_offchain_http_response_wait> |.>>>>|> - > + > |A.5.13.1Version 1- Prototype |.>>>>|> - > + > |A.5.14|language||ext_offchain_http_response_headers> |.>>>>|> - > + > |A.5.14.1Version 1 - Prototype |.>>>>|> - > + > |A.5.15|language||ext_offchain_http_response_read_body> |.>>>>|> - > + > |A.5.15.1Version 1 - Prototype |.>>>>|> - > + > A.6Trie |.>>>>|> - + |A.6.1|language||ext_trie_blake2_256_root> |.>>>>|> - > + > |A.6.1.1Version 1 - Prototype |.>>>>|> - > + > |A.6.2|language||ext_trie_blake2_256_ordered_root> |.>>>>|> - > + > |A.6.2.1Version 1 - Prototype |.>>>>|> - > + > |A.6.3|language||ext_trie_keccak_256_root> |.>>>>|> - > + > |A.6.3.1Version 1 - Prototype |.>>>>|> - > + > |A.6.4|language||ext_trie_keccak_256_ordered_root> |.>>>>|> - > + > |A.6.4.1Version 1 - Prototype |.>>>>|> - > + > A.7Miscellaneous |.>>>>|> - + |A.7.1|language||ext_misc_chain_id> |.>>>>|> - > + > |A.7.1.1Version 1 - Prototype |.>>>>|> - > + > |A.7.2|language||ext_misc_print_num> |.>>>>|> - > + > |A.7.2.1Version 1 - Prototype |.>>>>|> - > + > |A.7.3|language||ext_misc_print_utf8> |.>>>>|> - > + > |A.7.3.1Version 1 - Prototype |.>>>>|> - > + > |A.7.4|language||ext_misc_print_hex> |.>>>>|> - > + > |A.7.4.1Version 1 - Prototype |.>>>>|> - > + > |A.7.5|language||ext_misc_runtime_version> |.>>>>|> - > + > |A.7.5.1Version 1 - Prototype |.>>>>|> - > + > A.8Allocator |.>>>>|> - + |A.8.1|language||ext_allocator_malloc> |.>>>>|> - > + > |A.8.1.1Version 1 - Prototype |.>>>>|> - > + > |A.8.2|language||ext_allocator_free> |.>>>>|> - > + > |A.8.2.1Version 1 - Prototype |.>>>>|> - > + > A.9Logging |.>>>>|> - + |A.9.1|language||ext_logging_log> |.>>>>|> - > + > |A.9.1.1Version 1 - Prototype |.>>>>|> - > + > \ No newline at end of file From 308d9a33ba317846edb53cfb6e019976c33a4107 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 18:08:26 +0100 Subject: [PATCH 20/65] spec offchain_network_state --- host-spec/ae-hostapi.tm | 43 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/host-spec/ae-hostapi.tm b/host-spec/ae-hostapi.tm index edf3c1716..92c7c35c3 100644 --- a/host-spec/ae-hostapi.tm +++ b/host-spec/ae-hostapi.tm @@ -2352,7 +2352,47 @@ success or failure. The cause of a failure is implementation specific. + + + Returns information about the local node's network state. + + <\definition> + The + structure, >, is a SCALE encoded blob holding information + about the the PeerId, >, of the local node and a list of + Multiaddresses, (\M>), the node knows it + can be reached at: + + <\eqnarray*> + >||,\M|)>|)>>>>> + + + where: + + <\eqnarray*> + >||\b|)>>>|>||\b|)>>>>> + + + The information contained in this structure is naturally opaque to the + caller of this function. + + + + + + \; + + : + + <\itemize-dot> + : a pointer-size as defined in Definition + indicating the SCALE encoded Result as + defined in Definition containing + structure as defined in Definition + . On failure no additional data is + provided, where its cause is implementation specific. + <\initial> @@ -2443,6 +2483,8 @@ > > > + > + > > > > @@ -2542,6 +2584,7 @@ > > > + > > > > From 6723ab984b2f1749519ff090b4823ed569b847e8 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 18:11:42 +0100 Subject: [PATCH 21/65] spec offchain_timestamp --- host-spec/ae-hostapi.tm | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/host-spec/ae-hostapi.tm b/host-spec/ae-hostapi.tm index 92c7c35c3..462d4eb4c 100644 --- a/host-spec/ae-hostapi.tm +++ b/host-spec/ae-hostapi.tm @@ -2393,6 +2393,23 @@ . On failure no additional data is provided, where its cause is implementation specific. + + + + Returns the current UNIX timestamp in milliseconds. + + + + + + \; + + : + + <\itemize-dot> + result: an unsigned 64-bit integer containing the current UNIX + timestamp in milliseconds. + <\initial> @@ -2485,6 +2502,8 @@ > > > + > + > > > > From 23defb030533bc34bbddfd3c7a630e9fb2e14e87 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 19:10:51 +0100 Subject: [PATCH 22/65] spec offchain_sleep_until --- host-spec/ae-hostapi.tm | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/host-spec/ae-hostapi.tm b/host-spec/ae-hostapi.tm index 462d4eb4c..546e2050d 100644 --- a/host-spec/ae-hostapi.tm +++ b/host-spec/ae-hostapi.tm @@ -2407,8 +2407,25 @@ : <\itemize-dot> - result: an unsigned 64-bit integer containing the current UNIX - timestamp in milliseconds. + : an unsigned 64-bit integer containing the + current UNIX timestamp in milliseconds. + + + + + Pause the execution until the deadline is reached. + + + + + + \; + + : + + <\itemize-dot> + : an unsigned 64-bit integer containing the UNIX + timestamp in milliseconds until the execution should pause to. @@ -2504,7 +2521,9 @@ > > > + > > + > > > > From 89466517bbdf5a71d29ff4e28ac123ffab83f846 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 19:16:47 +0100 Subject: [PATCH 23/65] spec offchain_random_seed --- host-spec/ae-hostapi.tm | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/host-spec/ae-hostapi.tm b/host-spec/ae-hostapi.tm index 546e2050d..be6b6c658 100644 --- a/host-spec/ae-hostapi.tm +++ b/host-spec/ae-hostapi.tm @@ -2427,6 +2427,26 @@ : an unsigned 64-bit integer containing the UNIX timestamp in milliseconds until the execution should pause to. + + + + Returns a cryptographically safe, random and non-deterministic seed + generated by the host environment. + + + + + + \; + + : + + <\itemize-dot> + : a regular pointer indicating the memory location + of the generated seed of 32 bytes. + + + \; <\initial> @@ -2524,6 +2544,8 @@ > > > + > + > > > > From dd5974e0bc6befc87d04e9f75721ad1683561b73 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 19:30:32 +0100 Subject: [PATCH 24/65] revert offchain API changes, those are already specced --- host-spec/ae-hostapi.tm | 242 +++++++++++++++------------------------- 1 file changed, 88 insertions(+), 154 deletions(-) diff --git a/host-spec/ae-hostapi.tm b/host-spec/ae-hostapi.tm index be6b6c658..5d0b6b373 100644 --- a/host-spec/ae-hostapi.tm +++ b/host-spec/ae-hostapi.tm @@ -1530,9 +1530,9 @@ > - Verifies if the local node is a potential validator. Even if this function - returns true, it does not mean that any keys are configured or that the - validator is registered in the chain. + Check whether the local node is a potential validator. Even if this + function returns , it does not mean that any keys are + configured or that the validator is registered in the chain. @@ -1575,18 +1575,35 @@ : a pointer-size as defined in Definition indicating the SCALE encoded as defined in Definition . - Neither on success or failure is there any additional data provided. + Neither on success or failure is there any additional data provided. The + cause of a failure is implementation specific. > Returns the SCALE encoded, opaque information about the local node's - network state. This information is fetched by calling into - , which include the and - possible by which the node is publicly known - by. Those values are unique and have to be known by the node individually. - Due to its opaque nature, it's unknown whether that information is - available prior to execution. + network state. + + <\definition> + The + structure, >, is a SCALE encoded blob holding information + about the the PeerId, >, of the local node and a list of + Multiaddresses, (\M>), the node knows it + can be reached at: + + <\eqnarray*> + >||,\M|)>|)>>>>> + + + where: + + <\eqnarray*> + >||\b|)>>>|>||\b|)>>>>> + + + The information contained in this structure is naturally opaque to the + caller of this function. + > @@ -1602,10 +1619,9 @@ : a pointer-size as defined in Definition indicating the SCALE encoded as defined in Definition . - On success it contains the SCALE encoded network state. This includes - none or one followed by none, one or more IPv4 or IPv6 - by which the node is publicly known by. On - failure no additional data is provided. + On success it contains the structure as + defined in Definition . On failure, + an empty value is yielded where its cause is implementation specific. > @@ -2309,144 +2325,6 @@ : a pointer-size as defined in Definition indicating the log message. - - - - Interface that provides functions for offchain functionality. - - - - Determines whether the local node is a running as a validator. Even if this - function returns , it does not mean that any keys are - configured and that the validators is registered in the chain. The - mechanism for determining this condition is implementation specific. - - - - - - \; - - : - - <\itemize-dot> - : if the node is is running as a - validator or if otherwise. - - - - - Submits an encoded transaction to the pool. - - - - - - \; - - : - - <\itemize-dot> - : a SCALE encoded as defined in - Definition yielding an empty value on both - success or failure. The cause of a failure is implementation specific. - - - - - Returns information about the local node's network state. - - <\definition> - The - structure, >, is a SCALE encoded blob holding information - about the the PeerId, >, of the local node and a list of - Multiaddresses, (\M>), the node knows it - can be reached at: - - <\eqnarray*> - >||,\M|)>|)>>>>> - - - where: - - <\eqnarray*> - >||\b|)>>>|>||\b|)>>>>> - - - The information contained in this structure is naturally opaque to the - caller of this function. - - - - - - - \; - - : - - <\itemize-dot> - : a pointer-size as defined in Definition - indicating the SCALE encoded Result as - defined in Definition containing - structure as defined in Definition - . On failure no additional data is - provided, where its cause is implementation specific. - - - - - Returns the current UNIX timestamp in milliseconds. - - - - - - \; - - : - - <\itemize-dot> - : an unsigned 64-bit integer containing the - current UNIX timestamp in milliseconds. - - - - - Pause the execution until the deadline is reached. - - - - - - \; - - : - - <\itemize-dot> - : an unsigned 64-bit integer containing the UNIX - timestamp in milliseconds until the execution should pause to. - - - - - Returns a cryptographically safe, random and non-deterministic seed - generated by the host environment. - - - - - - \; - - : - - <\itemize-dot> - : a regular pointer indicating the memory location - of the generated seed of 32 bytes. - - - \; <\initial> @@ -2546,6 +2424,7 @@ > > > + > > > > @@ -2643,8 +2522,8 @@ > > > - > - > + > + > > > > @@ -3291,6 +3170,61 @@ |A.9.1.1Version 1 - Prototype |.>>>>|> > + + A.10Offchain |.>>>>|> + + + |A.10.1offchain_is_validator + |.>>>>|> + > + + |A.10.1.1Version 1 - Prototype + |.>>>>|> + > + + |A.10.2offchain_submit_transaction + |.>>>>|> + > + + |A.10.2.1Version 1 - Prototype + |.>>>>|> + > + + |A.10.3offchain_network_state + |.>>>>|> + > + + |A.10.3.1Version 1 - Prototype + |.>>>>|> + > + + |A.10.4offchain_timestamp + |.>>>>|> + > + + |A.10.4.1Version 1 - Prototype + |.>>>>|> + > + + |A.10.5offchain_sleep_until + |.>>>>|> + > + + |A.10.5.1Version 1 - Prototype + |.>>>>|> + > + + |A.10.6offchain_random_seed + |.>>>>|> + > + + |A.10.6.1Version 1 - Prototype + |.>>>>|> + > + + |A.10.7offchain_local_storage_set + |.>>>>|> + > \ No newline at end of file From bc0974315bb91870b4fb705e1e822417f00bf8ae Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 19:39:47 +0100 Subject: [PATCH 25/65] spec offchain_local_storage_clear --- host-spec/ae-hostapi.tm | 119 ++++++++++++++++++++++-------------- host-spec/c01-background.tm | 54 ++++++++-------- 2 files changed, 100 insertions(+), 73 deletions(-) diff --git a/host-spec/ae-hostapi.tm b/host-spec/ae-hostapi.tm index 5d0b6b373..ce2dcc50a 100644 --- a/host-spec/ae-hostapi.tm +++ b/host-spec/ae-hostapi.tm @@ -1631,7 +1631,7 @@ <\verbatim> - (func $ext_offchain_timestamp_version_1 (result i64)) + (func $ext_offchain_timestamp_version_1 (result u64)) \ @@ -1639,7 +1639,7 @@ : <\itemize> - : an i64 integer indicating the current UNIX + : an u64 integer indicating the current UNIX timestamp as defined in Definition . @@ -1650,7 +1650,7 @@ <\verbatim> - (func $ext_offchain_sleep_until_version_1 (param $deadline i64)) + (func $ext_offchain_sleep_until_version_1 (param $deadline u64)) \ @@ -1658,7 +1658,7 @@ : <\itemize> - : an i64 integer specifying the UNIX timestamp + : an u64 integer specifying the UNIX timestamp as defined in Definition . @@ -1678,8 +1678,8 @@ : <\itemize> - : a pointer to the buffer containing the 256-bit - seed. + : a regular pointer to the buffer containing the + 256-bit seed. > @@ -1702,9 +1702,9 @@ <\itemize> : an i32 integer indicating the storage kind. A - value equal to 1 is used for a persistent storage as defined in - Definition and a value equal - to 2 for local storage as defined in Definition + value equal to is used for a persistent storage as defined + in Definition and a value + equal to for local storage as defined in Definition . : a pointer-size as defined in Definition @@ -1714,6 +1714,33 @@ indicating the value. + > + + Remove a value from the local storage. + + + + <\verbatim> + (func $ext_offchain_local_storage_clear_version_1 + + \ \ (param $kind i32) (param $key i64)) + + + \; + + Arguments: + + <\itemize-dot> + : an i32 integer indicating the storage kind. A + value equal to is used for a persistent storage as defined + in Definition and a value + equal to for local storage as defined in Definition + . + + key: a pointer-size as defined in Definition + indicating the key. + + > Sets a new value in the local storage if the condition matches the current @@ -1736,9 +1763,9 @@ <\itemize> : an i32 integer indicating the storage kind. A - value equal to 1 is used for a persistent storage as defined in - Definition and a value equal - to 2 for local storage as defined in Definition + value equal to is used for a persistent storage as defined + in Definition and a value + equal to for local storage as defined in Definition . : a pointer-size as defined in Definition @@ -1774,9 +1801,9 @@ <\itemize> : an i32 integer indicating the storage kind. A - value equal to 1 is used for a persistent storage as defined in - Definition and a value equal - to 2 for local storage as defined in Definition + value equal to is used for a persistent storage as defined + in Definition and a value + equal to for local storage as defined in Definition . : a pointer-size as defined in Definition @@ -2378,40 +2405,40 @@ > > > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > + > + > > > > diff --git a/host-spec/c01-background.tm b/host-spec/c01-background.tm index 7c9bdb4a3..250c06592 100644 --- a/host-spec/c01-background.tm +++ b/host-spec/c01-background.tm @@ -1,8 +1,8 @@ - + - +> <\body> @@ -350,8 +350,8 @@ <\initial> <\collection> - - > + + @@ -359,32 +359,32 @@ <\references> <\collection> > - > - > - > - > + > + > + > + > > - > - > - > - > - > + > + > + > + > + > > - > - > - > - > - > - > - > - > + > + |>|?>> + |B,B>)|?>> + |\>BT|)>>|?>> + |\>|?>> + > + > + > > - > - > - > - > - > - > + |math-font-series|||P>>|?>> + |\>|?>> + > + |B>|?>> + |Enc>|?>> + > > > > From 0359a7d5f45cace684ff0f1734501d07037809ac Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 19:41:42 +0100 Subject: [PATCH 26/65] mention that cause of failure is implemenation specifc --- host-spec/ae-hostapi.tm | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/host-spec/ae-hostapi.tm b/host-spec/ae-hostapi.tm index ce2dcc50a..46643a4dc 100644 --- a/host-spec/ae-hostapi.tm +++ b/host-spec/ae-hostapi.tm @@ -1847,7 +1847,8 @@ indicating the SCALE encoded as defined in Definition containing the i16 ID of the newly started request. On failure no - additionally data is provided. + additionally data is provided. The cause of failure is implementation + specific. > @@ -1883,7 +1884,8 @@ : a pointer-size as defined in Definition indicating the SCALE encoded as defined in Definition . - Neither on success or failure is there any additional data provided. + Neither on success or failure is there any additional data provided. The + cause of failure is implemenation specific. > From 99ef20ce4860d391c7722d7ed87dd0b63d5b0307 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 20:14:33 +0100 Subject: [PATCH 27/65] spec ext_offchain_set_authorized_nodes --- host-spec/ae-hostapi.tm | 99 +++++++++++++++++++++++++++-------------- 1 file changed, 65 insertions(+), 34 deletions(-) diff --git a/host-spec/ae-hostapi.tm b/host-spec/ae-hostapi.tm index 46643a4dc..eb86a397e 100644 --- a/host-spec/ae-hostapi.tm +++ b/host-spec/ae-hostapi.tm @@ -1587,9 +1587,10 @@ <\definition> The structure, >, is a SCALE encoded blob holding information - about the the PeerId, >, of the local node and a list of - Multiaddresses, (\M>), the node knows it - can be reached at: + about the the , >, of + the local node and a list of + (\M>), the node knows it can be reached + at: <\eqnarray*> >||,\M|)>|)>>>>> @@ -2033,6 +2034,36 @@ on faiure. + > + + Set the authorized nodes which are allowed to connect to the local node. + This function is primarily used for private blockchains and is not + necessarily required for the public and open Polkadot protocol. + + + + <\verbatim> + (func $ext_offchain_set_authorized_nodes_version_1 + + \ \ (param $nodes i64) (param $authorized_only i32) + + + \; + + : + + <\itemize-dot> + : a pointer-size as defined in Definition + indicating the buffer of the SCALE + encoded array of 's. Invalid + 's are silently ignored. + + : If set to , then only the + authorized nodes are allowed to connect to the local node (whitelist). + All other nodes are rejected. If set to , then no such + restriction is placed. + + Interface that provides trie related functionality. @@ -2409,40 +2440,40 @@ > > > - > - > - > - > - > - > - > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > + > + > + > + > > > > From 15201a647b0a0abdcb7233ed4c667a57ed2b8de0 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 21:40:30 +0100 Subject: [PATCH 28/65] add transactions and consensus message subsection in networking --- host-spec/c04-networking.tm | 82 ++++++++++++++++++++++++++++++++----- 1 file changed, 72 insertions(+), 10 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 59d507f9a..70ea581c8 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -163,7 +163,7 @@ After the node establishes a connection with a peer, the use of multiplexing allows the Polkadot Host to open substreams. uses the protocol () - to mange substream and to allow the negotiation of + to manage substream and to allow the negotiation of , where each protocol servers a specific utility. @@ -214,6 +214,9 @@ information about this block. The mechanism for tracking announements and requesting the required data is implementation specific. + Block announcements and requests are conducted on the + substream. + <\definition> The initializes a substream to a remote peer. Once established, all messages created by @@ -223,7 +226,7 @@ following format: <\eqnarray*> - >||,h,h|)>>>>> + >||,h,h|)>>>>> where: @@ -248,7 +251,7 @@ following format: <\eqnarray*> - ||,ib|)>>>>> + ||,ib|)>>>>> where: @@ -350,13 +353,67 @@ - + + + Transactions are sent directly in its full form to connected peers. It's + considered good behavior to implement a mechanism which only sends a + transaction once to each peer and avoids sending duplicates. Such a + mechanism is implementation specific and any absence of such a mechanism + can result in consequences which are undefined. + + The transactions message is represented by > and is defined + as follows: + + <\equation*> + M\Enc,\,C|)> + + + in which: + + <\equation*> + C\Enc|)> + + + Where each > is a byte array and represents a sepearate + extrinsic. The Polkadot Host is indifferent about the content of an + extrinsic and treats it as a blob of data. + + The exchange of transactions is conducted on the + substream. + + + + A represented by > is sent to + communicate messages related to consensus process: + + <\equation*> + M\Enc,D|)> + + + Where: + + <\center> + ||||||>:>||>>>|>||>>>>>> + + + \; + + in which + + <\equation*> + E\BABE>||>>|FRNK>||>>>>>|\> + - The Polkadot Host must send certain events to its peers, which is referred - to as \Pgossiping\Q. This includes extrinsics, Grandpa votes, - justifications and equivocations. + \; - + The network agent should hand over to approperiate consensus + engine which identified by >. @@ -398,9 +455,12 @@ > > > - > + > > - > + > + > + > + > > > > @@ -410,6 +470,8 @@ > > > + > + > From e0cdf90b22608746acefe850f7de5e62c58b1e6b Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 22:36:30 +0100 Subject: [PATCH 29/65] reference transaction submission api --- host-spec/ae-hostapi.tm | 166 +++++++++++++----------------------- host-spec/c04-networking.tm | 16 ++-- 2 files changed, 69 insertions(+), 113 deletions(-) diff --git a/host-spec/ae-hostapi.tm b/host-spec/ae-hostapi.tm index eb86a397e..59c908648 100644 --- a/host-spec/ae-hostapi.tm +++ b/host-spec/ae-hostapi.tm @@ -1550,11 +1550,11 @@ if it is not. - > + > - Given an extrinsic as a SALE encoded byte array, the system decodes the - byte array and submits the extrinsic in the inherent pool as an extrinsic - to be included in the next produced block. + Given a SCALE encoded extrinsic, this function submits the extrinsic to the + Host's transaction pool, ready to be propagated to remote peers. This + process is critical for issuing the I'm Online message. @@ -2474,17 +2474,7 @@ > > > - > - > - > - > - > - > > - > - > - > - > > > > @@ -2593,6 +2583,7 @@ > > > + > > > > @@ -2617,7 +2608,7 @@ |A.4>|> Log Levels for the logging interface - |> + |> <\associate|toc> |math-font-series||Appendix @@ -3059,7 +3050,7 @@ |.>>>>|> > - |A.5.8|language||ext_offchain_local_storage_compare_and_set> + |A.5.8|language||ext_offchain_local_storage_clear> |.>>>>|> > @@ -3067,7 +3058,7 @@ |.>>>>|> > - |A.5.9|language||ext_offchain_local_storage_get> + |A.5.9|language||ext_offchain_local_storage_compare_and_set> |.>>>>|> > @@ -3075,7 +3066,7 @@ |.>>>>|> > - |A.5.10|language||ext_offchain_http_request_start> + |A.5.10|language||ext_offchain_local_storage_get> |.>>>>|> > @@ -3083,7 +3074,7 @@ |.>>>>|> > - |A.5.11|language||ext_offchain_http_request_add_header> + |A.5.11|language||ext_offchain_http_request_start> |.>>>>|> > @@ -3091,7 +3082,7 @@ |.>>>>|> > - |A.5.12|language||ext_offchain_http_request_write_body> + |A.5.12|language||ext_offchain_http_request_add_header> |.>>>>|> > @@ -3099,23 +3090,23 @@ |.>>>>|> > - |A.5.13|language||ext_offchain_http_response_wait> + |A.5.13|language||ext_offchain_http_request_write_body> |.>>>>|> > - |A.5.13.1Version 1- Prototype + |A.5.13.1Version 1 - Prototype |.>>>>|> > - |A.5.14|language||ext_offchain_http_response_headers> + |A.5.14|language||ext_offchain_http_response_wait> |.>>>>|> > - |A.5.14.1Version 1 - Prototype + |A.5.14.1Version 1- Prototype |.>>>>|> > - |A.5.15|language||ext_offchain_http_response_read_body> + |A.5.15|language||ext_offchain_http_response_headers> |.>>>>|> > @@ -3123,168 +3114,129 @@ |.>>>>|> > + |A.5.16|language||ext_offchain_http_response_read_body> + |.>>>>|> + > + + |A.5.16.1Version 1 - Prototype + |.>>>>|> + > + + |A.5.17|language||ext_offchain_set_authorized_nodes> + |.>>>>|> + > + + |A.5.17.1Version 1 - Prototype + |.>>>>|> + > + A.6Trie |.>>>>|> - + |A.6.1|language||ext_trie_blake2_256_root> |.>>>>|> - > + > |A.6.1.1Version 1 - Prototype |.>>>>|> - > + > |A.6.2|language||ext_trie_blake2_256_ordered_root> |.>>>>|> - > + > |A.6.2.1Version 1 - Prototype |.>>>>|> - > + > |A.6.3|language||ext_trie_keccak_256_root> |.>>>>|> - > + > |A.6.3.1Version 1 - Prototype |.>>>>|> - > + > |A.6.4|language||ext_trie_keccak_256_ordered_root> |.>>>>|> - > + > |A.6.4.1Version 1 - Prototype |.>>>>|> - > + > A.7Miscellaneous |.>>>>|> - + |A.7.1|language||ext_misc_chain_id> |.>>>>|> - > + > |A.7.1.1Version 1 - Prototype |.>>>>|> - > + > |A.7.2|language||ext_misc_print_num> |.>>>>|> - > + > |A.7.2.1Version 1 - Prototype |.>>>>|> - > + > |A.7.3|language||ext_misc_print_utf8> |.>>>>|> - > + > |A.7.3.1Version 1 - Prototype |.>>>>|> - > + > |A.7.4|language||ext_misc_print_hex> |.>>>>|> - > + > |A.7.4.1Version 1 - Prototype |.>>>>|> - > + > |A.7.5|language||ext_misc_runtime_version> |.>>>>|> - > + > |A.7.5.1Version 1 - Prototype |.>>>>|> - > + > A.8Allocator |.>>>>|> - + |A.8.1|language||ext_allocator_malloc> |.>>>>|> - > + > |A.8.1.1Version 1 - Prototype |.>>>>|> - > + > |A.8.2|language||ext_allocator_free> |.>>>>|> - > - - |A.8.2.1Version 1 - Prototype - |.>>>>|> - > - - A.9Logging |.>>>>|> - - - |A.9.1|language||ext_logging_log> - |.>>>>|> > - |A.9.1.1Version 1 - Prototype + |A.8.2.1Version 1 - Prototype |.>>>>|> > - A.10Offchain |.>>>>|> + A.9Logging |.>>>>|> - |A.10.1offchain_is_validator - |.>>>>|> - > - - |A.10.1.1Version 1 - Prototype + |A.9.1|language||ext_logging_log> |.>>>>|> > - |A.10.2offchain_submit_transaction + |A.9.1.1Version 1 - Prototype |.>>>>|> > - - |A.10.2.1Version 1 - Prototype - |.>>>>|> - > - - |A.10.3offchain_network_state - |.>>>>|> - > - - |A.10.3.1Version 1 - Prototype - |.>>>>|> - > - - |A.10.4offchain_timestamp - |.>>>>|> - > - - |A.10.4.1Version 1 - Prototype - |.>>>>|> - > - - |A.10.5offchain_sleep_until - |.>>>>|> - > - - |A.10.5.1Version 1 - Prototype - |.>>>>|> - > - - |A.10.6offchain_random_seed - |.>>>>|> - > - - |A.10.6.1Version 1 - Prototype - |.>>>>|> - > - - |A.10.7offchain_local_storage_set - |.>>>>|> - > \ No newline at end of file diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 70ea581c8..d2c5c151a 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -359,7 +359,9 @@ considered good behavior to implement a mechanism which only sends a transaction once to each peer and avoids sending duplicates. Such a mechanism is implementation specific and any absence of such a mechanism - can result in consequences which are undefined. + can result in consequences which are undefined. The Polkadot Host much be + able to handle duplicate transactions received from its peers in order to + prevent the inclusion of duplicate transactions in multiple blocks. The transactions message is represented by > and is defined as follows: @@ -428,7 +430,7 @@ signed extrinsic, but it's the responsiblity of the Host to gossip that extrinsic to the rest of the network. When the Runtime decides to create the heartbeat, it will call the - Host API as described in Section . + Host API as described in Section . The process of gossiping extrinsics is defined in section . @@ -459,8 +461,6 @@ > > > - > - > > > > @@ -555,13 +555,17 @@ |.>>>>|> > - |1.8Gossiping + |1.7.3Transactions |.>>>>|> > - |1.9I'm Online Heartbeat + |1.7.4Consensus Message |.>>>>|> > + + |1.8I'm Online Heartbeat + |.>>>>|> + > \ No newline at end of file From 499eb16bb6e31b86f041cc00c81f00c533b09de6 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 22:37:15 +0100 Subject: [PATCH 30/65] remove deprecated network messages section --- host-spec/ad-netmessages.tm | 400 ------------------------------------ 1 file changed, 400 deletions(-) delete mode 100644 host-spec/ad-netmessages.tm diff --git a/host-spec/ad-netmessages.tm b/host-spec/ad-netmessages.tm deleted file mode 100644 index 5b686c0fa..000000000 --- a/host-spec/ad-netmessages.tm +++ /dev/null @@ -1,400 +0,0 @@ - - - - - - -<\body> - - - In this section, we will specify various types of messages which the - Polkadot Host receives from the network. Furthermore, we also explain the - appropriate responses to those messages. - - <\definition> - A is a byte array, > of length - |M|\<\|\|\>>> such that: - - \; - - <\equation*> - >|>|\M|M|\<\|\|\>>>>|>>>>> - - - \; - - - The body of each message consists of different components based on its - type. The different possible message types are listed below in Table - . We describe the sub-components of each - message type individually in Section . - - |||||||||||||>>||>|||>>|||>>|||>>|||>>|||>|||>>|||>|||>|||>|||>|||>|||>|||>|||>|||>|||>|||>>>>|List of possible network - message types.> - - - - This section disucsses the detailed structure of each network message. - - - - A Message represented by > is sent after a - connection with a neighbouring node is established and has the following - structure: - - <\equation*> - M\Enc,Hash,Hash,C|)> - - - Where: - - <\center> - ||||||:>||>|:>>||>|:>||>|>:>||>|>>||>>>|>>||>>>|>>||>>>> - - - \; - - In which, Role is a bitmap value whose bits represent different roles for - the sender node as specified in Table : - - \; - - <\with|par-mode|center> - <\small-table> - \; - - <\center> - ||||||||||||||||||>|>||>|||>|||>|||>>>> - - - \; - Node role representation in the - status message.> - - - - - A Block request message, represented by >, is sent to - request block data for a range of blocks from a peer and has the following - structure: - - <\equation*> - M\Enc,S,Hash,d,Max|)> - - - where: - - <\center> - ||||||:>||>|>:>||>|>:>||,64bit - integer>}>>|>>||> optional - type>>|>||>|>||>>>> - - - \; - - \; - - in which\ - - <\itemize-minus> - >, the requested data, is a bitmap value, whose bits - represent the part of the block data requested, as explained in Table - : - - - <\with|par-mode|center> - <\small-table> - \; - - <\center> - ||||||||||||||||||>|>||>|||>|||>|||>|||>>>> - - - \; - Bit values for block attribute - >, to indicate the requested parts of the data.> - - - <\itemize-minus> - > is SCALE encoded varying data type (see Definition - ) of either > - representing the block hash, >, or integer - representing the block number of the starting block of the requested - range of blocks. - - > is optionally the block hash of the last block - in the range. - - is a flag; it defines the direction on the block chain - where the block range should be considered (starting with the starting - block), as follows - - <\equation*> - d=|>||>>>>|\> - - - - Optional data type is defined in Definition - . - - - - A represented by > is sent in a - response to a requested block message (see Section - ). It has the following structure: - - <\equation*> - M\Enc - - - where: - - <\center> - ||||||:>||>|:>||>>>> - - - \; - - \; - - In which block data is defined in Definition . - - <\definition> - is defined as the follownig - tuple: - - - <\equation*> - ,Header,Body,Receipt,MessageQueue,Justification|)> - - - Whose elements, with the exception of >, are all of the - following (see Definition - ) and are defined as follows: - - <\center> - ||||||>:>||>>>|>:>||)>>|||)>>|||>|||>|||>>>> - - - \; - - - - A represented by > is sent when - a node becomes aware of a new complete block on the network and has the - following structure: - - <\equation*> - M\Enc|)> - - - Where: - - <\center> - ||||||>:>||)>>>>> - - - - - \ \ \ \ The transactions Message is represented by > and is - defined as follows: - - <\equation*> - M\Enc,\,C|)> - - - in which: - - <\equation*> - C\Enc|)> - - - Where each > is a byte array and represents a sepearate - extrinsic. The Polkadot Host is indifferent about the content of an - extrinsic and treats it as a blob of data. - - - - A represented by > is sent to - communicate messages related to consensus process: - - <\equation*> - M\Enc,D|)> - - - Where: - - <\center> - ||||||>:>||>>>|>||>>>>>> - - - \; - - in which - - <\equation*> - E\BABE>||>>|FRNK>||>>>>>|\> - - - \; - - The network agent should hand over to approperiate consensus - engine which identified by >. - - - - - -\; - - <\with|par-mode|right> - - - - \; - - - -<\initial> - <\collection> - - - - - - - -<\references> - <\collection> - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - - - -<\auxiliary> - <\collection> - <\associate|table> - |A.1>||List of possible - network message types.>|> - - |A.2>||Node role - representation in the status message.>|> - - |A.3>||Bit values for - block attribute |A>, to indicate the - requested parts of the data.>|> - - <\associate|toc> - |math-font-series||Appendix - ANetwork Messages> |.>>>>|> - - - A.1Detailed Message Structure - |.>>>>|> - - - |A.1.1Status Message - |.>>>>|> - > - - |A.1.2Block Request Message - |.>>>>|> - > - - |A.1.3Block Response Message - |.>>>>|> - > - - |A.1.4Block Announce Message - |.>>>>|> - > - - |A.1.5Transactions - |.>>>>|> - > - - |A.1.6Consensus Message - |.>>>>|> - > - - - \ No newline at end of file From 42c91a7ade4748e7bd345735d23ea6d8ba49dbbc Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Tue, 15 Dec 2020 22:43:26 +0100 Subject: [PATCH 31/65] fix typo --- host-spec/c04-networking.tm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index d2c5c151a..f907bb3f6 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -154,8 +154,8 @@ After these three steps, both the initiator and responder derive a new - sharedsecret using the static and session-defined Noise keys, - which is used to encrypt all further communication. The Noise specification + shared secret using the static and session-defined Noise keys, which are + used to encrypt all further communication. The Noise specification describes this process in detail. From 0dcc60feeae571d87817ac001f44c50df24e8f5c Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Wed, 16 Dec 2020 15:52:18 +0100 Subject: [PATCH 32/65] append grandpa and yamux protocols --- host-spec/c04-networking.tm | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index f907bb3f6..2832a63b3 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -158,7 +158,7 @@ used to encrypt all further communication. The Noise specification describes this process in detail. - + After the node establishes a connection with a peer, the use of multiplexing allows the Polkadot Host to open substreams. @@ -170,11 +170,14 @@ The Polkadot Host adoptes the following substreams: <\itemize> - - Open a substream for the Noise protocol to - establish a encryption layer. + - a protol that is announced when a connection to + a peer is established. - - Open a substream for handshakes to - negotiate a new protocol. + - a protocol that is announced when + negotiating an encryption protocol or a substream. + + - a protocol used during the + negotiation. - Open a substream to a peer and initialize a ping to verify if a connection is till alive. If the peer @@ -183,7 +186,7 @@ - Open a substream to a peer to ask information about that peer. - - Open a substream for Kademlia + - Open a substream for Kademlia requests. @@ -194,11 +197,14 @@ - a request and response protocol that allows a light client to perform request information about the state. - - a notification protocol which - sends transactions to connected peers. + - a substream/notification protocol + which sends transactions to connected peers. + + - a substream/notification + protocol which sends blocks to connected peers. - - a notification protocol which - sends blocks to connected peers. + - a substream/notification + protocol which sends Grandpa votes to connected peers. From 0837dbb34cb3cbf5728059a0cab1e770d106f796 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Wed, 16 Dec 2020 16:20:03 +0100 Subject: [PATCH 33/65] mention substream protocol in consensus messages section --- host-spec/c04-networking.tm | 3 +++ 1 file changed, 3 insertions(+) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 2832a63b3..0c7d14143 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -423,6 +423,9 @@ The network agent should hand over to approperiate consensus engine which identified by >. + The exchange of consensus messages is conducted on the + substream + The I'm Online heartbeat is a crucial part of the Polkadot validation From 379ec8f86f5f4ff8955f2f675e46526f52055450 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Wed, 16 Dec 2020 16:37:29 +0100 Subject: [PATCH 34/65] rewrite grandpa vote networking section --- host-spec/c04-networking.tm | 55 ++++++++++--------------------------- 1 file changed, 15 insertions(+), 40 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 0c7d14143..02ce049e4 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -17,9 +17,9 @@ authority or entity in order to achieve a its fullest potential of provided functionality. Each node with the network can authenticate itself and its peers by using cryptographic keys, including establishing fully encrypted - connections. The networking protocol is based on the open and standardized - protocol, including the usage of the distributed Kademlia - hash table for peer discovery. + connections. The networking protocol is based on a family of open + protocols, including the usage of the distributed Kademlia hash table for + peer discovery. @@ -74,10 +74,12 @@ Kademlia requests - Kademlia is a distributed hash table for decentralized networks and supports requests, where - nodes respond with their list of available peers. + nodes respond with their list of available peers. Kademlia requests must + also contain protocol identifiers as described in Section + . - + Kademlia nodes only communicate with other nodes using the same protocol identifier. The Polkadot network is identified by @@ -389,42 +391,11 @@ The exchange of transactions is conducted on the substream. - + - A represented by > is sent to - communicate messages related to consensus process: - - <\equation*> - M\Enc,D|)> - - - Where: - - <\center> - ||||||>:>||>>>|>||>>>>>> - - - \; - - in which - - <\equation*> - E\BABE>||>>|FRNK>||>>>>>|\> - - - \; - - The network agent should hand over to approperiate consensus - engine which identified by >. - - The exchange of consensus messages is conducted on the - substream + The exchange of GRANDPA messages is conducted on the + substream. The process for the creation of + such votes is described in Section . @@ -479,8 +450,12 @@ > > > + > + > > + > > + > From b27d920e5a99cd1c7b513430bc301c3dcc371c9c Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Wed, 16 Dec 2020 17:15:13 +0100 Subject: [PATCH 35/65] fix typos --- host-spec/c04-networking.tm | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 02ce049e4..eb447872a 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -97,19 +97,19 @@ <\itemize-dot> . After the protocol is - negotiated, the opening node sends a single message containing a request. - The remote node then sends a response, after which the substream is then + negotiated, the initiator sends a single message containing a request. + The responder then sends a response, after which the substream is then immediately closed. . After the protocol is negotiated, - the opening node sends a single hand shake message. The remote node can - then either accept or reject the substream. After the substream has been - accepted, the opening node can send an unbound number of individual + the initiator sends a single hand shake message. The responder can then + either accept or reject the substream. After the substream has been + accepted, the initiator can send an unbound number of individual messages. The Polkadot Host can establish a connection with any peer it knows the - address. The Polkadot Host supports multiple base-layer protocols: + address of. The Polkadot Host supports multiple base-layer protocols: <\itemize> TCP/IP - addresses in the form of From 4eec0968a629bd2c4d9b9f92b1d56b44e54cbf9a Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Wed, 16 Dec 2020 17:18:38 +0100 Subject: [PATCH 36/65] adjust sentencing --- host-spec/c04-networking.tm | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index eb447872a..d961e396a 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -212,15 +212,14 @@ The Polkadot Host must actively communicate with the network in order to - participate in the validation process. This section describes the expected - behaviors of the node. + participate in the validation process or act as a full node. - When the node creates a new block, it must be announced to the network. - Other nodes within the network will track this announcement and can request - information about this block. The mechanism for tracking announements and - requesting the required data is implementation specific. + When the node creates or receives a new block, it must be announced to the + network. Other nodes within the network will track this announcement and + can request information about this block. The mechanism for tracking + announcement and requesting the required data is implementation specific. Block announcements and requests are conducted on the substream. From d21a6a85054b85086371ef260823e4702f19a28c Mon Sep 17 00:00:00 2001 From: Skalman Date: Wed, 13 Jan 2021 05:47:26 -0500 Subject: [PATCH 37/65] Edit and review `c04-networking.tm` in the documents. --- host-spec/c04-networking.tm | 368 +++++++++++++++++++++++------------- 1 file changed, 234 insertions(+), 134 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index d961e396a..b5bdf14e8 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -1,4 +1,4 @@ - + @@ -9,27 +9,39 @@ This document in its current form is incomplete and considered work in progress. Any reports regarding - falseness or clarifications are appreciated. + deviation from the current Polkadot network protocol or clarifications are + appreciated. The Polkadot network is decentralized and does not rely on any central - authority or entity in order to achieve a its fullest potential of provided - functionality. Each node with the network can authenticate itself and its - peers by using cryptographic keys, including establishing fully encrypted - connections. The networking protocol is based on a family of open - protocols, including the usage of the distributed Kademlia hash table for - peer discovery. + authority or entity in order to achieve its fullest potential of provided + functionality.\ + + The networking protocol is based on a family of open protocols, including + the distributed Kademlia hash table which is used for peer discovery. + + The nodes which constitute the network can authenticate to their peers + using cryptographic keys and subsequently establishing fully encrypted + connections. a sentence for each of the 9 + sections> + + The completeness of implementing the Polkadot networking protocol requires - the usage of external documentation. + the usage of external documentation. <\itemize> - + : is a + distributed hash table for decentralized networks\ @@ -38,17 +50,31 @@ + > + + <\todo> + libp2p address format + + + + Each Polkadot Host node maintains a ED25519 key pair which is used to - identify the node. The public key is shared with the rest of the network. - This allows nodes to establish secure communication channels. Nodes are - discovered as described in the Discovery Mechanism Section - (), where each node can be identified - by their corresponding (). + identify the node. The public key is shared with the rest of the network + which allows nodes to establish secure communication channels.\ Each node must have its own unique ED25519 key pair. Using the same pair - among two or more nodes is interpreted as bad behavior. + among two or more nodes is prohibited by the protocol interpreted as bad + behavior . + + Furthermore, the node's as defined in Definition + is derived from its public key. + () is used to identify each node when they are + discoved in the course of the discovery mechanism described in Section + . <\definition> The Polkadot node's , formally @@ -61,22 +87,22 @@ The Polkadot Host uses various mechanisms to find peers within the network, to establish and maintain a list of peers and to share that list with other - peers from the network. + peers from the network, as follows: <\itemize> - Bootstrap nodes - hard-coded node identities and addresses provided - by the genesis state specification as described in Appendix - . - - mDNS - performs a broadcast to the local network. Nodes that might - be listing can respond the the broadcast. The libp2p mDNS specification - defines this process in more detail (). - - Kademlia requests - Kademlia is a distributed hash table for - decentralized networks and supports requests, where - nodes respond with their list of available peers. Kademlia requests must - also contain protocol identifiers as described in Section - . + are hard-coded node identities and + addresses provided by the genesis state specification as described in + Appendix . + + protocol which performs a broadcast to the local + network. Nodes that might be listing can respond the the broadcast. The + libp2p mDNS specification defines this process in more detail (). + + invoking Kademlia + requests, where nodes respond with their list of available peers. + Kademlia requests must also contain protocol identifiers as described in + Section . @@ -89,76 +115,98 @@ Polkadot nodes connect to peers by establishing a TCP connection. Once established, the node initiates a handshake with the remote peer's - encryption layer. An additional layer, know as the multiplexing layer, - allows a connection to be split into substreams, either by the local or - remote node. - - The Polkadot node supports two kind of substream protocols: + encryption layer . An + additional layer , known as the multiplexing layer, allows a + connection to be split into substreams , either by the local + or remote node. + + The Polkadot node supports two types of substream protocols: <\itemize-dot> - . After the protocol is + : After the protocol is negotiated, the initiator sends a single message containing a request. The responder then sends a response, after which the substream is then immediately closed. . After the protocol is negotiated, - the initiator sends a single hand shake message. The responder can then + the initiator sends a single handshake message. The responder can then either accept or reject the substream. After the substream has been accepted, the initiator can send an unbound number of individual messages. The Polkadot Host can establish a connection with any peer it knows the - address of. The Polkadot Host supports multiple base-layer protocols: + address ofof\<#2019\>. A preposition is a very bad word to end a sentence + with.>. The Polkadot Host supports multiple base-layer protocols: <\itemize> - TCP/IP - addresses in the form of - establish a TCP connection and negotiate a encryption and multiplexing - layer. - - Websockets - addresses in the form of - establish a TCP connection and negotiate the Websocket protocol within - the connection. Additionally, a encryption and multiplexing layer is - negotiated within the Websocket connection. - - DNS - addresses in form of and + with addresses in the form of + establish a TCP connection and negotiate an encryption and + multiplexing layer. + + with addresses in the form of + establish a TCP connection and + negotiate the Websocket protocol within the connection. Additionally, an + encryption and multiplexing layer is negotiated within the Websocket + connection. + + addresses in form of + and . After a base-layer protocol is established, the Polkadot Host will apply - the Noise protocol. + the Noise protocol - + > - The Noise protocol is a framework for bulding encryption protocols. - utilizes that protocol for establishing encrypted - communication channels. + Polkadot protocol uses the > Noise framework to + build an encryption protocol. The Noise protocol is a framework for bulding + encryption protocols. utilizes that protocol for + establishing encrypted communication channels. Polkadot nodes use the XX handshake pattern - () to establish a connection - between peers. Three steps are required to successfully complete the - handshake process: + () + to establish a connection between peers. Three following steps are required + to successfully complete the handshake process: <\enumerate-numeric> The initiator generates a keypair and sends the public key to the - responder. + responder. The responder generates its own keypair and sends its public key back to the initiator. After that, the responder derives a shared secret and uses it to encrypt all further communication. The responder now sends - its static Noise public key (which is non-persistant and generated on - every node startup), its public key and a signature of - the static Noise public key signed with the public key. + its static Noise public key (which is non-persistent and + generated on every node startup), its public key and a + signature of the static Noise public key signed with the + public key. The initiator derives a shared secret and uses it to encrypt all further communication. It also sends its static Noise public key, public key and a signature to the responder. + + After these three steps, both the initiator and responder derive a new shared secret using the static and session-defined Noise keys, which are used to encrypt all further communication. The Noise specification - describes this process in detail. + describes this process in detail. @@ -169,14 +217,15 @@ , where each protocol servers a specific utility. - The Polkadot Host adoptes the following substreams: + The Polkadot Host adoptes adopt\<#2019\> mean + here?> the following substreams: <\itemize> - a protol that is announced when a connection to - a peer is established. + a peer is established. - a protocol that is announced when - negotiating an encryption protocol or a substream. + negotiating an encryption protocol or a substream. - a protocol used during the negotiation. @@ -206,7 +255,7 @@ protocol which sends blocks to connected peers. - a substream/notification - protocol which sends Grandpa votes to connected peers. + protocol which sends GRANDPA votes to connected peers. @@ -214,6 +263,14 @@ The Polkadot Host must actively communicate with the network in order to participate in the validation process or act as a full node. + + + + + + When the node creates or receives a new block, it must be announced to the @@ -222,12 +279,15 @@ announcement and requesting the required data is implementation specific. Block announcements and requests are conducted on the - substream. + substream conducted\<#2019\> and refe to def 2> <\definition> The initializes a substream to a remote - peer. Once established, all messages created by - the node are sent to that substream. + peer. Once established, all messages created by + the node are sent to that substream. The is a SCALE encoded structure of the following format: @@ -271,13 +331,14 @@ - + Block requests can be used to retrieve a range of blocks from peers. <\definition> - The message is a Protobuf serialized structure of - the following format: + The message is a Protobuf serialized + structure of the following + format: <\big-table||||||>|>|>|>>||||>>>||| is a Protobuf structure indicating the sequence direction of the requested blocks. The structure is a varying - data type of the following format: + data type of the following format: <\big-table||||||||||>|>>||>|| - Transactions are sent directly in its full form to connected peers. It's - considered good behavior to implement a mechanism which only sends a + Transactions as defined and explained in are sent directly in its full form to + connected peers. It is considered good + behavior to implement a mechanism which only sends a transaction once to each peer and avoids sending duplicates. Such a mechanism is implementation specific and any absence of such a mechanism - can result in consequences which are undefined. The Polkadot Host much be - able to handle duplicate transactions received from its peers in order to - prevent the inclusion of duplicate transactions in multiple blocks. + can result in consequences which are undefined . On the othrer hand, the Polkadot host must be able to + handle duplicate transactions received from its peers and prevents the + inclusion of duplicate copies of a same transaction in + - The transactions message is represented by > and is defined - as follows: + The transactions message is represented + by > and is defined as follows: <\equation*> M\Enc,\,C|)> @@ -384,35 +452,58 @@ Where each > is a byte array and represents a sepearate - extrinsic. The Polkadot Host is indifferent about the content of an - extrinsic and treats it as a blob of data. + extrinsic. The Polkadot Host is + indifferent about the content of an extrinsic and treats it + as a blob of data. The exchange of transactions is conducted on the substream. - + The exchange of GRANDPA messages is conducted on the substream. The process for the creation of such votes is described in Section . - + ?> Heartbeat> + + + + The I'm Online heartbeat is a crucial part of the Polkadot validation - process, as it signals the active participation of validators and confirms - their reachability. The Polkadot network punishes unreachable validators - which have been elected to an authority by slashing their bonded funds. - This is achieved by requiring validators to issue an I'm Online heartbeat, - which comes in the form of a signed extrinsic, on the start of every Era. - - The Polkadot Runtime fully manages the creation and the timing of that - signed extrinsic, but it's the responsiblity of the Host to gossip that + process, as it signals the active participation of the Polkadot validator + node and confirms its reachability. The Polkadot network punishes + unreachable validators which have been elected to an authority set by + slashing their bonded funds . This is achieved by requiring validators to + issue a signed extrinsic known as I'm Online heartbeat, on the + start of every Era . + + The Polkadot Runtime fully manages the creation and the timing of the + extrinsic, but it's the responsiblity of the Polkadot host to gossip that extrinsic to the rest of the network. When the Runtime decides to create - the heartbeat, it will call the - Host API as described in Section . + and propagate the heartbeat extrinsic, it calls the + Host API function as described + in Section . The process of gossiping extrinsics is defined in section . + + \; + + <\with|par-mode|right> + + + + \; <\initial> @@ -428,33 +519,31 @@ <\collection> > > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > > - > - > + > + > + > > > > - > - > + > + > > > > - > - > - > - > - > - > - > + > + > + > + > @@ -464,30 +553,30 @@ |1>|> |language||BlockRequest> Protobuf message. - |> + |> |2>|> Bits of block data to be requested. - |> + |> |3>|> Protobuf message indicating the block to start from. - |> + |> |4>|> |language||Direction> Protobuf structure. - |> + |> |5>|> |language||BlockResponse> Protobuf message. - |> + |> |6>|> |math-font-series||BlockData> Protobuf structure. - |> + |> <\associate|toc> |math-font-series||font-size||4Networking> @@ -502,53 +591,64 @@ |.>>>>|> > - |1.2Node Identities - |.>>>>|> + |1.2|||||||||||||>>>>>||||>>|[libp2p + convention perhaps]>> |.>>>>|> > - |1.3Discovery mechanism + |1.3Node Identities |.>>>>|> > - |1.3.1Protocol Identifier + |1.4Discovery mechanism |.>>>>|> > - |1.4Connection establishment + |1.4.1Protocol Identifier |.>>>>|> > - |1.5Noise Protocol + |1.5Connection establishment |.>>>>|> > - |1.6Substreams - |.>>>>|> + |1.6Noise Protocol + |||||||||||||>>>>>||||>>|[encryption + protocol/layer is a better name]>> |.>>>>|> > - |1.7Network Messages + |1.7Protocols and Substreams |.>>>>|> > - |1.7.1Announcing blocks + |1.8Network Messages |.>>>>|> > - |1.7.2Requesting blocks + |1.8.1Announcing blocks |.>>>>|> > - |1.7.3Transactions + |1.8.2Requesting blocks |.>>>>|> - > + > - |1.7.4Consensus Message + |1.8.3Transactions |.>>>>|> > - |1.8I'm Online Heartbeat + |1.8.4GRANDPA Votes |.>>>>|> > + + |1.9I'mOnline + |||||||||||||>>>>>||||>>|[perhaps + |language||ImOnline>?]>> + Heartbeat |.>>>>|> + > \ No newline at end of file From 38970f4fcef17ff8fa8a2ebe96cdaa2223af4e39 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Wed, 27 Jan 2021 14:25:43 +0100 Subject: [PATCH 38/65] fix some open points, define behavior when multiple nodes use the same key --- host-spec/c04-networking.tm | 43 ++++++++++++++----------------------- 1 file changed, 16 insertions(+), 27 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index b5bdf14e8..1444545db 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -1,4 +1,4 @@ - + @@ -16,17 +16,9 @@ The Polkadot network is decentralized and does not rely on any central authority or entity in order to achieve its fullest potential of provided - functionality.\ - - The networking protocol is based on a family of open protocols, including - the distributed Kademlia hash table which is used for peer discovery. - - The nodes which constitute the network can authenticate to their peers - using cryptographic keys and subsequently establishing fully encrypted - connections. a sentence for each of the 9 - sections> + functionality. The networking protocol is based on a family of open + protocols, including the distributed Kademlia hash table which is used for + peer discovery. @@ -64,12 +56,9 @@ identify the node. The public key is shared with the rest of the network which allows nodes to establish secure communication channels.\ - Each node must have its own unique ED25519 key pair. Using the same pair - among two or more nodes is prohibited by the protocol interpreted as bad - behavior . - + Each node must have its own unique ED25519 key pair. When two or more nodes + use the same key, the network will interpret those nodes as a single node, + which will result in undefined behavior and can result in equivocation. Furthermore, the node's as defined in Definition is derived from its public key. () is used to identify each node when they are @@ -147,15 +136,15 @@ protocols over ip?> protocols: <\itemize> - with addresses in the form of - establish a TCP connection and negotiate an encryption and + with addresses in the form of + to establish a TCP connection and negotiate an encryption and multiplexing layer. with addresses in the form of - establish a TCP connection and - negotiate the Websocket protocol within the connection. Additionally, an - encryption and multiplexing layer is negotiated within the Websocket + to establish a TCP connection + and negotiate the Websocket protocol within the connection. Additionally, + an encryption and multiplexing layer is negotiated within the Websocket connection. addresses in form of @@ -593,7 +582,7 @@ |1.2|||||||||||||>>>>>||||>>|[libp2p + red>|||||>>>>>||||>>|[libp2p convention perhaps]>> |.>>>>|> > @@ -615,7 +604,7 @@ |1.6Noise Protocol |||||||||||||>>>>>||||>>|[encryption + red>|||||>>>>>||||>>|[encryption protocol/layer is a better name]>> |.>>>>|> > @@ -645,7 +634,7 @@ |1.9I'mOnline |||||||||||||>>>>>||||>>|[perhaps + red>|||||>>>>>||||>>|[perhaps |language||ImOnline>?]>> Heartbeat |.>>>>|> > From 3c351e3795c56d103ee5c176ae5bb7001c11d26b Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Wed, 27 Jan 2021 16:22:31 +0100 Subject: [PATCH 39/65] mention that mDNS is optional --- host-spec/c04-networking.tm | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 1444545db..f771b8dbc 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -84,9 +84,12 @@ Appendix . protocol which performs a broadcast to the local - network. Nodes that might be listing can respond the the broadcast. The + network. Nodes that might be listening can respond the the broadcast. The libp2p mDNS specification defines this process in more detail (). + This protocol is an optional implementation detail for Polkadot Host + implementers and is not required in order to participate in the Polkadot + network. invoking Kademlia requests, where nodes respond with their list of available peers. From 43cafad068982b057f957391a0f0fa6e467b7805 Mon Sep 17 00:00:00 2001 From: Fabio Lama Date: Wed, 27 Jan 2021 16:22:09 +0100 Subject: [PATCH 40/65] Update host-spec/c04-networking.tm update id for node roles Co-authored-by: Pierre Krieger --- host-spec/c04-networking.tm | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index f771b8dbc..7bb98f832 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -291,9 +291,9 @@ where: <\eqnarray*> - |||>>||>>||||>>||>>||>>>>>>>|>||>>|>||>>|>||> - \ No newline at end of file + From 0f9258a525a0cfc56e556d3fed7578c62d92eeda Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Wed, 27 Jan 2021 16:29:41 +0100 Subject: [PATCH 41/65] mention LEB128 encoded length prefix for substream requests and responses --- host-spec/c04-networking.tm | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 7bb98f832..9b1965800 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -123,7 +123,8 @@ : After the protocol is negotiated, the initiator sends a single message containing a request. The responder then sends a response, after which the substream is then - immediately closed. + immediately closed. The requests and responses are prefixed with their + encoded length. . After the protocol is negotiated, the initiator sends a single handshake message. The responder can then @@ -643,4 +644,4 @@ > - + \ No newline at end of file From e773b55fc20c7d2f56a7c43b9b2100f2767df6df Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Wed, 27 Jan 2021 16:33:05 +0100 Subject: [PATCH 42/65] mention LEB128 prefix encoding for handshakes --- host-spec/c04-networking.tm | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 9b1965800..b597db529 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -127,10 +127,12 @@ encoded length. . After the protocol is negotiated, - the initiator sends a single handshake message. The responder can then - either accept or reject the substream. After the substream has been - accepted, the initiator can send an unbound number of individual - messages. + the initiator sends a single handshake message. Handshakes are prefixed + with their encoded + length. A handshake can be empty, in which case the length prefix would + be . The responder can then either accept or reject the + substream. After the substream has been accepted, the initiator can send + an unbound number of individual messages. The Polkadot Host can establish a connection with any peer it knows the From d57e6f8529a2b6faac77ab4863c569b39eb1fb4c Mon Sep 17 00:00:00 2001 From: Skalman Date: Wed, 27 Jan 2021 10:26:38 -0500 Subject: [PATCH 43/65] Appendix E Review, comments are inline --- host-spec/ae-hostapi.tm | 410 ++++++++++++++++++++-------------------- 1 file changed, 210 insertions(+), 200 deletions(-) diff --git a/host-spec/ae-hostapi.tm b/host-spec/ae-hostapi.tm index 59c908648..78a961ecf 100644 --- a/host-spec/ae-hostapi.tm +++ b/host-spec/ae-hostapi.tm @@ -1,4 +1,4 @@ - + @@ -758,7 +758,7 @@ containing the BIP-39 seed which must be valid UTF8. : a regular pointer to the buffer containing the - public key. + 256-bit public key. > @@ -982,7 +982,7 @@ verified. : a regular pointer to the buffer containing the - public key. + 256-bit public key. : a i32 integer value equal to if the signature is valid or a value equal to if otherwise. @@ -1009,7 +1009,7 @@ verified. : a regular pointer to the buffer containing the - public key. + 256-bit public key. : a i32 integer value equal to if the signature is valid or a value equal to if otherwise. @@ -1181,7 +1181,7 @@ . : a regular pointer to the buffer containing the - Blake2 hash of the message. + 256-bit Blake2 hash of the message. : a pointer-size as defined in Definition indicating the SCALE encoded @@ -1554,7 +1554,8 @@ Given a SCALE encoded extrinsic, this function submits the extrinsic to the Host's transaction pool, ready to be propagated to remote peers. This - process is critical for issuing the I'm Online message. + process is critical for issuing the message + . @@ -1679,8 +1680,9 @@ : <\itemize> - : a regular pointer to the buffer containing the - 256-bit seed. + : a regular pointer to the buffer containing the 256-bit + seed. > @@ -2037,8 +2039,9 @@ > Set the authorized nodes which are allowed to connect to the local node. - This function is primarily used for private blockchains and is not - necessarily required for the public and open Polkadot protocol. + This function is primarily used for private blockchains and is not necessarily + required for the public and open Polkadot protocol. @@ -2385,209 +2388,216 @@ : a pointer-size as defined in Definition indicating the log message. + + \; + + <\with|par-mode|right> + + \ <\initial> <\collection> - + + - - + + <\references> <\collection> - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > From 8bf9e16b80e6d9f98f6bfc630433f0d1743b9d4f Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Wed, 27 Jan 2021 17:18:38 +0100 Subject: [PATCH 44/65] unify LEB128 description for handshakes and messages --- host-spec/c04-networking.tm | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index b597db529..05ea3d31e 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -127,12 +127,15 @@ encoded length. . After the protocol is negotiated, - the initiator sends a single handshake message. Handshakes are prefixed - with their encoded - length. A handshake can be empty, in which case the length prefix would - be . The responder can then either accept or reject the - substream. After the substream has been accepted, the initiator can send - an unbound number of individual messages. + the initiator sends a single handshake message. The responder can then + either accept the substream by sending its own handshake, or reject it by + closing the substream. After the substream has been accepted, the + initiator can send an unbound number of individual messages. + + Handshakes and messages are prefixed with their + encoded lengths. A + handshake can be empty, in which case the length prefix would be + . The Polkadot Host can establish a connection with any peer it knows the @@ -527,6 +530,8 @@ > > > + > + > > > > From 5896fb914aa5c9dd9645482e7309e81b5323c075 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Thu, 28 Jan 2021 14:38:06 +0100 Subject: [PATCH 45/65] remove Receipt and Message Queue from BlockRequest --- host-spec/c04-networking.tm | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 05ea3d31e..e935895b1 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -357,9 +357,8 @@ is (17). <\big-table||||||>|>>||>||>||>||>||>>>>> + 0001>>||>||>>>>> Bits of block data to be requested. @@ -530,8 +529,6 @@ > > > - > - > > > > From 420cc0ef183bf54b8705d77e910bfb01e9367e54 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Thu, 28 Jan 2021 14:43:03 +0100 Subject: [PATCH 46/65] mention rename of /paritytech/grandpa/1 --- host-spec/c04-networking.tm | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index e935895b1..1fa0f6aa4 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -253,7 +253,9 @@ protocol which sends blocks to connected peers. - a substream/notification - protocol which sends GRANDPA votes to connected peers. + protocol which sends GRANDPA votes to connected peers. .> From 69b2e8417c1174f052675927894b19cf5ffa245c Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Thu, 28 Jan 2021 14:49:32 +0100 Subject: [PATCH 47/65] refer to the libp2p secure handshake pattern in Noise protocol section --- host-spec/c04-networking.tm | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 1fa0f6aa4..36c2d9a47 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -171,12 +171,14 @@ Polkadot protocol uses the > Noise framework to build an encryption protocol. The Noise protocol is a framework for bulding encryption protocols. utilizes that protocol for - establishing encrypted communication channels. - - Polkadot nodes use the XX handshake pattern - () - to establish a connection between peers. Three following steps are required - to successfully complete the handshake process: + establishing encrypted communication channels. Refer to the + specification for a detailed description. + + Polkadot nodes use the to establish a connection + between peers. Three following steps are required to successfully complete + the handshake process: <\enumerate-numeric> The initiator generates a keypair and sends the public key to the From 4aa899964cec9d19bfd395880121029c14f40577 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Thu, 28 Jan 2021 14:57:50 +0100 Subject: [PATCH 48/65] clarify mplex and yamux for protocol negotiation --- host-spec/c04-networking.tm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 36c2d9a47..8319bea1b 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -212,7 +212,8 @@ After the node establishes a connection with a peer, the use of multiplexing allows the Polkadot Host to open substreams. - uses the protocol () + uses the protocol|https://docs.libp2p.io/concepts/stream-multiplexing/#mplex> + or the protocol|https://docs.libp2p.io/concepts/stream-multiplexing/#yamux> to manage substream and to allow the negotiation of , where each protocol servers a specific utility. From da5d2e4f4d67cb9431dceef23a927d252510d8d0 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Thu, 28 Jan 2021 18:03:24 +0100 Subject: [PATCH 49/65] separate multiplexing protocols from substreams --- host-spec/c04-networking.tm | 35 +++++++++++++++++++++-------------- 1 file changed, 21 insertions(+), 14 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 8319bea1b..efed676ad 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -37,7 +37,9 @@ - + + + @@ -138,6 +140,20 @@ . + Connections are established by using the following protocols: + + <\itemize-dot> + - a protol that is announced when a connection to + a peer is established. + + - a protocol that is announced when + negotiating an encryption protocol or a substream. + + - a protocol used during the + or negotiation. See Section + for more information. + + The Polkadot Host can establish a connection with any peer it knows the address ofof\<#2019\>. A preposition is a very bad word to end a sentence @@ -146,9 +162,8 @@ <\itemize> with addresses in the form of - to establish a TCP connection and negotiate an encryption and - multiplexing layer. + to establish a TCP connection and negotiate + an encryption and a multiplexing layer. with addresses in the form of to establish a TCP connection @@ -208,7 +223,7 @@ used to encrypt all further communication. The Noise specification describes this process in detail. - + After the node establishes a connection with a peer, the use of multiplexing allows the Polkadot Host to open substreams. @@ -222,15 +237,6 @@ here?> the following substreams: <\itemize> - - a protol that is announced when a connection to - a peer is established. - - - a protocol that is announced when - negotiating an encryption protocol or a substream. - - - a protocol used during the - negotiation. - - Open a substream to a peer and initialize a ping to verify if a connection is till alive. If the peer does not respond, the connection is dropped. @@ -546,6 +552,7 @@ > > > + > From d6cbf355205f0125b00ef971c83e893b1d05b8c2 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Thu, 28 Jan 2021 18:07:07 +0100 Subject: [PATCH 50/65] remove protocol identifier section --- host-spec/c04-networking.tm | 39 +++++++++++++++---------------------- 1 file changed, 16 insertions(+), 23 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index efed676ad..55f0308ad 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -32,8 +32,7 @@ <\itemize> - : is a - distributed hash table for decentralized networks\ + \ @@ -99,12 +98,6 @@ Section . - - - Kademlia nodes only communicate with other nodes using the same protocol - identifier. The Polkadot network is identified by - (respectively for Kusama). - Polkadot nodes connect to peers by establishing a TCP connection. Once @@ -526,27 +519,27 @@ <\references> <\collection> > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > + > + > > > > > > - > - > - > + > + > + > > > > From c2f7b0b982e07987c57f1e653891e5986d821a5b Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Fri, 29 Jan 2021 14:14:55 +0100 Subject: [PATCH 51/65] mention that the responder of the notification stream keeps its sending side open --- host-spec/c04-networking.tm | 41 +++++++++++++++++-------------------- 1 file changed, 19 insertions(+), 22 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 55f0308ad..638a13cf3 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -125,7 +125,10 @@ the initiator sends a single handshake message. The responder can then either accept the substream by sending its own handshake, or reject it by closing the substream. After the substream has been accepted, the - initiator can send an unbound number of individual messages. + initiator can send an unbound number of individual messages. The + responder keeps its sending side of the substream open, despite not + sending anything anymore, and can later close it in order to signal to + the initiator that it no longer wishes to communicate. Handshakes and messages are prefixed with their encoded lengths. A @@ -532,7 +535,6 @@ > > > - > > > > @@ -544,7 +546,6 @@ > > > - > > @@ -555,30 +556,30 @@ |1>|> |language||BlockRequest> Protobuf message. - |> + |> |2>|> Bits of block data to be requested. - |> + |> |3>|> Protobuf message indicating the block to start from. - |> + |> |4>|> |language||Direction> Protobuf structure. - |> + |> |5>|> |language||BlockResponse> Protobuf message. - |> + |> |6>|> |math-font-series||BlockData> Protobuf structure. - |> + |> <\associate|toc> |math-font-series||font-size||4Networking> @@ -607,50 +608,46 @@ |.>>>>|> > - |1.4.1Protocol Identifier - |.>>>>|> - > - |1.5Connection establishment |.>>>>|> - > + > |1.6Noise Protocol ||||||>>>>>||||>>|[encryption protocol/layer is a better name]>> |.>>>>|> - > + > |1.7Protocols and Substreams |.>>>>|> - > + > |1.8Network Messages |.>>>>|> - > + > |1.8.1Announcing blocks |.>>>>|> - > + > |1.8.2Requesting blocks |.>>>>|> - > + > |1.8.3Transactions |.>>>>|> - > + > |1.8.4GRANDPA Votes |.>>>>|> - > + > |1.9I'mOnline ||||||>>>>>||||>>|[perhaps |language||ImOnline>?]>> Heartbeat |.>>>>|> - > + > \ No newline at end of file From a4cd434a40d141289e3687f703bbcb4121753d5f Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Fri, 29 Jan 2021 15:12:26 +0100 Subject: [PATCH 52/65] clarify open points by reviewer --- host-spec/c04-networking.tm | 54 ++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 31 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 638a13cf3..43bf84ab8 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -85,12 +85,11 @@ Appendix . protocol which performs a broadcast to the local - network. Nodes that might be listening can respond the the broadcast. The - libp2p mDNS specification defines this process in more detail (). - This protocol is an optional implementation detail for Polkadot Host - implementers and is not required in order to participate in the Polkadot - network. + network. Nodes that might be listening can respond the the broadcast. + + defines this process in more detail. This protocol is an optional + implementation detail for Polkadot Host implementers and is not required + in order to participate in the Polkadot network. invoking Kademlia requests, where nodes respond with their list of available peers. @@ -101,18 +100,13 @@ Polkadot nodes connect to peers by establishing a TCP connection. Once - established, the node initiates a handshake with the remote peer's - encryption layer . An - additional layer , known as the multiplexing layer, allows a - connection to be split into substreams , either by the local - or remote node. - - The Polkadot node supports two types of substream protocols: + established, the node initiates a handshake with the remote peer's on the + encryption layer. An additional layer on top of the encryption layer, known + as the multiplexing layer, allows a connection to be split into substreams, + as described by the , + either by the local or remote node. + + The Polkadot node supports two types of substream protocols: <\itemize-dot> : After the protocol is @@ -150,11 +144,9 @@ for more information. - The Polkadot Host can establish a connection with any peer it knows the - address ofof\<#2019\>. A preposition is a very bad word to end a sentence - with.>. The Polkadot Host supports multiple base-layer protocols: + The Polkadot Host can establish a connection with any peer of which it + knows the address. The Polkadot Host supports multiple networking + protocols: <\itemize> with addresses in the form of @@ -162,10 +154,9 @@ an encryption and a multiplexing layer. with addresses in the form of - to establish a TCP connection - and negotiate the Websocket protocol within the connection. Additionally, - an encryption and multiplexing layer is negotiated within the Websocket - connection. + to establish a TCP connection and negotiate + the Websocket protocol within the connection. Additionally, an encryption + and multiplexing layer is negotiated within the Websocket connection. addresses in form of and @@ -173,11 +164,10 @@ After a base-layer protocol is established, the Polkadot Host will apply - the Noise protocol + the Noise protocol to establish the encryption layer as described in + Section . - > + Polkadot protocol uses the > Noise framework to build an encryption protocol. The Noise protocol is a framework for bulding @@ -544,8 +534,10 @@ > > > + > > > + > > From 6cc949d2f69d9599b3c1d1203c8bc666487101de Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Fri, 29 Jan 2021 15:18:43 +0100 Subject: [PATCH 53/65] delete ImOnline section --- host-spec/c04-networking.tm | 31 ------------------------------- 1 file changed, 31 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 43bf84ab8..cd738ceb0 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -460,37 +460,6 @@ substream. The process for the creation of such votes is described in Section . - ?> Heartbeat> - - - - - - The I'm Online heartbeat is a crucial part of the Polkadot validation - process, as it signals the active participation of the Polkadot validator - node and confirms its reachability. The Polkadot network punishes - unreachable validators which have been elected to an authority set by - slashing their bonded funds . This is achieved by requiring validators to - issue a signed extrinsic known as I'm Online heartbeat, on the - start of every Era . - - The Polkadot Runtime fully manages the creation and the timing of the - extrinsic, but it's the responsiblity of the Polkadot host to gossip that - extrinsic to the rest of the network. When the Runtime decides to create - and propagate the heartbeat extrinsic, it calls the - Host API function as described - in Section . - - The process of gossiping extrinsics is defined in section - . - \; <\with|par-mode|right> From e5023c38ee69293ca834a21bf38ec3a7567a14de Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Fri, 29 Jan 2021 21:12:08 +0100 Subject: [PATCH 54/65] reference definitions in section Announcing blocks --- host-spec/c04-networking.tm | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index cd738ceb0..39f26d319 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -273,16 +273,16 @@ can request information about this block. The mechanism for tracking announcement and requesting the required data is implementation specific. - Block announcements and requests are conducted on the - substream conducted\<#2019\> and refe to def 2> + Block announcements, requests and responses are sent over the + substream as defined in Definition + . <\definition> - The initializes a substream to a remote - peer. Once established, all messages created by - the node are sent to that substream. + The + initializes a substream to a remote + peer. Once established, all messages as defined + in Definition are created by the node are + sent to the substream. The is a SCALE encoded structure of the following format: @@ -305,9 +305,9 @@ <\definition> - The message is sent to the specified substream - and indicates to remote peers the that node has either created or - received a new block. + The message is sent + to the specified substream and indicates to remote peers the that node + has either created or received a new block. The message is a SCALE encoded structure of the following format: @@ -501,6 +501,8 @@ > > > + > + > > > > From 13614461485e414f1f6fda5babbf3ec4ce4b5578 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Fri, 29 Jan 2021 23:44:21 +0100 Subject: [PATCH 55/65] clean up transactions subsection --- host-spec/c04-networking.tm | 78 ++++++++++++++++++------------------- 1 file changed, 39 insertions(+), 39 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 39f26d319..ff1627ce6 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -313,14 +313,14 @@ following format: <\eqnarray*> - ||,ib|)>>>>> + ||,b|)>>>>> where: <\eqnarray*> >||>>||||>>||||>>||>>>>>>>>> @@ -373,7 +373,8 @@ is a Protobuf structure indicating the sequence direction of the requested blocks. The structure is a varying - data type of the following format: + data type as defined in Definition + of the following format: <\big-table||||||||||>|>>||>|| - Transactions as defined and explained in are sent directly in its full form to - connected peers. It is considered good - behavior to implement a mechanism which only sends a - transaction once to each peer and avoids sending duplicates. Such a - mechanism is implementation specific and any absence of such a mechanism - can result in consequences which are undefined . On the othrer hand, the Polkadot host must be able to - handle duplicate transactions received from its peers and prevents the - inclusion of duplicate copies of a same transaction in - - - The transactions message is represented - by > and is defined as follows: - - <\equation*> - M\Enc,\,C|)> - - - in which: - - <\equation*> - C\Enc|)> - - - Where each > is a byte array and represents a sepearate - extrinsic. The Polkadot Host is - indifferent about the content of an extrinsic and treats it - as a blob of data. - - The exchange of transactions is conducted on the - substream. + Transactions as defined and described in Section + are sent directly to peers with which the + Polkadot Host has an open transaction substream, as defined in Definition + . Polkadot Host implementers should + implement a mechanism which only sends a transaction once to each peer and + avoids sending duplicates. Sending duplicate transactions might result in + undefined consequences such as being blocked for bad behavior by peers. + + The mechanism for managing transactions is further described in Section + respectively Section + and Section + . + + <\definition> + The is the + structure of how the transactions are sent over the network. It is + represented by > and is defined as follows: + + <\equation*> + M\Enc,\,C|)> + + + in which: + + <\equation*> + C\Enc|)> + + + Where each > is a byte array and represents a sepearate + extrinsic. The Polkadot Host is agnostic about the content of an + extrinsic and treats it as a blob of data. + + Transactions are sent over the substream. + @@ -504,6 +503,7 @@ > > > + > > > > From b599149117893a4f4eddb65aafb8000d78e6fbb9 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Sat, 30 Jan 2021 00:01:37 +0100 Subject: [PATCH 56/65] add additional references to noise specification and libp2p specification. mention scale and protobuf situation --- host-spec/c04-networking.tm | 36 ++++++++++++++---------------------- 1 file changed, 14 insertions(+), 22 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index ff1627ce6..834506234 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -26,8 +26,7 @@ The completeness of implementing the Polkadot networking protocol requires - the usage of external documentation. + the usage of external documentation. <\itemize> @@ -49,8 +48,6 @@ libp2p address format - - Each Polkadot Host node maintains a ED25519 key pair which is used to @@ -183,9 +180,9 @@ <\enumerate-numeric> The initiator generates a keypair and sends the public key to the - responder. + responder. The + and the + describe keypairs in more detail. The responder generates its own keypair and sends its public key back to the initiator. After that, the responder derives a shared secret @@ -201,13 +198,12 @@ public key and a signature to the responder. - After these three steps, both the initiator and responder derive a new shared secret using the static and session-defined Noise keys, which are - used to encrypt all further communication. The Noise specification - describes this process in detail. + used to encrypt all further communication. @@ -219,8 +215,7 @@ , where each protocol servers a specific utility. - The Polkadot Host adoptes adopt\<#2019\> mean - here?> the following substreams: + The Polkadot Host uses multiple substreams, each having a specific purpose: <\itemize> - Open a substream to a peer and @@ -258,13 +253,11 @@ The Polkadot Host must actively communicate with the network in order to participate in the validation process or act as a full node. - - - - - + : The Polkadot network originally only used SCALE encoding for + all message formats. Meanwhile, Protobuf has been adopted for certain + messages. The encoding of each message is explicitly mentioned in their + corresponding definition. Encoding and message formats are subject to + change. @@ -331,9 +324,8 @@ Block requests can be used to retrieve a range of blocks from peers. <\definition> - The message is a Protobuf serialized - structure of the following - format: + The message is a Protobuf serialized structure of + the following format: <\big-table||||||>|>|>|>>||||>>>||| Date: Mon, 1 Feb 2021 13:19:57 +0100 Subject: [PATCH 57/65] add description for networking chapter --- host-spec/c04-networking.tm | 32 ++++++++++---------------------- 1 file changed, 10 insertions(+), 22 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 834506234..b1d05f0da 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -20,10 +20,12 @@ protocols, including the distributed Kademlia hash table which is used for peer discovery. - + This chapter walks through the behavior of the networking implemenation of + the Polkadot Host and defines the network messages. Implementation details + of the used protocols are specified in external sources + as described in Section . - + The completeness of implementing the Polkadot networking protocol requires the usage of external documentation. @@ -187,9 +189,8 @@ The responder generates its own keypair and sends its public key back to the initiator. After that, the responder derives a shared secret and uses it to encrypt all further communication. The responder now sends - its static Noise public key (which is non-persistent and - generated on every node startup), its public key and a + its static Noise public key (which may change anytime and does not need + to be persisted on disk), its public key and a signature of the static Noise public key signed with the public key. @@ -198,9 +199,6 @@ public key and a signature to the responder. - - After these three steps, both the initiator and responder derive a new shared secret using the static and session-defined Noise keys, which are used to encrypt all further communication. @@ -484,7 +482,6 @@ > > > - > > > > @@ -500,7 +497,7 @@ > > > - > + > > @@ -567,10 +564,8 @@ |.>>>>|> > - |1.6Noise Protocol - ||||||>>>>>||||>>|[encryption - protocol/layer is a better name]>> |.>>>>|> + |1.6Encryption Layer + |.>>>>|> > |1.7Protocols and Substreams @@ -596,13 +591,6 @@ |1.8.4GRANDPA Votes |.>>>>|> > - - |1.9I'mOnline - ||||||>>>>>||||>>|[perhaps - |language||ImOnline>?]>> - Heartbeat |.>>>>|> - > \ No newline at end of file From 9ce57aacaaafcbfd9291c87786bd1ad0b0b7e8dd Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Mon, 1 Feb 2021 14:50:09 +0100 Subject: [PATCH 58/65] add description to each external doc source --- host-spec/c04-networking.tm | 31 +++++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index b1d05f0da..f3d03e74e 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -31,17 +31,39 @@ the usage of external documentation. <\itemize> - + - + is a modular peer-to-peer networking stack composed of many modules and + different parts. Included in libp2p are multiplexing protocols mplex and + yamux. - \ + - + is a distributed hash table for decentralized + peer-to-peer networks. The Polkadot Host uses Kademlia for peer + discovery. - + - The Noise protocol is a + framework for building cryptographic protocols. The Polkadot Host uses + Noise to establish the encryption layer to remote peers. + - is a multiplexing protocol developed by libp2p. The + protocol allows to divide a connection to a peer into multiple + substreams, each substream serving a specific purpose. Generally, + Polkadot Host implementers are encouraged to prioritize implementing + yamux, since it's the de facto standard in Polkadot. is + only required in order to communicate with + . + - is a multiplexing protocol like and + the de facto standard for the Polkadot Host. This protocol should be + prioritzed over . + - Protocol Buffers is a language-neutral, platform-neutral mechanism for + serializing structured data and is developed by Google. The Polkadot Host + uses Protocol Buffers to serialze specific messages, as clarified in + Section . > @@ -246,7 +268,7 @@ #7252|https://github.com/paritytech/substrate/issues/7252>.> - + The Polkadot Host must actively communicate with the network in order to participate in the validation process or act as a full node. @@ -497,6 +519,7 @@ > > > + > > > From adc549570ae734d46c2a1e08a3508909da63145b Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Mon, 1 Feb 2021 14:52:38 +0100 Subject: [PATCH 59/65] link to libp2p addressing system --- host-spec/c04-networking.tm | 42 ++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 22 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index f3d03e74e..2e0c40231 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -36,6 +36,10 @@ different parts. Included in libp2p are multiplexing protocols mplex and yamux. + + - The Polkadot Host uses the addressing system to + identity and connect to peers. + - is a distributed hash table for decentralized peer-to-peer networks. The Polkadot Host uses Kademlia for peer @@ -66,12 +70,6 @@ Section . - > - - <\todo> - libp2p address format - - Each Polkadot Host node maintains a ED25519 key pair which is used to @@ -492,16 +490,16 @@ <\references> <\collection> > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > > > @@ -515,13 +513,13 @@ > > > - > - > - > - > - > + > + > + > + > + > > - > + > From 9cae9583028abc27cf2dd8a03970c122d6fa9da2 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Mon, 1 Feb 2021 15:16:02 +0100 Subject: [PATCH 60/65] update buffer --- host-spec/c04-networking.tm | 73 +++++++++++++++++++------------------ 1 file changed, 37 insertions(+), 36 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 2e0c40231..1b57344d9 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -32,9 +32,10 @@ <\itemize> - - is a modular peer-to-peer networking stack composed of many modules and - different parts. Included in libp2p are multiplexing protocols mplex and - yamux. + is a modular peer-to->peer networking stack + composed of many modules and different parts. Included in + are multiplexing protocols and + . - The Polkadot Host uses the addressing system to @@ -50,18 +51,20 @@ Noise to establish the encryption layer to remote peers. - - is a multiplexing protocol developed by libp2p. The - protocol allows to divide a connection to a peer into multiple - substreams, each substream serving a specific purpose. Generally, - Polkadot Host implementers are encouraged to prioritize implementing - yamux, since it's the de facto standard in Polkadot. is - only required in order to communicate with + - is a multiplexing protocol developed by + . The protocol allows to divide a connection to a peer + into multiple substreams, each substream serving a specific purpose. + Generally, Polkadot Host implementers are encouraged to prioritize + implementing , since it's the de facto standard in + Polkadot. is only required in order to communicate with . - is a multiplexing protocol like and - the de facto standard for the Polkadot Host. This protocol should be - prioritzed over . + developed by HashiCorp. It's the de facto standard for the Polkadot Host. + This protocol should be prioritzed over . Section + described the subprotocol in more + detail. - Protocol Buffers is a language-neutral, platform-neutral mechanism for @@ -182,6 +185,8 @@ . + The addressing system is described in the specification. After a base-layer protocol is established, the Polkadot Host will apply the Noise protocol to establish the encryption layer as described in Section . @@ -501,7 +506,6 @@ > > > - > > > > @@ -525,34 +529,37 @@ <\auxiliary> <\collection> + <\associate|bib> + protocol_labs_libp2p_2019 + <\associate|table> |1>|> |language||BlockRequest> Protobuf message. - |> + |> |2>|> Bits of block data to be requested. - |> + |> |3>|> Protobuf message indicating the block to start from. - |> + |> |4>|> |language||Direction> Protobuf structure. - |> + |> |5>|> |language||BlockResponse> Protobuf message. - |> + |> |6>|> |math-font-series||BlockData> Protobuf structure. - |> + |> <\associate|toc> |math-font-series||font-size||4Networking> @@ -567,51 +574,45 @@ |.>>>>|> > - |1.2||||||>>>>>||||>>|[libp2p - convention perhaps]>> |.>>>>|> + |1.2Node Identities + |.>>>>|> > - |1.3Node Identities + |1.3Discovery mechanism |.>>>>|> > - |1.4Discovery mechanism + |1.4Connection establishment |.>>>>|> > - |1.5Connection establishment + |1.5Encryption Layer |.>>>>|> > - |1.6Encryption Layer + |1.6Protocols and Substreams |.>>>>|> > - |1.7Protocols and Substreams + |1.7Network Messages |.>>>>|> > - |1.8Network Messages + |1.7.1Announcing blocks |.>>>>|> > - |1.8.1Announcing blocks + |1.7.2Requesting blocks |.>>>>|> > - |1.8.2Requesting blocks + |1.7.3Transactions |.>>>>|> - > + > - |1.8.3Transactions + |1.7.4GRANDPA Votes |.>>>>|> > - - |1.8.4GRANDPA Votes - |.>>>>|> - > \ No newline at end of file From 95c8321bc8e8b9d9803e43818b6e6111c8bae84e Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Wed, 3 Feb 2021 13:39:21 +0100 Subject: [PATCH 61/65] small cleanups, add note about protocol identifiers --- host-spec/c04-networking.tm | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 1b57344d9..fe1fcf332 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -115,8 +115,8 @@ invoking Kademlia requests, where nodes respond with their list of available peers. - Kademlia requests must also contain protocol identifiers as described in - Section . + Kademlia requests are performed on a specific substream as described in + Section . @@ -132,10 +132,11 @@ <\itemize-dot> : After the protocol is - negotiated, the initiator sends a single message containing a request. - The responder then sends a response, after which the substream is then - immediately closed. The requests and responses are prefixed with their - encoded length. + negotiated by the multiplexing layer, the initiator sends a single + message containing a request. The responder then sends a response, after + which the substream is then immediately closed. The requests and + responses are prefixed with their + encoded length. . After the protocol is negotiated, the initiator sends a single handshake message. The responder can then @@ -257,7 +258,8 @@ allows the Polkadot Host to perform information about blocks. - a request and response protocol that - allows a light client to perform request information about the state. + allows a light client to request information about the state. - a substream/notification protocol which sends transactions to connected peers. @@ -271,6 +273,12 @@ #7252|https://github.com/paritytech/substrate/issues/7252>.> + : the prefixes on those substreams are known + as protocol identifiers and are used to segregate communications to + specific networks. This prevents any interference with other networks. + is used exclusively for Polkadot. Kusama, for example, + uses the protocol identifier. + The Polkadot Host must actively communicate with the network in order to @@ -337,8 +345,8 @@ <\eqnarray*> >||>>||||>>||>>>>>>>>> + not part of the best chain>>>||>>>>>>>>> @@ -529,9 +537,6 @@ <\auxiliary> <\collection> - <\associate|bib> - protocol_labs_libp2p_2019 - <\associate|table> |1>|> |language||BlockRequest> From 0b782a39f9a8479de1455f913d246fb7931bc74a Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Wed, 3 Feb 2021 13:55:00 +0100 Subject: [PATCH 62/65] mention protocol type for each substream --- host-spec/c04-networking.tm | 39 ++++++++++++++++++++++++------------- 1 file changed, 25 insertions(+), 14 deletions(-) diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index fe1fcf332..97e70f48a 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -119,7 +119,7 @@ Section . - + Polkadot nodes connect to peers by establishing a TCP connection. Once established, the node initiates a handshake with the remote peer's on the @@ -128,7 +128,9 @@ as described by the , either by the local or remote node. - The Polkadot node supports two types of substream protocols: + The Polkadot node supports two types of substream protocols. Section + describes the usage of each type in + more detail: <\itemize-dot> : After the protocol is @@ -239,38 +241,46 @@ , where each protocol servers a specific utility. - The Polkadot Host uses multiple substreams, each having a specific purpose: + The Polkadot Host uses multiple substreams, where its usage depends on a + specific purpose. Each substream is either a or a , as described in Section + . <\itemize> - Open a substream to a peer and initialize a ping to verify if a connection is till alive. If the peer - does not respond, the connection is dropped. + does not respond, the connection is dropped. This is a + . - Open a substream to a peer to ask - information about that peer. + information about that peer. This is a . - Open a substream for Kademlia - requests. + requests. This is a , + as defined by the standard. <\itemize> - a request and response protocol that - allows the Polkadot Host to perform information about blocks. + allows the Polkadot Host to perform information about blocks. This is a + . - a request and response protocol that - allows a light client to request information about the state. + allows a light client to request information about the state. This is a + . - a substream/notification protocol - which sends transactions to connected peers. + which sends transactions to connected peers. This is a . - a substream/notification - protocol which sends blocks to connected peers. + protocol which sends blocks to connected peers. This is a + . - a substream/notification - protocol which sends GRANDPA votes to connected peers. .> + protocol which sends GRANDPA votes to connected peers. This is a + . .> : the prefixes on those substreams are known @@ -525,6 +535,7 @@ > > > + > > > > From d4ef4ed227a5648fc0982c9e1af74a8804ff6711 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Thu, 4 Feb 2021 16:31:10 +0100 Subject: [PATCH 63/65] update all in host-spec.tm --- host-spec/host-spec.tm | 3840 ++++++++++++++++++++-------------------- 1 file changed, 1923 insertions(+), 1917 deletions(-) diff --git a/host-spec/host-spec.tm b/host-spec/host-spec.tm index 19b30be80..5ae817dac 100644 --- a/host-spec/host-spec.tm +++ b/host-spec/host-spec.tm @@ -1,4 +1,4 @@ - + > @@ -194,1377 +194,1395 @@ |.>>>>|> > - Discovery mechanism + Node Identities |.>>>>|> > - Connection establishment + Discovery mechanism |.>>>>|> > - Substreams + Connection establishment |.>>>>|> > - 4.2.Network Messages |.>>>>|> - + Encryption Layer + |.>>>>|> + > - API Package + Protocols and Substreams |.>>>>|> > - BlockRequest + Network Messages |.>>>>|> > - BlockResponse + Announcing blocks |.>>>>|> > - Light Package + Requesting blocks |.>>>>|> > - Request - |.>>>>|> - > - - Response - |.>>>>|> - > - - RemoteCallRequest - |.>>>>|> - > - - RemoteCallResponse - |.>>>>|> - > - - RemoteReadRequest - |.>>>>|> - > - - RemoteReadResponse - |.>>>>|> - > - - RemoteReadChildRequest + Transactions |.>>>>|> > - RemoteHeaderRequest + GRANDPA Votes |.>>>>|> > - RemoteHeaderResponse - |.>>>>|> - > - - RemoteChangesRequest - |.>>>>|> - > - - RemoteChangesResponse - |.>>>>|> - > - - Finality Package - |.>>>>|> - > - - FinalityProofRequest - |.>>>>|> - > - - FinalityProofResponse - |.>>>>|> - > - - 4.3. To be migrated |.>>>>|> - - - Node Identities and Addresses - |.>>>>|> - > - - Discovery Mechanisms - |.>>>>|> - > - - Transport Protocol - |.>>>>|> - > - - Encryption - |.>>>>|> - > - - Multiplexing - |.>>>>|> - > - - Substreams - |.>>>>|> - > - - Periodic Ephemeral Substreams - |.>>>>|> - > - - Polkadot Communication Substream - |.>>>>|> - > - Bootstrapping> |.>>>>|> - + Consensus> |.>>>>|> - + 6.1.Common Consensus Structures |.>>>>|> - + Consensus Authority Set |.>>>>|> - > + > Runtime-to-Consensus Engine Message |.>>>>|> - > + > 6.2.Block Production |.>>>>|> - + Preliminaries |.>>>>|> - > + > Block Production Lottery |.>>>>|> - > + > Slot Number Calculation |.>>>>|> - > + > Block Production |.>>>>|> - > + > Epoch Randomness |.>>>>|> - > + > Verifying Authorship Right |.>>>>|> - > + > Block Building Process |.>>>>|> - > + > 6.3.Finality |.>>>>|> - + Preliminaries |.>>>>|> - > + > GRANDPA Messages Specification |.>>>>|> - > + > Vote Messages |.>>>>|> - > + > Finalizing Message |.>>>>|> - > + > Catch-up Messages |.>>>>|> - > + > Initiating the GRANDPA State |.>>>>|> - > + > Voter Set Changes |.>>>>|> - > + > Voting Process in Round |.>>>>|> - > + > 6.4.Block Finalization |.>>>>|> - + Catching up |.>>>>|> - > + > Sending catch-up requests |.>>>>|> - > + > Processing catch-up requests |.>>>>|> - > + > Processing catch-up responses |.>>>>|> - > + > Availability & Validity> |.>>>>|> - + 7.1.Introduction |.>>>>|> - + 7.2.Preliminaries |.>>>>|> - + Extra Validation Data |.>>>>|> - > + > 7.3.Overal process |.>>>>|> - + 7.4.Candidate Selection |.>>>>|> - + 7.5.Candidate Backing |.>>>>|> - + Inclusion of candidate receipt on the relay chain |.>>>>|> - > + > 7.6.PoV Distribution |.>>>>|> - + Primary Validation Disagreement |.>>>>|> - > + > 7.7.Availability |.>>>>|> - + 7.8.Distribution of Chunks |.>>>>|> - + 7.9.Announcing Availability |.>>>>|> - + Processing on-chain availability data |.>>>>|> - > + > 7.10.Publishing Attestations |.>>>>|> - + 7.11.Secondary Approval checking |.>>>>|> - + Approval Checker Assignment |.>>>>|> - > + > VRF computation |.>>>>|> - > + > One-Shot Approval Checker Assignment |.>>>>|> - > + > Extra Approval Checker Assigment |.>>>>|> - > + > Additional Checking in Case of Equivocation |.>>>>|> - > + > 7.12.The Approval Check |.>>>>|> - + Retrieval |.>>>>|> - > + > Reconstruction |.>>>>|> - > + > Verification |.>>>>|> - > + > Process validity and invalidity messages |.>>>>|> - > + > Invalidity Escalation |.>>>>|> - > + > Message Passing> |.>>>>|> - + 8.1.Overview |.>>>>|> - + 8.2.Message Queue Chain (MQC) |.>>>>|> - + 8.3.HRMP |.>>>>|> - + Channels |.>>>>|> - > + > Opening Channels |.>>>>|> - > + > Workflow |.>>>>|> - > + > Accepting Channels |.>>>>|> - > + > Workflow |.>>>>|> - > + > Closing Channels |.>>>>|> - > + > Workflow |.>>>>|> - > + > Sending messages |.>>>>|> - > + > Receiving Messages |.>>>>|> - > + > 8.4.XCMP |.>>>>|> - + CST: Channel State Table |.>>>>|> - > + > Message content |.>>>>|> - > + > Watermark |.>>>>|> - > + > 8.5.SPREE |.>>>>|> - + Cryptographic Algorithms> |.>>>>|> - + A.1.Hash Functions |.>>>>|> - + A.2.BLAKE2 |.>>>>|> - + A.3.Randomness |.>>>>|> - + A.4.VRF |.>>>>|> - + A.5.Cryptographic Keys |.>>>>|> - + Holding and staking funds |.>>>>|> - > + > Creating a Controller key |.>>>>|> - > + > Designating a proxy for voting |.>>>>|> - > + > Controller settings |.>>>>|> - > + > Certifying keys |.>>>>|> - > + > Auxiliary Encodings> |.>>>>|> - + B.1.SCALE Codec |.>>>>|> - + Length and Compact Encoding |.>>>>|> - > + > B.2.Hex Encoding |.>>>>|> - + Genesis State Specification> |.>>>>|> - + Network Messages> |.>>>>|> - + D.Polkadot Host API> |.>>>>|> + + + D.1.Storage |.>>>>|> + - D.1.Detailed Message Structure + |.>>>>|> - + > + + Version 1 - Prototype + |.>>>>|> + > + + + |.>>>>|> + > + + Version 1 - Prototype + |.>>>>|> + > - Status Message + + |.>>>>|> + > + + Version 1 - Prototype + |.>>>>|> + > + + + |.>>>>|> + > + + Version 1 - Prototype + |.>>>>|> + > + + + |.>>>>|> + > + + Version 1 - Prototype + |.>>>>|> + > + + + |.>>>>|> + > + + Version 1 - Prototype + |.>>>>|> + > + + + |.>>>>|> + > + + Version 1 - Prototype + |.>>>>|> + > + + + |.>>>>|> + > + + Version 1 - Prototype + |.>>>>|> + > + + |.>>>>|> > - Block Request Message + Version 1 - Prototype + |.>>>>|> + > + + |.>>>>|> > - Block Response Message + Version 1 - Prototype + |.>>>>|> + > + + |.>>>>|> > - Block Announce Message + Version 1 - Prototype |.>>>>|> > - Transactions + |.>>>>|> > - Consensus Message + Version 1 - Prototype |.>>>>|> > - Neighbor Packet + |.>>>>|> > - Polkadot Host API> |.>>>>|> - + Version 1 - Prototype + |.>>>>|> + > - E.1.Storage |.>>>>|> + D.2.Child Storage |.>>>>|> - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - - |.>>>>|> - > - - Version 1 - Prototype - |.>>>>|> - > + D.3.Crypto |.>>>>|> + - - |.>>>>|> - > - - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - E.2.Child Storage |.>>>>|> - + Version 1 - Prototype + |.>>>>|> + > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 2 - Prototype |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - E.3.Crypto |.>>>>|> - + Version 1 - Prototype + |.>>>>|> + > + + + |.>>>>|> + > + + Version 1 - Prototype + |.>>>>|> + > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - - |.>>>>|> - > + D.4.Hashing |.>>>>|> + - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 2 - Prototype + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - - |.>>>>|> - > - - Version 1 - Prototype - |.>>>>|> - > + D.5.Offchain |.>>>>|> + - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - E.4.Hashing |.>>>>|> - + + |.>>>>|> + > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - E.5.Offchain |.>>>>|> - + Version 1 - Prototype + |.>>>>|> + > + + + |.>>>>|> + > - + Version 1- Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype - |.>>>>|> - > + D.6.Trie |.>>>>|> + - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - - |.>>>>|> - > + D.7.Miscellaneous |.>>>>|> + - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1- Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype - |.>>>>|> - > + D.8.Allocator |.>>>>|> + - + |.>>>>|> > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> > - E.6.Trie |.>>>>|> - - - + |.>>>>|> - > + > - Version 1 - Prototype + Version 1 - Prototype |.>>>>|> - > + > - - |.>>>>|> - > + D.9.Logging |.>>>>|> + - Version 1 - Prototype + |.>>>>|> > - + Version 1 - Prototype |.>>>>|> > - Version 1 - Prototype + Legacy Polkadot Host API> |.>>>>|> - > + - - |.>>>>|> - > + E.1.Storage |.>>>>|> + - Version 1 - Prototype + |.>>>>|> > - E.7.Miscellaneous |.>>>>|> - + + |.>>>>|> + > - + |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - E.8.Allocator |.>>>>|> - + + |.>>>>|> + > - + |.>>>>|> > - Version 1 - Prototype + Memory |.>>>>|> > - + |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - E.9.Logging |.>>>>|> - + Input/Output + |.>>>>|> + > - + Cryptograhpic Auxiliary Functions + |.>>>>|> + > + + |.>>>>|> > - Version 1 - Prototype + |.>>>>|> > - Legacy Polkadot Host API> + |.>>>>|> - + > - F.1.Storage |.>>>>|> - + + |.>>>>|> + > - + |.>>>>|> > - + To be Specced |.>>>>|> > - - |.>>>>|> + Offchain Worker + \ |.>>>>|> > - + |.>>>>|> > - + |.>>>>|> > - + |.>>>>|> > - + |.>>>>|> > - + |.>>>>|> > - + |.>>>>|> > - + |.>>>>|> > - + |.>>>>|> > - + |.>>>>|> > - + |.>>>>|> > - + |.>>>>|> > - Memory + |.>>>>|> > - + |.>>>>|> > - + |.>>>>|> > - Input/Output + |.>>>>|> > - Cryptograhpic Auxiliary Functions + Sandboxing |.>>>>|> > - + To be Specced |.>>>>|> > - + Auxillary Debugging API |.>>>>|> > - + |.>>>>|> > - + |.>>>>|> > - + Misc |.>>>>|> > - To be Specced + To be Specced |.>>>>|> > - Offchain Worker - \ |.>>>>|> + Block Production + |.>>>>|> > - + E.2.Validation |.>>>>|> + + + Runtime Entries> |.>>>>|> + + + F.1.List of Runtime Entries |.>>>>|> - > + - + F.2.JSON-RPC API for external services |.>>>>|> - > + - + F.3.Argument Specification |.>>>>|> - > + - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - Sandboxing + |.>>>>|> - > + > - To be Specced + |.>>>>|> - > + > - Auxillary Debugging API + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - Misc + |.>>>>|> - > + > - To be Specced + |.>>>>|> - > + > - Block Production + |.>>>>|> - > + > - F.2.Validation |.>>>>|> - + + |.>>>>|> + > - Runtime Entries> |.>>>>|> - + + |.>>>>|> + > - G.1.List of Runtime Entries + |.>>>>|> - + > - G.2.Argument Specification + |.>>>>|> - + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + |.>>>>|> - > + > - + + |.>>>>|> + > + + |.>>>>|> > - + |.>>>>|> > + + |.>>>>|> + > + + + |.>>>>|> + > + + + |.>>>>|> + > + + + |.>>>>|> + > + |.>>>>|> - + |.>>>>|> - + |.>>>>|> - + \; @@ -1712,7 +1730,7 @@ <\bibliography|bib|tm-alpha|host-spec> - <\bib-list|12> + <\bib-list|8> Jeff Burdges. Schnorr VRFs and signatures on the Ristretto group. , 2019. @@ -1748,18 +1766,6 @@ >, 8032. 2017. - Protocol labs. - Libp2p Specification. , - Protocol labs, , 2019. - - Ilari - LiusvaaraSimon Josefsson. Edwards-Curve - Digital Signature Algorithm (EdDSA). 2017. - - Trevor Perrin. - The Noise Protocol Framework. , , 2018. - MarkkuJuhani SaarinenJean-Philippe Aumasson. The BLAKE2 cryptographic hash and message authentication code (MAC). @@ -1769,11 +1775,6 @@ Alistair Stewart. GRANDPA: A Byzantine Finality Gadget. 2019. - - Parity - Technologies. Substrate Reference Documentation. - Rust , Parity Technologies, - , 2019. @@ -1795,803 +1796,785 @@ <\references> <\collection> - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - |\>|125>> - > - > - > - > - > - > - |\>|41>> - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + |>|15|c01-background.tm>> + > + > + > + > + > + > + > + > + > + > + |B,B>)|15|c01-background.tm>> + > + > + > + > + > + > + > + > + > + > + |\>BT|)>>|15|c01-background.tm>> + > + > + > + > + > + > + > + > + > + > + |\>|15|c01-background.tm>> + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + |math-font-series|||P>>|13|c01-background.tm>> + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + |\>|139>> + |\>|141>> + > + > + > + > + > + |\>|14|c01-background.tm>> + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + |B>|14|c01-background.tm>> + > + > + > + > + > + > + > + > + > + > + |Enc>|14|c01-background.tm>> + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + |\>|111>> + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > <\auxiliary> <\collection> <\associate|bib> - parity_technologies_substrate_2019 - - protocol_labs_libp2p_2019 - - liusvaara_edwards-curve_2017 - - protocol_labs_libp2p_2019 - - protocol_labs_libp2p_2019 - - perrin_noise_2018 - - protocol_labs_libp2p_2019 - - protocol_labs_libp2p_2019 - - protocol_labs_libp2p_2019 - w3f_research_group_blind_2019 david_ouroboros_2018 @@ -2619,22 +2602,22 @@ Examplary result of Median Algorithm in first sync epoch with |s=9> and |k=1>. - |> + |> |7.1>||Overall process - to acheive availability and validity in Polkadot>|> + to acheive availability and validity in Polkadot>|> |8.1>||Parachain Message - Passing Overview>|> + Passing Overview>|> - |8.2>||MQC - Overview>|> + |8.2>||Message Queue + Chain Overview>|> |8.3>||Parachain XCMP - Overview>|> + Overview>|> - |G.1>||Snippet to export - entries into tho Wasm runtime module.>|> + |F.1>||Snippet to export + entries into tho Wasm runtime module.>|> <\associate|gly> |math-font-series|||P>>|a @@ -2750,37 +2733,37 @@ |> <\associate|parts> - |subsection-nr|0> + - |subsection-nr|1> + - |subsection-nr|1> + - |subsection-nr|4> + - |subsection-nr|3> + - |subsection-nr|4> + - |subsection-nr|4> + - |subsection-nr|1> + - |subsection-nr|3> + - |subsection-nr|0> + - |subsection-nr|5> + - |subsection-nr|0> + - |subsection-nr|0> + - |subsection-nr|7> + - |subsection-nr|1> + - |subsection-nr|0> + <\associate|table> |3.1>|> @@ -2794,123 +2777,128 @@ Possible types of keys of mappings in the Changes Trie |> + |4.1>|> + |language||BlockRequest> + Protobuf message. + |> + + |4.2>|> + Bits of block data to be requested. + |> + + |4.3>|> + Protobuf message indicating the block to start from. + |> + + |4.4>|> + |language||Direction> + Protobuf structure. + |> + + |4.5>|> + |language||BlockResponse> + Protobuf message. + |> + + |4.6>|> + |math-font-series||BlockData> + Protobuf structure. + |> + |6.1>|> The consensus digest item for BABE authorities - |> + |> |6.2>|> The consensus digest item for GRANDPA authorities - |> + |> |6.3>|> \; - |> + |> |6.4>|> Signature for a message in a round. - |> + |> |A.1>|> List of public key scheme which can be used for an account key - |> + |> |A.2>|> List of key schemes which are used for session keys depending on the protocol - |> + |> |C.1>|> Genesis header values - |> + |> - |D.1>||List of possible - network message types.>|> - - |D.2>||Node role - representation in the status message.>|> - - |D.3>||Bit values for - block attribute |A>, to indicate the - requested parts of the data.>|> - - |E.1>|> + |D.1>|> Table of known key type identifiers - |> + |> - |E.2>|> + |D.2>|> Table of error types in ECDSA recovery - |> + |> - |E.3>|> + |D.3>|> Table of possible HTTP error types - |> + |> - |E.4>|> + |D.4>|> Log Levels for the logging interface - |> + |> - |G.1>||Detail of the + |F.1>||Detail of the version data type returns from runtime |language||version> - function.>|> + function.>|> - |G.2>|> - The tuple provided by |math-font-series||BabeApi_configuration>. - |> - - |G.3>|> - The tuple provided by |language||TaggedTransactionQueue_transaction_validity> - - in the case the transaction is judged to be valid. - |> - - |G.4>||Type variation - for the return value of |language||TaggedTransactionQueue_transaction_validity>.>|> - - |G.5>|> - Type variant whichs gets appended to Id 0 of - |math-font-series||TransactionValidityError>. - |> - - |G.6>|> - Type variant whichs gets appended to Id 1 of - |math-font-series||TransactionValidityError>. - |> - - |G.7>|> + |F.2>|> Possible values of varying data type |math-font-series||ApplyExtrinsicResult>. - |> + |> - |G.8>|> + |F.3>|> Possible values of varying data type |math-font-series||DispatchOutcome>. - |> + |> - |G.9>|> + |F.4>|> Possible values of varying data type |math-font-series||DispatchError>. - |> + |> - |G.10>|> + |F.5>|> Possible values of varying data type |math-font-series||CustomModuleError>. - |> + |> - |G.11>|> + |F.6>|> Possible values of varying data type |math-font-series||TransactionValidityError>. - |> + |> - |G.12>|> + |F.7>|> Possible values of varying data type |math-font-series||InvalidTransaction>. - |> + |> - |G.13>|> + |F.8>|> Possible values of varying data type |math-font-series||UnknownTransaction>. - |> + |> + + |F.9>|> + The tuple provided by |language||TaggedTransactionQueue_transaction_validity> + + in the case the transaction is judged to be valid. + |> + + |F.10>|> + The tuple provided by |math-font-series||BabeApi_configuration>. + |> <\associate|toc> |math-font-series||font-shape||1.Background> @@ -3037,1410 +3025,1428 @@ Block |.>>>>|> > - |3.3.3.Managaing Multiple - Variants of State |.>>>>|> - > - - |3.3.4.Changes Trie - |.>>>>|> - > - - |3.3.4.1.Key to extrinsics pairs - |.>>>>|> - > - - |3.3.4.2.Key to block pairs - |.>>>>|> - > - - |3.3.4.3.Key to Child Changes - Trie pairs |.>>>>|> - > - - |math-font-series||font-shape||4.Networking> - |.>>>>|> - - - 4.1.Introduction |.>>>>|> - - - |4.1.1.External Documentation - |.>>>>|> - > - - |4.1.2.Discovery mechanism - |.>>>>|> - > - - |4.1.3.Connection establishment - |.>>>>|> - > - - |4.1.4.Substreams - |.>>>>|> - > - - 4.2.Network Messages |.>>>>|> - - - |4.2.1.API Package - |.>>>>|> - > - - |4.2.1.1.BlockRequest - |.>>>>|> - > - - |4.2.1.2.BlockResponse - |.>>>>|> - > - - |4.2.2.Light Package - |.>>>>|> - > - - |4.2.2.1.Request - |.>>>>|> - > - - |4.2.2.2.Response - |.>>>>|> - > + |3.3.3.Managaing Multiple + Variants of State |.>>>>|> + > - |4.2.2.3.RemoteCallRequest + |3.3.4.Changes Trie |.>>>>|> - > + > - |4.2.2.4.RemoteCallResponse + |3.3.4.1.Key to extrinsics pairs |.>>>>|> - > + > - |4.2.2.5.RemoteReadRequest + |3.3.4.2.Key to block pairs |.>>>>|> - > + > - |4.2.2.6.RemoteReadResponse - |.>>>>|> - > + |3.3.4.3.Key to Child Changes + Trie pairs |.>>>>|> + > - |4.2.2.7.RemoteReadChildRequest + |math-font-series||font-shape||4.Networking> |.>>>>|> - > + - |4.2.2.8.RemoteHeaderRequest - |.>>>>|> - > + 4.1.Introduction |.>>>>|> + - |4.2.2.9.RemoteHeaderResponse + |4.1.1.External Documentation |.>>>>|> - > + > - |4.2.2.10.RemoteChangesRequest + |4.1.2.Node Identities |.>>>>|> - > + > - |4.2.2.11.RemoteChangesResponse + |4.1.3.Discovery mechanism |.>>>>|> - > + > - |4.2.3.Finality Package + |4.1.4.Connection establishment |.>>>>|> - > + > - |4.2.3.1.FinalityProofRequest + |4.1.5.Encryption Layer |.>>>>|> - > + > - |4.2.3.2.FinalityProofResponse + |4.1.6.Protocols and Substreams |.>>>>|> - > - - 4.3. To be migrated |.>>>>|> - - - |4.3.1.Node Identities and - Addresses |.>>>>|> - > + > - |4.3.2.Discovery Mechanisms + |4.1.7.Network Messages |.>>>>|> - > + > - |4.3.3.Transport Protocol + |4.1.7.1.Announcing blocks |.>>>>|> - > + > - |4.3.3.1.Encryption + |4.1.7.2.Requesting blocks |.>>>>|> - > + > - |4.3.3.2.Multiplexing + |4.1.7.3.Transactions |.>>>>|> - > + > - |4.3.4.Substreams + |4.1.7.4.GRANDPA Votes |.>>>>|> - > - - |4.3.4.1.Periodic Ephemeral - Substreams |.>>>>|> - > - - |4.3.4.2.Polkadot Communication - Substream |.>>>>|> - > + > |math-font-series||font-shape||5.Bootstrapping> |.>>>>|> - + |math-font-series||font-shape||6.Consensus> |.>>>>|> - + 6.1.Common Consensus Structures |.>>>>|> - + |6.1.1.Consensus Authority Set |.>>>>|> - > + > |6.1.2.Runtime-to-Consensus Engine Message |.>>>>|> - > + > 6.2.Block Production |.>>>>|> - + |6.2.1.Preliminaries |.>>>>|> - > + > |6.2.2.Block Production Lottery |.>>>>|> - > + > |6.2.3.Slot Number Calculation |.>>>>|> - > + > |6.2.4.Block Production |.>>>>|> - > + > |6.2.5.Epoch Randomness |.>>>>|> - > + > |6.2.6.Verifying Authorship Right |.>>>>|> - > + > |6.2.7.Block Building Process |.>>>>|> - > + > 6.3.Finality |.>>>>|> - + |6.3.1.Preliminaries |.>>>>|> - > + > |6.3.2.GRANDPA Messages Specification |.>>>>|> - > + > |6.3.2.1.Vote Messages |.>>>>|> - > + > |6.3.2.2.Finalizing Message |.>>>>|> - > + > |6.3.2.3.Catch-up Messages |.>>>>|> - > + > |6.3.3.Initiating the GRANDPA State |.>>>>|> - > + > |6.3.3.1.Voter Set Changes |.>>>>|> - > + > |6.3.4.Voting Process in Round |r> |.>>>>|> - > + > 6.4.Block Finalization |.>>>>|> - + |6.4.1.Catching up |.>>>>|> - > + > |6.4.1.1.Sending catch-up requests |.>>>>|> - > + > |6.4.1.2.Processing catch-up requests |.>>>>|> - > + > |6.4.1.3.Processing catch-up responses |.>>>>|> - > + > |math-font-series||font-shape||7.Availability & Validity> |.>>>>|> - + 7.1.Introduction |.>>>>|> - + 7.2.Preliminaries |.>>>>|> - + |7.2.1.Extra Validation Data |.>>>>|> - > + > 7.3.Overal process |.>>>>|> - + 7.4.Candidate Selection |.>>>>|> - + 7.5.Candidate Backing |.>>>>|> - + |7.5.1.Inclusion of candidate receipt on the relay chain |.>>>>|> - > + > 7.6.PoV Distribution |.>>>>|> - + |7.6.1.Primary Validation Disagreement |.>>>>|> - > + > 7.7.Availability |.>>>>|> - + 7.8.Distribution of Chunks |.>>>>|> - + 7.9.Announcing Availability |.>>>>|> - + |7.9.1.Processing on-chain availability data |.>>>>|> - > + > 7.10.Publishing Attestations |.>>>>|> - + 7.11.Secondary Approval checking |.>>>>|> - + |7.11.1.Approval Checker Assignment |.>>>>|> - > + > |7.11.2.VRF computation |.>>>>|> - > + > |7.11.3.One-Shot Approval Checker Assignment |.>>>>|> - > + > |7.11.4.Extra Approval Checker Assigment |.>>>>|> - > + > |7.11.5.Additional Checking in Case of Equivocation |.>>>>|> - > + > 7.12.The Approval Check |.>>>>|> - + |7.12.0.1.Retrieval |.>>>>|> - > + > |7.12.0.2.Reconstruction |.>>>>|> - > + > |7.12.1.Verification |.>>>>|> - > + > |7.12.2.Process validity and invalidity messages |.>>>>|> - > + > |7.12.3.Invalidity Escalation |.>>>>|> - > + > |math-font-series||font-shape||8.Message Passing> |.>>>>|> - + 8.1.Overview |.>>>>|> - + 8.2.Message Queue Chain (MQC) |.>>>>|> - + 8.3.HRMP |.>>>>|> - + |8.3.1.Channels |.>>>>|> - > + > |8.3.2.Opening Channels |.>>>>|> - > + > |8.3.2.1.Workflow |.>>>>|> - > + > |8.3.3.Accepting Channels |.>>>>|> - > + > |8.3.3.1.Workflow |.>>>>|> - > + > |8.3.4.Closing Channels |.>>>>|> - > + > |8.3.5.Workflow |.>>>>|> - > + > |8.3.6.Sending messages |.>>>>|> - > + > |8.3.7.Receiving Messages |.>>>>|> - > + > 8.4.XCMP |.>>>>|> - + |8.4.1.CST: Channel State Table |.>>>>|> - > + > |8.4.2.Message content |.>>>>|> - > + > |8.4.3.Watermark |.>>>>|> - > + > 8.5.SPREE |.>>>>|> - + |math-font-series||font-shape||Appendix A.Cryptographic Algorithms> |.>>>>|> - + A.1.Hash Functions |.>>>>|> - + A.2.BLAKE2 |.>>>>|> - + A.3.Randomness |.>>>>|> - + A.4.VRF |.>>>>|> - + A.5.Cryptographic Keys |.>>>>|> - + |A.5.1.Holding and staking funds |.>>>>|> - > + > |A.5.2.Creating a Controller key |.>>>>|> - > + > |A.5.3.Designating a proxy for voting |.>>>>|> - > + > |A.5.4.Controller settings |.>>>>|> - > + > |A.5.5.Certifying keys |.>>>>|> - > + > |math-font-series||font-shape||Appendix B.Auxiliary Encodings> |.>>>>|> - + B.1.SCALE Codec |.>>>>|> - + |B.1.1.Length and Compact Encoding |.>>>>|> - > + > B.2.Hex Encoding |.>>>>|> - + |math-font-series||font-shape||Appendix C.Genesis State Specification> |.>>>>|> - + |math-font-series||font-shape||Appendix - D.Network Messages> |.>>>>|> - + D.Polkadot Host API> |.>>>>|> + + + D.1.Storage |.>>>>|> + + + |D.1.1.|language||ext_storage_set> + |.>>>>|> + > + + |D.1.1.1.Version 1 - Prototype + |.>>>>|> + > + + |D.1.2.|language||ext_storage_get> + |.>>>>|> + > - D.1.Detailed Message Structure + |D.1.2.1.Version 1 - Prototype |.>>>>|> - + > + + |D.1.3.|language||ext_storage_read> + |.>>>>|> + > + + |D.1.3.1.Version 1 - Prototype + |.>>>>|> + > + + |D.1.4.|language||ext_storage_clear> + |.>>>>|> + > + + |D.1.4.1.Version 1 - Prototype + |.>>>>|> + > + + |D.1.5.|language||ext_storage_exists> + |.>>>>|> + > + + |D.1.5.1.Version 1 - Prototype + |.>>>>|> + > + + |D.1.6.|language||ext_storage_clear_prefix> + |.>>>>|> + > + + |D.1.6.1.Version 1 - Prototype + |.>>>>|> + > + + |D.1.7.|language||ext_storage_append> + |.>>>>|> + > + + |D.1.7.1.Version 1 - Prototype + |.>>>>|> + > + + |D.1.8.|language||ext_storage_root> + |.>>>>|> + > + + |D.1.8.1.Version 1 - Prototype + |.>>>>|> + > - |D.1.1.Status Message + |D.1.9.|language||ext_storage_changes_root> |.>>>>|> > - |D.1.2.Block Request Message + |D.1.9.1.Version 1 - Prototype + |.>>>>|> + > + + |D.1.10.|language||ext_storage_next_key> |.>>>>|> > - |D.1.3.Block Response Message + |D.1.10.1.Version 1 - Prototype + |.>>>>|> + > + + |D.1.11.|language||ext_storage_start_transaction> |.>>>>|> > - |D.1.4.Block Announce Message + |D.1.11.1.Version 1 - Prototype |.>>>>|> > - |D.1.5.Transactions + |D.1.12.|language||ext_storage_rollback_transaction> |.>>>>|> > - |D.1.6.Consensus Message + |D.1.12.1.Version 1 - Prototype |.>>>>|> > - |D.1.7.Neighbor Packet + |D.1.13.|language||ext_storage_commit_transaction> |.>>>>|> > - |math-font-series||font-shape||Appendix - E.Polkadot Host API> |.>>>>|> - + |D.1.13.1.Version 1 - Prototype + |.>>>>|> + > - E.1.Storage |.>>>>|> + D.2.Child Storage |.>>>>|> - |E.1.1.|language||ext_storage_set> + |D.2.1.|language||ext_default_child_storage_set> |.>>>>|> > - |E.1.1.1.Version 1 - Prototype + |D.2.1.1.Version 1 - Prototype |.>>>>|> > - |E.1.2.|language||ext_storage_get> + |D.2.2.|language||ext_default_child_storage_get> |.>>>>|> > - |E.1.2.1.Version 1 - Prototype + |D.2.2.1.Version 1 - Prototype |.>>>>|> > - |E.1.3.|language||ext_storage_read> + |D.2.3.|language||ext_default_child_storage_read> |.>>>>|> > - |E.1.3.1.Version 1 - Prototype + |D.2.3.1.Version 1 - Prototype |.>>>>|> > - |E.1.4.|language||ext_storage_clear> + |D.2.4.|language||ext_default_child_storage_clear> |.>>>>|> > - |E.1.4.1.Version 1 - Prototype + |D.2.4.1.Version 1 - Prototype |.>>>>|> > - |E.1.5.|language||ext_storage_exists> + |D.2.5.|language||ext_default_child_storage_storage_kill> |.>>>>|> > - |E.1.5.1.Version 1 - Prototype + |D.2.5.1.Version 1 - Prototype |.>>>>|> > - |E.1.6.|language||ext_storage_clear_prefix> + |D.2.6.|language||ext_default_child_storage_exists> |.>>>>|> > - |E.1.6.1.Version 1 - Prototype + |D.2.6.1.Version 1 - Prototype |.>>>>|> > - |E.1.7.|language||ext_storage_append> + |D.2.7.|language||ext_default_child_storage_clear_prefix> |.>>>>|> > - |E.1.7.1.Version 1 - Prototype + |D.2.7.1.Version 1 - Prototype |.>>>>|> > - |E.1.8.|language||ext_storage_root> + |D.2.8.|language||ext_default_child_storage_root> |.>>>>|> > - |E.1.8.1.Version 1 - Prototype + |D.2.8.1.Version 1 - Prototype |.>>>>|> > - |E.1.9.|language||ext_storage_changes_root> + |D.2.9.|language||ext_default_child_storage_next_key> |.>>>>|> > - |E.1.9.1.Version 1 - Prototype + |D.2.9.1.Version 1 - Prototype |.>>>>|> > - |E.1.10.|language||ext_storage_next_key> - |.>>>>|> - > - - |E.1.10.1.Version 1 - Prototype - |.>>>>|> - > - - |E.1.11.|language||ext_storage_start_transaction> - |.>>>>|> - > + D.3.Crypto |.>>>>|> + - |E.1.11.1.Version 1 - Prototype + |D.3.1.|language||ext_crypto_ed25519_public_keys> |.>>>>|> > - |E.1.12.|language||ext_storage_rollback_transaction> + |D.3.1.1.Version 1 - Prototype |.>>>>|> > - |E.1.12.1.Version 1 - Prototype + |D.3.2.|language||ext_crypto_ed25519_generate> |.>>>>|> > - |E.1.13.|language||ext_storage_commit_transaction> + |D.3.2.1.Version 1 - Prototype |.>>>>|> > - |E.1.13.1.Version 1 - Prototype + |D.3.3.|language||ext_crypto_ed25519_sign> |.>>>>|> > - E.2.Child Storage |.>>>>|> - + |D.3.3.1.Version 1 - Prototype + |.>>>>|> + > - |E.2.1.|language||ext_default_child_storage_set> + |D.3.4.|language||ext_crypto_ed25519_verify> |.>>>>|> > - |E.2.1.1.Version 1 - Prototype + |D.3.4.1.Version 1 - Prototype |.>>>>|> > - |E.2.2.|language||ext_default_child_storage_get> + |D.3.5.|language||ext_crypto_sr25519_public_keys> |.>>>>|> > - |E.2.2.1.Version 1 - Prototype + |D.3.5.1.Version 1 - Prototype |.>>>>|> > - |E.2.3.|language||ext_default_child_storage_read> + |D.3.6.|language||ext_crypto_sr25519_generate> |.>>>>|> > - |E.2.3.1.Version 1 - Prototype + |D.3.6.1.Version 1 - Prototype |.>>>>|> > - |E.2.4.|language||ext_default_child_storage_clear> + |D.3.7.|language||ext_crypto_sr25519_sign> |.>>>>|> > - |E.2.4.1.Version 1 - Prototype + |D.3.7.1.Version 1 - Prototype |.>>>>|> > - |E.2.5.|language||ext_default_child_storage_storage_kill> + |D.3.8.|language||ext_crypto_sr25519_verify> |.>>>>|> > - |E.2.5.1.Version 1 - Prototype + |D.3.8.1.Version 2 - Prototype |.>>>>|> > - |E.2.6.|language||ext_default_child_storage_exists> + |D.3.8.2.Version 1 - Prototype |.>>>>|> > - |E.2.6.1.Version 1 - Prototype + |D.3.9.|language||ext_crypto_ecdsa_public_keys> |.>>>>|> > - |E.2.7.|language||ext_default_child_storage_clear_prefix> + |D.3.9.1.Version 1 - Prototype |.>>>>|> > - |E.2.7.1.Version 1 - Prototype + |D.3.10.|language||ext_crypto_ecdsa_generate> |.>>>>|> > - |E.2.8.|language||ext_default_child_storage_root> + |D.3.10.1.Version 1 - Prototype |.>>>>|> > - |E.2.8.1.Version 1 - Prototype + |D.3.11.|language||ext_crypto_ecdsa_sign> |.>>>>|> > - |E.2.9.|language||ext_default_child_storage_next_key> + |D.3.11.1.Version 1 - Prototype |.>>>>|> > - |E.2.9.1.Version 1 - Prototype + |D.3.12.|language||ext_crypto_ecdsa_verify> |.>>>>|> > - E.3.Crypto |.>>>>|> - + |D.3.12.1.Version 1 - Prototype + |.>>>>|> + > + + |D.3.13.|language||ext_crypto_secp256k1_ecdsa_recover> + |.>>>>|> + > + + |D.3.13.1.Version 1 - Prototype + |.>>>>|> + > - |E.3.1.|language||ext_crypto_ed25519_public_keys> + |D.3.14.|language||ext_crypto_secp256k1_ecdsa_recover_compressed> |.>>>>|> > - |E.3.1.1.Version 1 - Prototype + |D.3.14.1.Version 1 - Prototype |.>>>>|> > - |E.3.2.|language||ext_crypto_ed25519_generate> + |D.3.15.|language||ext_crypto_start_batch_verify> |.>>>>|> > - |E.3.2.1.Version 1 - Prototype + |D.3.15.1.Version 1 - Prototype |.>>>>|> > - |E.3.3.|language||ext_crypto_ed25519_sign> + |D.3.16.|language||ext_crypto_finish_batch_verify> |.>>>>|> > - |E.3.3.1.Version 1 - Prototype + |D.3.16.1.Version 1 - Prototype |.>>>>|> > - |E.3.4.|language||ext_crypto_ed25519_verify> - |.>>>>|> - > + D.4.Hashing |.>>>>|> + - |E.3.4.1.Version 1 - Prototype + |D.4.1.|language||ext_hashing_keccak_256> |.>>>>|> > - |E.3.5.|language||ext_crypto_sr25519_public_keys> + |D.4.1.1.Version 1 - Prototype |.>>>>|> > - |E.3.5.1.Version 1 - Prototype + |D.4.2.|language||ext_hashing_sha2_256> |.>>>>|> > - |E.3.6.|language||ext_crypto_sr25519_generate> + |D.4.2.1.Version 1 - Prototype |.>>>>|> > - |E.3.6.1.Version 1 - Prototype + |D.4.3.|language||ext_hashing_blake2_128> |.>>>>|> > - |E.3.7.|language||ext_crypto_sr25519_sign> + |D.4.3.1.Version 1 - Prototype |.>>>>|> > - |E.3.7.1.Version 1 - Prototype + |D.4.4.|language||ext_hashing_blake2_256> |.>>>>|> > - |E.3.8.|language||ext_crypto_sr25519_verify> + |D.4.4.1.Version 1 - Prototype |.>>>>|> > - |E.3.8.1.Version 2 - Prototype + |D.4.5.|language||ext_hashing_twox_64> |.>>>>|> > - |E.3.8.2.Version 1 - Prototype + |D.4.5.1.Version 1 - Prototype |.>>>>|> > - |E.3.9.|language||ext_crypto_ecdsa_public_keys> + |D.4.6.|language||ext_hashing_twox_128> |.>>>>|> > - |E.3.9.1.Version 1 - Prototype + |D.4.6.1.Version 1 - Prototype |.>>>>|> > - |E.3.10.|language||ext_crypto_ecdsa_generate> + |D.4.7.|language||ext_hashing_twox_256> |.>>>>|> > - |E.3.10.1.Version 1 - Prototype + |D.4.7.1.Version 1 - Prototype |.>>>>|> > - |E.3.11.|language||ext_crypto_ecdsa_sign> - |.>>>>|> - > - - |E.3.11.1.Version 1 - Prototype - |.>>>>|> - > + D.5.Offchain |.>>>>|> + - |E.3.12.|language||ext_crypto_ecdsa_verify> + |D.5.1.|language||ext_offchain_is_validator> |.>>>>|> > - |E.3.12.1.Version 1 - Prototype + |D.5.1.1.Version 1 - Prototype |.>>>>|> > - |E.3.13.|language||ext_crypto_secp256k1_ecdsa_recover> + |D.5.2.|language||ext_offchain_submit_transaction> |.>>>>|> > - |E.3.13.1.Version 1 - Prototype + |D.5.2.1.Version 1 - Prototype |.>>>>|> > - |E.3.14.|language||ext_crypto_secp256k1_ecdsa_recover_compressed> + |D.5.3.|language||ext_offchain_network_state> |.>>>>|> > - |E.3.14.1.Version 1 - Prototype + |D.5.3.1.Version 1 - Prototype |.>>>>|> > - |E.3.15.|language||ext_crypto_start_batch_verify> + |D.5.4.|language||ext_offchain_timestamp> |.>>>>|> > - |E.3.15.1.Version 1 - Prototype + |D.5.4.1.Version 1 - Prototype |.>>>>|> > - |E.3.16.|language||ext_crypto_finish_batch_verify> + |D.5.5.|language||ext_offchain_sleep_until> |.>>>>|> > - |E.3.16.1.Version 1 - Prototype + |D.5.5.1.Version 1 - Prototype |.>>>>|> > - E.4.Hashing |.>>>>|> - + |D.5.6.|language||ext_offchain_random_seed> + |.>>>>|> + > - |E.4.1.|language||ext_hashing_keccak_256> + |D.5.6.1.Version 1 - Prototype |.>>>>|> > - |E.4.1.1.Version 1 - Prototype + |D.5.7.|language||ext_offchain_local_storage_set> |.>>>>|> > - |E.4.2.|language||ext_hashing_sha2_256> + |D.5.7.1.Version 1 - Prototype |.>>>>|> > - |E.4.2.1.Version 1 - Prototype + |D.5.8.|language||ext_offchain_local_storage_clear> |.>>>>|> > - |E.4.3.|language||ext_hashing_blake2_128> + |D.5.8.1.Version 1 - Prototype |.>>>>|> > - |E.4.3.1.Version 1 - Prototype + |D.5.9.|language||ext_offchain_local_storage_compare_and_set> |.>>>>|> > - |E.4.4.|language||ext_hashing_blake2_256> + |D.5.9.1.Version 1 - Prototype |.>>>>|> > - |E.4.4.1.Version 1 - Prototype + |D.5.10.|language||ext_offchain_local_storage_get> |.>>>>|> > - |E.4.5.|language||ext_hashing_twox_64> + |D.5.10.1.Version 1 - Prototype |.>>>>|> > - |E.4.5.1.Version 1 - Prototype + |D.5.11.|language||ext_offchain_http_request_start> |.>>>>|> > - |E.4.6.|language||ext_hashing_twox_128> + |D.5.11.1.Version 1 - Prototype |.>>>>|> > - |E.4.6.1.Version 1 - Prototype + |D.5.12.|language||ext_offchain_http_request_add_header> |.>>>>|> > - |E.4.7.|language||ext_hashing_twox_256> + |D.5.12.1.Version 1 - Prototype |.>>>>|> > - |E.4.7.1.Version 1 - Prototype + |D.5.13.|language||ext_offchain_http_request_write_body> |.>>>>|> > - E.5.Offchain |.>>>>|> - + |D.5.13.1.Version 1 - Prototype + |.>>>>|> + > + + |D.5.14.|language||ext_offchain_http_response_wait> + |.>>>>|> + > - |E.5.1.|language||ext_offchain_is_validator> + |D.5.14.1.Version 1- Prototype |.>>>>|> > - |E.5.1.1.Version 1 - Prototype + |D.5.15.|language||ext_offchain_http_response_headers> |.>>>>|> > - |E.5.2.|language||ext_offchain_submit_transaction> + |D.5.15.1.Version 1 - Prototype |.>>>>|> > - |E.5.2.1.Version 1 - Prototype + |D.5.16.|language||ext_offchain_http_response_read_body> |.>>>>|> > - |E.5.3.|language||ext_offchain_network_state> + |D.5.16.1.Version 1 - Prototype |.>>>>|> > - |E.5.3.1.Version 1 - Prototype + |D.5.17.|language||ext_offchain_set_authorized_nodes> |.>>>>|> > - |E.5.4.|language||ext_offchain_timestamp> + |D.5.17.1.Version 1 - Prototype |.>>>>|> > - |E.5.4.1.Version 1 - Prototype - |.>>>>|> - > + D.6.Trie |.>>>>|> + - |E.5.5.|language||ext_offchain_sleep_until> + |D.6.1.|language||ext_trie_blake2_256_root> |.>>>>|> > - |E.5.5.1.Version 1 - Prototype + |D.6.1.1.Version 1 - Prototype |.>>>>|> > - |E.5.6.|language||ext_offchain_random_seed> + |D.6.2.|language||ext_trie_blake2_256_ordered_root> |.>>>>|> > - |E.5.6.1.Version 1 - Prototype + |D.6.2.1.Version 1 - Prototype |.>>>>|> > - |E.5.7.|language||ext_offchain_local_storage_set> + |D.6.3.|language||ext_trie_keccak_256_root> |.>>>>|> > - |E.5.7.1.Version 1 - Prototype + |D.6.3.1.Version 1 - Prototype |.>>>>|> > - |E.5.8.|language||ext_offchain_local_storage_compare_and_set> + |D.6.4.|language||ext_trie_keccak_256_ordered_root> |.>>>>|> > - |E.5.8.1.Version 1 - Prototype + |D.6.4.1.Version 1 - Prototype |.>>>>|> > - |E.5.9.|language||ext_offchain_local_storage_get> - |.>>>>|> - > + D.7.Miscellaneous |.>>>>|> + - |E.5.9.1.Version 1 - Prototype + |D.7.1.|language||ext_misc_chain_id> |.>>>>|> > - |E.5.10.|language||ext_offchain_http_request_start> + |D.7.1.1.Version 1 - Prototype |.>>>>|> > - |E.5.10.1.Version 1 - Prototype + |D.7.2.|language||ext_misc_print_num> |.>>>>|> > - |E.5.11.|language||ext_offchain_http_request_add_header> + |D.7.2.1.Version 1 - Prototype |.>>>>|> > - |E.5.11.1.Version 1 - Prototype + |D.7.3.|language||ext_misc_print_utf8> |.>>>>|> > - |E.5.12.|language||ext_offchain_http_request_write_body> + |D.7.3.1.Version 1 - Prototype |.>>>>|> > - |E.5.12.1.Version 1 - Prototype + |D.7.4.|language||ext_misc_print_hex> |.>>>>|> > - |E.5.13.|language||ext_offchain_http_response_wait> + |D.7.4.1.Version 1 - Prototype |.>>>>|> > - |E.5.13.1.Version 1- Prototype + |D.7.5.|language||ext_misc_runtime_version> |.>>>>|> > - |E.5.14.|language||ext_offchain_http_response_headers> + |D.7.5.1.Version 1 - Prototype |.>>>>|> > - |E.5.14.1.Version 1 - Prototype - |.>>>>|> - > + D.8.Allocator |.>>>>|> + - |E.5.15.|language||ext_offchain_http_response_read_body> + |D.8.1.|language||ext_allocator_malloc> |.>>>>|> > - |E.5.15.1.Version 1 - Prototype + |D.8.1.1.Version 1 - Prototype |.>>>>|> > - E.6.Trie |.>>>>|> - - - |E.6.1.|language||ext_trie_blake2_256_root> + |D.8.2.|language||ext_allocator_free> |.>>>>|> - > + > - |E.6.1.1.Version 1 - Prototype + |D.8.2.1.Version 1 - Prototype |.>>>>|> - > + > - |E.6.2.|language||ext_trie_blake2_256_ordered_root> - |.>>>>|> - > + D.9.Logging |.>>>>|> + - |E.6.2.1.Version 1 - Prototype + |D.9.1.|language||ext_logging_log> |.>>>>|> > - |E.6.3.|language||ext_trie_keccak_256_root> + |D.9.1.1.Version 1 - Prototype |.>>>>|> > - |E.6.3.1.Version 1 - Prototype + |math-font-series||font-shape||Appendix + E.Legacy Polkadot Host API> |.>>>>|> - > + - |E.6.4.|language||ext_trie_keccak_256_ordered_root> - |.>>>>|> - > + E.1.Storage |.>>>>|> + - |E.6.4.1.Version 1 - Prototype + |E.1.1.|language||ext_set_storage> |.>>>>|> > - E.7.Miscellaneous |.>>>>|> - + |E.1.2.|language||ext_storage_root> + |.>>>>|> + > - |E.7.1.|language||ext_misc_chain_id> + |E.1.3.|language||ext_blake2_256_enumerated_trie_root> |.>>>>|> > - |E.7.1.1.Version 1 - Prototype + |E.1.4.|language||ext_clear_prefix> |.>>>>|> > - |E.7.2.|language||ext_misc_print_num> + |E.1.5.|language||>|language||ext_clear_storage> |.>>>>|> > - |E.7.2.1.Version 1 - Prototype + |E.1.6.|language||ext_exists_storage> |.>>>>|> > - |E.7.3.|language||ext_misc_print_utf8> + |E.1.7.|language||ext_get_allocated_storage> |.>>>>|> > - |E.7.3.1.Version 1 - Prototype + |E.1.8.|language||ext_get_storage_into> |.>>>>|> > - |E.7.4.|language||ext_misc_print_hex> + |E.1.9.|language||ext_set_child_storage> |.>>>>|> > - |E.7.4.1.Version 1 - Prototype + |E.1.10.|language||ext_clear_child_storage> |.>>>>|> > - |E.7.5.|language||ext_misc_runtime_version> + |E.1.11.|language||ext_exists_child_storage> |.>>>>|> > - |E.7.5.1.Version 1 - Prototype + |E.1.12.|language||ext_get_allocated_child_storage> |.>>>>|> > - E.8.Allocator |.>>>>|> - + |E.1.13.|language||ext_get_child_storage_into> + |.>>>>|> + > - |E.8.1.|language||ext_allocator_malloc> + |E.1.14.|language||ext_kill_child_storage> |.>>>>|> > - |E.8.1.1.Version 1 - Prototype + |E.1.15.Memory |.>>>>|> > - |E.8.2.|language||ext_allocator_free> + |E.1.15.1.|language||ext_malloc> |.>>>>|> > - |E.8.2.1.Version 1 - Prototype + |E.1.15.2.|language||ext_free> |.>>>>|> > - E.9.Logging |.>>>>|> - + |E.1.15.3.Input/Output + |.>>>>|> + > + + |E.1.16.Cryptograhpic Auxiliary + Functions |.>>>>|> + > - |E.9.1.|language||ext_logging_log> + |E.1.16.1.|language||ext_blake2_256> |.>>>>|> > - |E.9.1.1.Version 1 - Prototype + |E.1.16.2.|language||ext_keccak_256> |.>>>>|> > - |math-font-series||font-shape||Appendix - F.Legacy Polkadot Host API> + |E.1.16.3.|language||ext_twox_128> |.>>>>|> - + > - F.1.Storage |.>>>>|> - + |E.1.16.4.|language||ext_ed25519_verify> + |.>>>>|> + > - |F.1.1.|language||ext_set_storage> + |E.1.16.5.|language||ext_sr25519_verify> |.>>>>|> > - |F.1.2.|language||ext_storage_root> + |E.1.16.6.To be Specced |.>>>>|> > - |F.1.3.|language||ext_blake2_256_enumerated_trie_root> - |.>>>>|> + |E.1.17.Offchain Worker + \ |.>>>>|> > - |F.1.4.|language||ext_clear_prefix> + |E.1.17.1.|language||ext_is_validator> |.>>>>|> > - |F.1.5.|language||>|language||ext_clear_storage> + |E.1.17.2.|language||ext_submit_transaction> |.>>>>|> > - |F.1.6.|language||ext_exists_storage> + |E.1.17.3.|language||ext_network_state> |.>>>>|> > - |F.1.7.|language||ext_get_allocated_storage> + |E.1.17.4.|language||ext_timestamp> |.>>>>|> > - |F.1.8.|language||ext_get_storage_into> + |E.1.17.5.|language||ext_sleep_until> |.>>>>|> > - |F.1.9.|language||ext_set_child_storage> + |E.1.17.6.|language||ext_random_seed> |.>>>>|> > - |F.1.10.|language||ext_clear_child_storage> + |E.1.17.7.|language||ext_local_storage_set> |.>>>>|> > - |F.1.11.|language||ext_exists_child_storage> + |E.1.17.8.|language||ext_local_storage_compare_and_set> |.>>>>|> > - |F.1.12.|language||ext_get_allocated_child_storage> + |E.1.17.9.|language||ext_local_storage_get> |.>>>>|> > - |F.1.13.|language||ext_get_child_storage_into> + |E.1.17.10.|language||ext_http_request_start> |.>>>>|> > - |F.1.14.|language||ext_kill_child_storage> + |E.1.17.11.|language||ext_http_request_add_header> |.>>>>|> > - |F.1.15.Memory + |E.1.17.12.|language||ext_http_request_write_body> |.>>>>|> > - |F.1.15.1.|language||ext_malloc> + |E.1.17.13.|language||ext_http_response_wait> |.>>>>|> > - |F.1.15.2.|language||ext_free> + |E.1.17.14.|language||ext_http_response_headers> |.>>>>|> > - |F.1.15.3.Input/Output + |E.1.17.15.|language||ext_http_response_read_body> |.>>>>|> > - |F.1.16.Cryptograhpic Auxiliary - Functions |.>>>>|> + |E.1.18.Sandboxing + |.>>>>|> > - |F.1.16.1.|language||ext_blake2_256> + |E.1.18.1.To be Specced |.>>>>|> > - |F.1.16.2.|language||ext_keccak_256> + |E.1.19.Auxillary Debugging API |.>>>>|> > - |F.1.16.3.|language||ext_twox_128> + |E.1.19.1.|language||ext_print_hex> |.>>>>|> > - |F.1.16.4.|language||ext_ed25519_verify> + |E.1.19.2.|language||ext_print_utf8> |.>>>>|> > - |F.1.16.5.|language||ext_sr25519_verify> + |E.1.20.Misc |.>>>>|> > - |F.1.16.6.To be Specced + |E.1.20.1.To be Specced |.>>>>|> > - |F.1.17.Offchain Worker - \ |.>>>>|> + |E.1.21.Block Production + |.>>>>|> > - |F.1.17.1.|language||ext_is_validator> + E.2.Validation |.>>>>|> + + + |math-font-series||font-shape||Appendix + F.Runtime Entries> |.>>>>|> + + + F.1.List of Runtime Entries |.>>>>|> - > + - |F.1.17.2.|language||ext_submit_transaction> + F.2.JSON-RPC API for external services |.>>>>|> - > + - |F.1.17.3.|language||ext_network_state> + F.3.Argument Specification |.>>>>|> - > + - |F.1.17.4.|language||ext_timestamp> + |F.3.1.|language||Core_version> |.>>>>|> - > + > - |F.1.17.5.|language||ext_sleep_until> + |F.3.2.|language||Core_execute_block> |.>>>>|> - > + > - |F.1.17.6.|language||ext_random_seed> + |F.3.3.|language||Core_initialize_block> |.>>>>|> - > + > - |F.1.17.7.|language||ext_local_storage_set> + |F.3.4.|language||Metadata_metadata> |.>>>>|> - > + > - |F.1.17.8.|language||ext_local_storage_compare_and_set> + |F.3.5.|language||BlockBuilder_apply_extrinsic> |.>>>>|> - > + > - |F.1.17.9.|language||ext_local_storage_get> + |F.3.6.|language||BlockBuilder_finalize_block> |.>>>>|> - > + > - |F.1.17.10.|language||ext_http_request_start> + |F.3.7.|language||BlockBuilder_inherent_extrinsics> |.>>>>|> - > + > - |F.1.17.11.|language||ext_http_request_add_header> + |F.3.8.|language||BlockBuilder_check_inherents> |.>>>>|> - > + > - |F.1.17.12.|language||ext_http_request_write_body> + |F.3.9.|language||BlockBuilder_random_seed> |.>>>>|> - > + > + + |F.3.10.|language||TaggedTransactionQueue_validate_transaction> + |.>>>>|> + > - |F.1.17.13.|language||ext_http_response_wait> + |F.3.11.|language||OffchainWorkerApi_offchain_worker> |.>>>>|> - > + > - |F.1.17.14.|language||ext_http_response_headers> + |F.3.12.|language||ParachainHost_validators> |.>>>>|> - > + > - |F.1.17.15.|language||ext_http_response_read_body> + |F.3.13.|language||ParachainHost_validator_groups> |.>>>>|> - > + > - |F.1.18.Sandboxing + |F.3.14.|language||ParachainHost_availability_cores> |.>>>>|> - > + > - |F.1.18.1.To be Specced + |F.3.15.|language||ParachainHost_persisted_validation_data> |.>>>>|> - > + > - |F.1.19.Auxillary Debugging API + |F.3.16.|language||ParachainHost_check_validation_outputs> |.>>>>|> - > + > - |F.1.19.1.|language||ext_print_hex> + |F.3.17.|language||ParachainHost_session_index_for_child> |.>>>>|> - > + > - |F.1.19.2.|language||ext_print_utf8> + |F.3.18.|language||ParachainHost_session_info> |.>>>>|> - > + > - |F.1.20.Misc + |F.3.19.|language||ParachainHost_validation_code> |.>>>>|> - > + > - |F.1.20.1.To be Specced + |F.3.20.|language||ParachainHost_historical_validation_code> |.>>>>|> - > + > - |F.1.21.Block Production + |F.3.21.|language||ParachainHost_candidate_pending_availability> |.>>>>|> - > + > - F.2.Validation |.>>>>|> - + |F.3.22.|language||ParachainHost_candidate_events> + |.>>>>|> + > - |math-font-series||font-shape||Appendix - G.Runtime Entries> |.>>>>|> - + |F.3.23.|language||ParachainHost_dmq_contents> + |.>>>>|> + > - G.1.List of Runtime Entries + |F.3.24.|language||ParachainHost_inbound_hrmp_channel_contents> |.>>>>|> - + > - G.2.Argument Specification + |F.3.25.|language||GrandpaApi_grandpa_authorities> |.>>>>|> - + > - |G.2.1.|language||Core_version> + |F.3.26.|language||GrandpaApi_submit_report_equivocation_unsigned_extrinsic> |.>>>>|> - > + > - |G.2.2.|language||Core_execute_block> + |F.3.27.|language||GrandpaApi_generate_key_ownership_proof> |.>>>>|> - > + > - |G.2.3.|language||Core_initialize_block> + |F.3.28.|language||BabeApi_configuration> |.>>>>|> - > + > - |G.2.4.|language||hash_and_length> + |F.3.29.|language||BabeApi_current_epoch_start> |.>>>>|> - > + > - |G.2.5.|language||BabeApi_configuration> + |F.3.30.|language||BabeApi_current_epoch> |.>>>>|> - > + > - |G.2.6.|language||GrandpaApi_grandpa_authorities> + |F.3.31.|language||BabeApi_next_epoch> |.>>>>|> - > + > - |G.2.7.|language||TaggedTransactionQueue_validate_transaction> + |F.3.32.|language||BabeApi_generate_key_ownership_proof> |.>>>>|> - > + > - |G.2.8.|language||BlockBuilder_apply_extrinsic> + |F.3.33.|language||BabeApi_submit_report_equivocation_unsigned_extrinsic> |.>>>>|> - > + > - |G.2.9.|language||BlockBuilder_inherent_extrinsics> + |F.3.34.|language||AuthorityDiscoveryApi_authorities> |.>>>>|> > - |G.2.10.|language||BlockBuilder_finalize_block> + |F.3.35.|language||SessionKeys_generate_session_keys> |.>>>>|> > + |F.3.36.|language||SessionKeys_decode_session_keys> + |.>>>>|> + > + + |F.3.37.|language||AccountNonceApi_account_nonce> + |.>>>>|> + > + + |F.3.38.|language||TransactionPaymentApi_query_info> + |.>>>>|> + > + + |F.3.39.|language||TransactionPaymentApi_query_fee_details> + |.>>>>|> + > + |math-font-series||font-shape||Glossary> |.>>>>|> - + |math-font-series||font-shape||Bibliography> |.>>>>|> - + |math-font-series||font-shape||Index> |.>>>>|> - + \ No newline at end of file From c957e2d74d4074a7078b302588a175daf1dc3b51 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Thu, 4 Feb 2021 18:12:51 +0100 Subject: [PATCH 64/65] fix small mistakes pointed out by Florian --- host-spec/ae-hostapi.tm | 162 +++++++++++++++++++++--------------- host-spec/c04-networking.tm | 6 +- 2 files changed, 96 insertions(+), 72 deletions(-) diff --git a/host-spec/ae-hostapi.tm b/host-spec/ae-hostapi.tm index 78a961ecf..717e9fc18 100644 --- a/host-spec/ae-hostapi.tm +++ b/host-spec/ae-hostapi.tm @@ -1,4 +1,4 @@ - + @@ -1304,10 +1304,34 @@ : a pointer-size as defined in Definition indicating the data to be hashed. - : a reglar pointer to the buffer containing the + : a regular pointer to the buffer containing the 256-bit hash result. + > + + Conducts a 512-bit Keccak hash. + + + + <\verbatim> + (func $ext_hashing_keccak_512_version_1 + + \ \ (param $data i64) (return i32)) + + + \; + + : + + <\itemize> + : a pointer-size as defined in Definition + indicating the data to be hashed. + + : a regular pointer to the buffer containing the + 512-bit hash result. + + > Conducts a 256-bit Sha2 hash. @@ -2399,7 +2423,7 @@ <\initial> <\collection> - + @@ -2414,76 +2438,78 @@ > > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > > - > - > - > - > + > + > + > + > + > + > > > > @@ -2574,7 +2600,7 @@ > > > - > + > > > > diff --git a/host-spec/c04-networking.tm b/host-spec/c04-networking.tm index 97e70f48a..73cf6f9fc 100644 --- a/host-spec/c04-networking.tm +++ b/host-spec/c04-networking.tm @@ -452,7 +452,7 @@ Transactions as defined and described in Section - are sent directly to peers with which the + are sent directly to peers with which the Polkadot Host has an open transaction substream, as defined in Definition . Polkadot Host implementers should implement a mechanism which only sends a transaction once to each peer and @@ -460,9 +460,7 @@ undefined consequences such as being blocked for bad behavior by peers. The mechanism for managing transactions is further described in Section - respectively Section - and Section - . + . <\definition> The is the From 3a4eb99f776b32ec42d9f0b1d82d9efa5678a294 Mon Sep 17 00:00:00 2001 From: lamafab <42901763+lamafab@users.noreply.github.com> Date: Thu, 4 Feb 2021 18:16:07 +0100 Subject: [PATCH 65/65] rename 'regular pointer' to '32-bit pointer' --- host-spec/ae-hostapi.tm | 73 ++++++++++++++++++++--------------------- 1 file changed, 36 insertions(+), 37 deletions(-) diff --git a/host-spec/ae-hostapi.tm b/host-spec/ae-hostapi.tm index 717e9fc18..9421acffc 100644 --- a/host-spec/ae-hostapi.tm +++ b/host-spec/ae-hostapi.tm @@ -259,7 +259,7 @@ : <\itemize> - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit Blake2 storage root. @@ -286,7 +286,7 @@ Definition indicating the SCALE encoded block hash. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit Blake2 changes root. @@ -757,7 +757,7 @@ as defined in Definition containing the BIP-39 seed which must be valid UTF8. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit public key. @@ -783,7 +783,7 @@ : an i32 integer indicating the key type ID as defined in Definition . - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit public key. : a pointer-size as defined in Definition @@ -826,14 +826,14 @@ : <\itemize> - : a regular pointer to the buffer containing + : a 32-bit pointer to the buffer containing the 64-byte signature. : a pointer-size as defined in Definition indicating the message that is to be verified. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit public key. : a i32 integer value equal to if the @@ -898,7 +898,7 @@ as defined in Definition containing the BIP-39 seed which must be valid UTF8. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit public key. @@ -924,7 +924,7 @@ : an i32 integer containg the key ID as defined in Definition - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit public key. : a pointer-size as defined in Definition @@ -974,14 +974,14 @@ : <\itemize> - : a regular pointer to the buffer containing + : a 32-bit pointer to the buffer containing the 64-byte signature. : a pointer-size as defined in Definition indicating the message that is to be verified. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit public key. : a i32 integer value equal to if the @@ -1001,14 +1001,14 @@ : <\itemize> - : a regular pointer to the buffer containing + : a 32-bit pointer to the buffer containing the 64-byte signature. : a pointer-size as defined in Definition indicating the message that is to be verified. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit public key. : a i32 integer value equal to if the @@ -1072,7 +1072,7 @@ as defined in Definition containing the BIP-39 seed which must be valid UTF8. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 33-byte compressed public key. @@ -1098,7 +1098,7 @@ : an i32 integer containg the key ID as defined in Definition - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 33-byte compressed public key. : a pointer-size as defined in Definition @@ -1143,7 +1143,7 @@ : <\itemize> - : a regular pointer to the buffer containing + : a 32-bit pointer to the buffer containing the 65-byte signature. The signature is 65-bytes in size, where the first 512-bits represent the signature and the other 8 bits represent the recovery ID. @@ -1152,7 +1152,7 @@ indicating the message that is to be verified. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 33-byte compressed public key. : a i32 integer value equal to if the @@ -1176,11 +1176,11 @@ : <\itemize> - : a regular pointer to the buffer containing + : a 32-bit pointer to the buffer containing the 65-byte signature in RSV format. V should be either or . - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit Blake2 hash of the message. : a pointer-size as defined in Definition @@ -1207,11 +1207,11 @@ : <\itemize> - : a regular pointer to the buffer containing + : a 32-bit pointer to the buffer containing the 65-byte signature in RSV format. V should be either or . - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit Blake2 hash of the message. : a pointer-size as defined in Definition @@ -1304,7 +1304,7 @@ : a pointer-size as defined in Definition indicating the data to be hashed. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit hash result. @@ -1328,7 +1328,7 @@ : a pointer-size as defined in Definition indicating the data to be hashed. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 512-bit hash result. @@ -1352,7 +1352,7 @@ : a pointer-size as defined in Definition indicating the data to be hashed. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit hash result. @@ -1376,7 +1376,7 @@ : a pointer-size as defined in Definition indicating the data to be hashed. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 128-bit hash result. @@ -1400,7 +1400,7 @@ : a pointer-size as defined in Definition indicating the data to be hashed. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit hash result. @@ -1424,7 +1424,7 @@ : a pointer-size as defined in Definition indicating the data to be hashed. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 64-bit hash result. @@ -1448,7 +1448,7 @@ : a pointer-size as defined in Definition indicating the data to be hashed. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 128-bit hash result. @@ -1472,7 +1472,7 @@ : a pointer-size as defined in Definition indicating the data to be hashed. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit hash result. @@ -1704,9 +1704,8 @@ : <\itemize> - : a regular pointer to the buffer containing the 256-bit - seed. + : a 32-bit pointer to the buffer containing the + 256-bit seed. > @@ -2117,7 +2116,7 @@ the trie root gets formed. The items consist of a SCALE encoded array containing arbitrary key/value pairs. - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit trie root. @@ -2146,7 +2145,7 @@ compact encoded integers as described in Definition . - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit trie root result. @@ -2172,7 +2171,7 @@ the trie root gets formed. The items consist of a SCALE encoded array containing arbitrary key/value pairs. - result: a regular pointer to the buffer containing the 256-bit trie + result: a 32-bit pointer to the buffer containing the 256-bit trie root. @@ -2201,7 +2200,7 @@ compact encoded integers as described in Definition . - : a regular pointer to the buffer containing the + : a 32-bit pointer to the buffer containing the 256-bit trie root result. @@ -2342,7 +2341,7 @@ <\itemize> : the size of the buffer to be allocated. - : a regular pointer to the allocated buffer. + : a 32-bit pointer to the allocated buffer. > @@ -2360,7 +2359,7 @@ : <\itemize> - : a regular pointer to the memory buffer to be freed. + : a 32-bit pointer to the memory buffer to be freed.