Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Network specification update #1404

Merged
merged 10 commits into from
Sep 17, 2019
Merged

Network specification update #1404

Changes from 3 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
4937fa9
Network specification update
AgeManning Sep 8, 2019
3a79ad5
Adds chunked responses to the RPC
AgeManning Sep 8, 2019
acb86e8
Apply Danny's suggestions
AgeManning Sep 8, 2019
cc12e29
cleanup response_chunk refactor
djrtwo Sep 8, 2019
b743deb
cleanup max size vars
djrtwo Sep 8, 2019
3ead898
p2p spec copy cleanups
djrtwo Sep 8, 2019
4a7d8a4
Applies github suggestions
AgeManning Sep 10, 2019
9fa720b
Removes a max chunk count and corrects timeout for chunked responses
AgeManning Sep 11, 2019
8bb9354
Renames REQ_RESP_MAX_SIZE to MAX_CHUNK_SIZE
AgeManning Sep 12, 2019
4b2596d
ensure BeaconBlocksByRoot requests are lists rather than containers
djrtwo Sep 16, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 53 additions & 26 deletions specs/networking/p2p-interface.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ This section outlines constants that are used in this spec.

| Name | Value | Description |
|---|---|---|
| `REQ_RESP_MAX_SIZE` | `TODO` | The maximum size of uncompressed req/resp messages that clients will allow. |
| `REQ_RESP_MAX_SIZE` | `2**20` (1048576, 1 MiB) | The maximum size of uncompressed req/resp messages that clients will allow. |
| `SSZ_MAX_LIST_SIZE` | `TODO` | The maximum size of SSZ-encoded variable lists. |
| `GOSSIP_MAX_SIZE` | `2**20` (= 1048576, 1 MiB) | The maximum size of uncompressed gossip messages. |
| `SHARD_SUBNET_COUNT` | `TODO` | The number of shard subnets used in the gossipsub protocol. |
Expand Down Expand Up @@ -231,23 +231,24 @@ Request/response messages MUST adhere to the encoding specified in the protocol

```
request ::= <encoding-dependent-header> | <encoded-payload>
response ::= <result> | <encoding-dependent-header> | <encoded-payload>
response ::= <response_chunk>+
response_chunk ::= <result> | <encoding-dependent-header> | <encoded-payload>
djrtwo marked this conversation as resolved.
Show resolved Hide resolved
result ::= “0” | “1” | “2” | [“128” ... ”255”]
```

The encoding-dependent header may carry metadata or assertions such as the encoded payload length, for integrity and attack proofing purposes. Because req/resp streams are single-use and stream closures implicitly delimit the boundaries, it is not strictly necessary to length-prefix payloads; however, certain encodings like SSZ do, for added security.

`encoded-payload` has a maximum byte size of `REQ_RESP_MAX_SIZE`.
A `response` is formed by one or more `response_chunk`. The exact request determines whether a response consists of a single `response_chunk` or many. Responses that consist of a single SSZ-list of objects (such as `BlocksByRange` and `BlocksByRoot`) will send each list item as a `response_chunk`. The total `response` has a maximum uncompressed byte size of `REQ_RESP_MAX_SIZE`.

Clients MUST ensure the payload size is less than or equal to `REQ_RESP_MAX_SIZE`; if not, they SHOULD reset the stream immediately. Clients tracking peer reputation MAY decrement the score of the misbehaving peer under this circumstance.
Clients MUST ensure the total `response` is less than or equal to `REQ_RESP_MAX_SIZE`; if not, they SHOULD reset the stream immediately. Clients tracking peer reputation MAY decrement the score of the misbehaving peer under this circumstance.

#### Requesting side

Once a new stream with the protocol ID for the request type has been negotiated, the full request message should be sent immediately. It should be encoded according to the encoding strategy.
Once a new stream with the protocol ID for the request type has been negotiated, the full request message should be sent immediately. It MUST be encoded according to the encoding strategy.

The requester MUST close the write side of the stream once it finishes writing the request message—at this point, the stream will be half-closed.

The requester MUST wait a maximum of `TTFB_TIMEOUT` for the first response byte to arrive (time to first byte—or TTFB—timeout). On that happening, the requester will allow further `RESP_TIMEOUT` to receive the full response.
The requester MUST wait a maximum of `TTFB_TIMEOUT` for the first response byte to arrive (time to first byte—or TTFB—timeout). On that happening, the requester will allow further `RESP_TIMEOUT` to receive the full response. For requests consisting of many `response_chunk` the requester should read from the stream until either; a) An error is received by one of the chunks, b) The responder closes the stream or c) `REQ_RESP_MAX_SIZE` bytes have been read. Requests that have a single `response_chunk` and a length-prefix, requesters can read the exact number of bytes defined by the length-prefix before closing the stream.

If any of these timeouts fire, the requester SHOULD reset the stream and deem the req/resp operation to have failed.

Expand All @@ -260,14 +261,14 @@ The responder MUST:
1. Use the encoding strategy to read the optional header.
2. If there are any length assertions for length `N`, it should read exactly `N` bytes from the stream, at which point an EOF should arise (no more bytes). Should this not be the case, it should be treated as a failure.
3. Deserialize the expected type, and process the request.
4. Write the response (result, optional header, payload).
4. Write the response which may consist of one or many `response_chunks` (result, optional header, payload).
5. Close their write side of the stream. At this point, the stream will be fully closed.

If steps (1), (2), or (3) fail due to invalid, malformed, or inconsistent data, the responder MUST respond in error. Clients tracking peer reputation MAY record such failures, as well as unexpected events, e.g. early stream resets.

The entire request should be read in no more than `RESP_TIMEOUT`. Upon a timeout, the responder SHOULD reset the stream.

The responder SHOULD send a response promptly, starting with a **single-byte** response code which determines the contents of the response (`result` particle in the BNF grammar above).
The responder SHOULD send a response promptly, starting with a **single-byte** response code which determines the contents of the `response_chunk` (`result` particle in the BNF grammar above).
AgeManning marked this conversation as resolved.
Show resolved Hide resolved

It can have one of the following values, encoded as a single unsigned byte:

Expand All @@ -289,7 +290,8 @@ The `ErrorMessage` schema is:

*Note*: The String type is encoded as UTF-8 bytes without NULL terminator when SSZ-encoded. As the `ErrorMessage` is not an SSZ-container, only the UTF-8 bytes will be sent when SSZ-encoded.

A response therefore has the form:
A response therefore has the form of 1 or more `response_chunk` which look
like:
```
+--------+--------+--------+--------+--------+--------+
| result | header (opt) | encoded_response |
Expand All @@ -301,7 +303,7 @@ Here, `result` represents the 1-byte response code.

The token of the negotiated protocol ID specifies the type of encoding to be used for the req/resp interaction. Two values are possible at this time:

- `ssz`: the contents are [SSZ-encoded](#ssz-encoding). This encoding type MUST be supported by all clients. For objects containing a single field, only the field is SSZ-encoded not a container with a single field. For example, the `BeaconBlocks` response would be an SSZ-encoded list of `BeaconBlock`s. All SSZ-Lists in the Req/Resp domain will have a maximum list size of `SSZ_MAX_LIST_SIZE`.
- `ssz`: the contents are [SSZ-encoded](../simple-serialize.md). This encoding type MUST be supported by all clients. For objects containing a single field, only the field is SSZ-encoded not a container with a single field. For example, the `BeaconBlocksByRange` response would be an SSZ-encoded list of `BeaconBlock`s. All SSZ-Lists in the Req/Resp domain will have a maximum list size of `SSZ_MAX_LIST_SIZE`.
AgeManning marked this conversation as resolved.
Show resolved Hide resolved
- `ssz_snappy`: The contents are SSZ-encoded and then compressed with [Snappy](https://github.com/google/snappy). MAY be supported in the interoperability testnet; MUST be supported in mainnet.

#### SSZ-encoding strategy (with or without Snappy)
Expand All @@ -310,18 +312,22 @@ The [SimpleSerialize (SSZ) specification](../simple-serialize.md) outlines how o

**Encoding-dependent header:** Req/Resp protocols using the `ssz` or `ssz_snappy` encoding strategies MUST prefix all encoded and compressed (if applicable) payloads with an unsigned [protobuf varint](https://developers.google.com/protocol-buffers/docs/encoding#varints).

*Note*: Parameters defined as `[]VariableName` are SSZ-encoded containerless vectors.
All messages that have a single field, are not encoded as SSZ containers.

Responses that are SSZ list objects (for example `[]BeaconBlocks`) send their
constituents individually as `response_chunk`. For example, the
`[]BeaconBlocks` response send many `response_chunk`s with each payload being a `BeaconBlock`.

### Messages

#### Hello

**Protocol ID:** ``/eth2/beacon_chain/req/hello/1/``

**Content**:
Request, Response Content:
```
(
fork_version: bytes4
head_fork_version: bytes4
finalized_root: bytes32
finalized_epoch: uint64
head_root: bytes32
Expand All @@ -330,26 +336,29 @@ The [SimpleSerialize (SSZ) specification](../simple-serialize.md) outlines how o
```
The fields are:

- `fork_version`: The beacon_state `Fork` version.
- `head_fork_version`: The beacon_state `Fork` version.
- `finalized_root`: The latest finalized root the node knows about.
- `finalized_epoch`: The latest finalized epoch the node knows about.
- `head_root`: The block hash tree root corresponding to the head of the chain as seen by the sending node.
- `head_slot`: The slot corresponding to the `head_root`.

Clients exchange hello messages upon connection, forming a two-phase handshake. The first message the initiating client sends MUST be the hello message. In response, the receiving client MUST respond with its own hello message.
The dialing client MUST send a `Hello` request upon connection.

This MUST be encoded as an SSZ-container. The response consists of a single
`response_chunk`.

Clients SHOULD immediately disconnect from one another following the handshake above under the following conditions:

1. If `fork_version` doesn’t match the local fork version, since the client’s chain is on another fork. `fork_version` can also be used to segregate testnets.
1. If `head_fork_version` doesn’t match the expected fork version at the epoch of the `head_slot`, since the client’s chain is on another fork. `head_fork_version` can also be used to segregate testnets.
2. If the (`finalized_root`, `finalized_epoch`) shared by the peer is not in the client's chain at the expected epoch. For example, if Peer 1 sends (root, epoch) of (A, 5) and Peer 2 sends (B, 3) but Peer 1 has root C at epoch 3, then Peer 1 would disconnect because it knows that their chains are irreparably disjoint.

Once the handshake completes, the client with the lower `finalized_epoch` or `head_slot` (if the clients have equal `finalized_epoch`s) SHOULD request beacon blocks from its counterparty via the `BeaconBlocks` request.
Once the handshake completes, the client with the lower `finalized_epoch` or `head_slot` (if the clients have equal `finalized_epoch`s) SHOULD request beacon blocks from its counterparty via the `BeaconBlocksByRange` request.

#### Goodbye

**Protocol ID:** ``/eth2/beacon_chain/req/goodbye/1/``

**Content:**
Request, Response Content:
```
(
reason: uint64
Expand All @@ -365,11 +374,14 @@ Clients MAY use reason codes above `128` to indicate alternative, erroneous requ

The range `[4, 127]` is RESERVED for future usage.

#### BeaconBlocks
This MUST be encoded as a single SSZ-field. The response consists of a
single `response_chunk`.

#### BeaconBlocksByRange

**Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks/1/`
**Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_range/1/`

Request Content
Request Content:
```
(
head_block_root: HashTreeRoot
Expand All @@ -388,15 +400,24 @@ Response Content:

Requests count beacon blocks from the peer starting from `start_slot` on the chain defined by `head_block_root`. The response MUST contain no more than count blocks. `step` defines the slot increment between blocks. For example, requesting blocks starting at `start_slot` 2 with a step value of 2 would return the blocks at [2, 4, 6, …]. In cases where a slot is empty for a given slot number, no block is returned. For example, if slot 4 were empty in the previous example, the returned array would contain [2, 6, …]. A step value of 1 returns all blocks on the range `[start_slot, start_slot + count)`.

`BeaconBlocks` is primarily used to sync historical blocks.
The request MUST be encoded as an SSZ-container. The response is sent as many
`response_chunk` with each chunk consisting of a single `BeaconBlock`.

`BeaconBlocksByRange` is primarily used to sync historical blocks.

Clients MUST support requesting blocks since the start of the weak subjectivity period and up to the given `head_block_root`.

Clients MUST support `head_block_root` values since the latest finalized epoch.

#### RecentBeaconBlocks
Clients MUST respond with at least one block, if they have it.

Clients MUST order blocks by increasing slot number.

**Protocol ID:** `/eth2/beacon_chain/req/recent_beacon_blocks/1/`
Clients MAY respond with fewer blocks than requested, for example when the size of the response would exceed `REQ_RESP_MAX_SIZE` or `SSZ_MAX_LIST_SIZE`.

#### BeaconBlocksByRoot

**Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_root/1/`

Request Content:
djrtwo marked this conversation as resolved.
Show resolved Hide resolved

Expand All @@ -414,12 +435,18 @@ Response Content:
)
```

Requests blocks by their block roots. The response is a list of `BeaconBlock` with the same length as the request. Blocks are returned in order of the request and any missing/unknown blocks are left empty (SSZ null `BeaconBlock`).
Requests blocks by their block roots. The response is a list of `BeaconBlock` whose length is less or equal to the number of requested blocks. It may be less in the case that the responding peer is missing blocks.

`BeaconBlocksByRoot` is primarily used to recover recent blocks (ex. when receiving a block or attestation whose parent is unknown).

`RecentBeaconBlocks` is primarily used to recover recent blocks (ex. when receiving a block or attestation whose parent is unknown).
The request MUST be encoded as an SSZ-field. The response is sent as many `response_chunk` with each chunk consisting of a single `BeaconBlock`.

Clients MUST support requesting blocks since the latest finalized epoch.

Clients MUST respond with at least one block, if they have it.

Clients MAY respond with fewer blocks than requested, for example when the size of the uncompressed response would exceed `REQ_RESP_MAX_SIZE` or `SSZ_MAX_LIST_SIZE`.

## The discovery domain: discv5

Discovery Version 5 ([discv5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)) is used for peer discovery, both in the interoperability testnet and mainnet.
Expand Down