Merge pull request #2815 from ethereum/dev

Release updated merge specs

Makefile

@@ -130,9 +130,11 @@ codespell:
 	codespell . --skip ./.git -I .codespell-whitelist
 
 # TODO: add future protocol upgrade patch packages to linting.
+# NOTE: we use `pylint` just for catching unused arguments in spec code
 lint: pyspec
 	. venv/bin/activate; cd $(PY_SPEC_DIR); \
 	flake8  --config $(LINTER_CONFIG_FILE) ./eth2spec \
+	&& pylint --disable=all --enable unused-argument ./eth2spec/phase0 ./eth2spec/altair ./eth2spec/bellatrix \
 	&& mypy --config-file $(LINTER_CONFIG_FILE) -p eth2spec.phase0 -p eth2spec.altair -p eth2spec.bellatrix
 
 lint_generators: pyspec

README.md

@@ -33,7 +33,7 @@ The current features are:
 * [Honest Validator guide changes](specs/altair/validator.md)
 * [P2P Networking](specs/altair/p2p-interface.md)
 
-### Bellatrix (as known as The Merge)
+### Bellatrix (also known as The Merge)
 
 The Bellatrix protocol upgrade is still actively in development. The exact specification has not been formally accepted as final and details are still subject to change.
 
@@ -45,7 +45,6 @@ The Bellatrix protocol upgrade is still actively in development. The exact speci
   * [Bellatrix fork](specs/bellatrix/fork.md)
   * [Fork Choice changes](specs/bellatrix/fork-choice.md)
   * [Validator additions](specs/bellatrix/validator.md)
-  * [Client settings](specs/bellatrix/client-settings.md)
   * [P2P Networking](specs/bellatrix/p2p-interface.md)
 
 ### Sharding

configs/README.md

@@ -9,7 +9,7 @@ Standard configs:
 - [`minimal.yaml`](./minimal.yaml): Minimal configuration, used in spec-testing along with the [`minimal`](../presets/minimal) preset.
 
 Not all network configurations are in scope for the specification,
-see [`github.com/eth2-clients/eth2-networks`](https://github.com/eth2-clients/eth2-networks) for common networks,
+see [`github.com/eth-clients/eth2-networks`](https://github.com/eth-clients/eth2-networks) for common networks,
 and additional testnet assets.
 
 ## Forking

configs/mainnet.yaml

@@ -3,6 +3,13 @@
 # Extends the mainnet preset
 PRESET_BASE: 'mainnet'
 
+# Free-form short name of the network that this configuration applies to - known
+# canonical network names include:
+# * 'mainnet' - there can be only one
+# * 'prater' - testnet
+# Must match the regex: [a-z0-9\-]
+CONFIG_NAME: 'mainnet'
+
 # Transition
 # ---------------------------------------------------------------
 # TBD, 2**256-2**10 is a placeholder

configs/minimal.yaml

@@ -3,6 +3,13 @@
 # Extends the minimal preset
 PRESET_BASE: 'minimal'
 
+# Free-form short name of the network that this configuration applies to - known
+# canonical network names include:
+# * 'mainnet' - there can be only one
+# * 'prater' - testnet
+# Must match the regex: [a-z0-9\-]
+CONFIG_NAME: 'minimal'
+
 # Transition
 # ---------------------------------------------------------------
 # TBD, 2**256-2**10 is a placeholder

setup.py

@@ -514,7 +514,7 @@ def get_pow_block(hash: Bytes32) -> Optional[PowBlock]:
     return PowBlock(block_hash=hash, parent_hash=Bytes32(), total_difficulty=uint256(0))
 
 
-def get_execution_state(execution_state_root: Bytes32) -> ExecutionState:
+def get_execution_state(_execution_state_root: Bytes32) -> ExecutionState:
     pass
 
 
@@ -524,7 +524,7 @@ def get_pow_chain_head() -> PowBlock:
 
 class NoopExecutionEngine(ExecutionEngine):
 
-    def execute_payload(self: ExecutionEngine, execution_payload: ExecutionPayload) -> bool:
+    def notify_new_payload(self: ExecutionEngine, execution_payload: ExecutionPayload) -> bool:
         return True
 
     def notify_forkchoice_updated(self: ExecutionEngine,
@@ -752,7 +752,7 @@ def parse_config_vars(conf: Dict[str, str]) -> Dict[str, str]:
     """
     out: Dict[str, str] = dict()
     for k, v in conf.items():
-        if isinstance(v, str) and (v.startswith("0x") or k == 'PRESET_BASE'):
+        if isinstance(v, str) and (v.startswith("0x") or k == 'PRESET_BASE' or k == 'CONFIG_NAME'):
             # Represent byte data with string, to avoid misinterpretation as big-endian int.
             # Everything is either byte data or an integer, with PRESET_BASE as one exception.
             out[k] = f"'{v}'"
@@ -868,6 +868,7 @@ def finalize_options(self):
                     specs/bellatrix/fork.md
                     specs/bellatrix/fork-choice.md
                     specs/bellatrix/validator.md
+                    sync/optimistic.md
                 """
             if len(self.md_doc_paths) == 0:
                 raise Exception('no markdown files specified, and spec fork "%s" is unknown', self.spec_fork)
@@ -1008,7 +1009,7 @@ def run(self):
     python_requires=">=3.8, <4",
     extras_require={
         "test": ["pytest>=4.4", "pytest-cov", "pytest-xdist"],
-        "lint": ["flake8==3.7.7", "mypy==0.812"],
+        "lint": ["flake8==3.7.7", "mypy==0.812", "pylint==2.12.2"],
         "generator": ["python-snappy==0.5.4"],
     },
     install_requires=[

specs/altair/beacon-chain.md

@@ -126,7 +126,7 @@ This patch updates a few configuration values to move penalty parameters closer
 
 | Name | Value | Unit | Duration |
 | - | - | - | - |
-| `SYNC_COMMITTEE_SIZE` | `uint64(2**9)` (= 512) | Validators | |
+| `SYNC_COMMITTEE_SIZE` | `uint64(2**9)` (= 512) | validators | |
 | `EPOCHS_PER_SYNC_COMMITTEE_PERIOD` | `uint64(2**8)` (= 256) | epochs | ~27 hours |
 
 ## Configuration
@@ -521,7 +521,7 @@ def process_deposit(state: BeaconState, deposit: Deposit) -> None:
         signing_root = compute_signing_root(deposit_message, domain)
         # Initialize validator if the deposit signature is valid
         if bls.Verify(pubkey, signing_root, deposit.data.signature):
-            state.validators.append(get_validator_from_deposit(state, deposit))
+            state.validators.append(get_validator_from_deposit(deposit))
             state.balances.append(amount)
             state.previous_epoch_participation.append(ParticipationFlags(0b0000_0000))
             state.current_epoch_participation.append(ParticipationFlags(0b0000_0000))

specs/altair/sync-protocol.md

@@ -42,17 +42,17 @@ uses sync committees introduced in [this beacon chain extension](./beacon-chain.
 
 | Name | Value |
 | - | - |
-| `FINALIZED_ROOT_INDEX` | `get_generalized_index(BeaconState, 'finalized_checkpoint', 'root')` |
-| `NEXT_SYNC_COMMITTEE_INDEX` | `get_generalized_index(BeaconState, 'next_sync_committee')` |
+| `FINALIZED_ROOT_INDEX` | `get_generalized_index(BeaconState, 'finalized_checkpoint', 'root')` (= 105) |
+| `NEXT_SYNC_COMMITTEE_INDEX` | `get_generalized_index(BeaconState, 'next_sync_committee')` (= 55) |
 
 ## Preset
 
 ### Misc
 
-| Name | Value | Notes |
-| - | - | - |
-| `MIN_SYNC_COMMITTEE_PARTICIPANTS` | `1` | |
-| `UPDATE_TIMEOUT` | `SLOTS_PER_EPOCH * EPOCHS_PER_SYNC_COMMITTEE_PERIOD` | ~27.3 hours |
+| Name | Value | Unit | Duration |
+| - | - | - | - |
+| `MIN_SYNC_COMMITTEE_PARTICIPANTS` | `1` | validators |
+| `UPDATE_TIMEOUT` | `SLOTS_PER_EPOCH * EPOCHS_PER_SYNC_COMMITTEE_PERIOD` | epochs | ~27.3 hours |
 
 ## Containers
 
@@ -69,7 +69,7 @@ class LightClientUpdate(Container):
     finalized_header: BeaconBlockHeader
     finality_branch: Vector[Bytes32, floorlog2(FINALIZED_ROOT_INDEX)]
     # Sync committee aggregate signature
-    sync_committee_aggregate: SyncAggregate
+    sync_aggregate: SyncAggregate
     # Fork version for the aggregate signature
     fork_version: Version
 ```
@@ -127,7 +127,7 @@ def get_safety_threshold(store: LightClientStore) -> uint64:
 
 ## Light client state updates
 
-A light client maintains its state in a `store` object of type `LightClientStore` and receives `update` objects of type `LightClientUpdate`. Every `update` triggers `process_light_client_update(store, update, current_slot)` where `current_slot` is the current slot based on some local clock. `process_slot_for_light_client_store` is processed every time the current slot increments.
+A light client maintains its state in a `store` object of type `LightClientStore` and receives `update` objects of type `LightClientUpdate`. Every `update` triggers `process_light_client_update(store, update, current_slot, genesis_validators_root)` where `current_slot` is the current slot based on a local clock. `process_slot_for_light_client_store` is triggered every time the current slot increments.
 
 #### `process_slot_for_light_client_store`
 
@@ -187,8 +187,8 @@ def validate_light_client_update(store: LightClientStore,
             index=get_subtree_index(NEXT_SYNC_COMMITTEE_INDEX),
             root=active_header.state_root,
         )
-        
-    sync_aggregate = update.sync_committee_aggregate
+
+    sync_aggregate = update.sync_aggregate
 
     # Verify sync committee has sufficient participants
     assert sum(sync_aggregate.sync_committee_bits) >= MIN_SYNC_COMMITTEE_PARTICIPANTS
@@ -225,12 +225,12 @@ def process_light_client_update(store: LightClientStore,
                                 genesis_validators_root: Root) -> None:
     validate_light_client_update(store, update, current_slot, genesis_validators_root)
 
-    sync_committee_bits = update.sync_committee_aggregate.sync_committee_bits
+    sync_committee_bits = update.sync_aggregate.sync_committee_bits
 
     # Update the best update in case we have to force-update to it if the timeout elapses
     if (
         store.best_valid_update is None
-        or sum(sync_committee_bits) > sum(store.best_valid_update.sync_committee_aggregate.sync_committee_bits)
+        or sum(sync_committee_bits) > sum(store.best_valid_update.sync_aggregate.sync_committee_bits)
     ):
         store.best_valid_update = update
 

specs/altair/validator.md

@@ -268,7 +268,7 @@ This process occurs each slot.
 If a validator is in the current sync committee (i.e. `is_assigned_to_sync_committee()` above returns `True`), then for every `slot` in the current sync committee period, the validator should prepare a `SyncCommitteeMessage` for the previous slot (`slot - 1`) according to the logic in `get_sync_committee_message` as soon as they have determined the head block of `slot - 1`. This means that when assigned to `slot` a `SyncCommitteeMessage` is prepared and broadcast in `slot-1 ` instead of `slot`.   
 
 This logic is triggered upon the same conditions as when producing an attestation.
-Meaning, a sync committee member should produce and broadcast a `SyncCommitteeMessage` either when (a) the validator has received a valid block from the expected block proposer for the current `slot` or (b) one-third of the slot has transpired (`SECONDS_PER_SLOT / 3` seconds after the start of the slot) -- whichever comes first.
+Meaning, a sync committee member should produce and broadcast a `SyncCommitteeMessage` either when (a) the validator has received a valid block from the expected block proposer for the current `slot` or (b) one-third of the slot has transpired (`SECONDS_PER_SLOT / INTERVALS_PER_SLOT` seconds after the start of the slot) -- whichever comes first.
 
 `get_sync_committee_message(state, block_root, validator_index, privkey)` assumes the parameter `state` is the head state corresponding to processing the block up to the current slot as determined by the fork choice (including any empty slots up to the current slot processed with `process_slots` on top of the latest block), `block_root` is the root of the head block, `validator_index` is the index of the validator in the registry `state.validators` controlled by `privkey`, and `privkey` is the BLS private key for the validator.
 
@@ -385,7 +385,7 @@ The collection of input signatures should include one signature per validator wh
 
 ##### Broadcast sync committee contribution
 
-If the validator is selected to aggregate (`is_sync_committee_aggregator()`), then they broadcast their best aggregate as a `SignedContributionAndProof` to the global aggregate channel (`sync_committee_contribution_and_proof` topic) two-thirds of the way through the `slot`-that is, `SECONDS_PER_SLOT * 2 / 3` seconds after the start of `slot`.
+If the validator is selected to aggregate (`is_sync_committee_aggregator()`), then they broadcast their best aggregate as a `SignedContributionAndProof` to the global aggregate channel (`sync_committee_contribution_and_proof` topic) two-thirds of the way through the `slot`-that is, `SECONDS_PER_SLOT * 2 / INTERVALS_PER_SLOT` seconds after the start of `slot`.
 
 Selection proofs are provided in `ContributionAndProof` to prove to the gossip channel that the validator has been selected as an aggregator.
 

specs/bellatrix/beacon-chain.md

@@ -35,7 +35,7 @@
     - [Modified `slash_validator`](#modified-slash_validator)
 - [Beacon chain state transition function](#beacon-chain-state-transition-function)
   - [Execution engine](#execution-engine)
-    - [`execute_payload`](#execute_payload)
+    - [`notify_new_payload`](#notify_new_payload)
   - [Block processing](#block-processing)
     - [Execution payload](#execution-payload)
       - [`process_execution_payload`](#process_execution_payload)
@@ -169,7 +169,7 @@ class ExecutionPayload(Container):
     parent_hash: Hash32
     fee_recipient: ExecutionAddress  # 'beneficiary' in the yellow paper
     state_root: Bytes32
-    receipt_root: Bytes32  # 'receipts root' in the yellow paper
+    receipts_root: Bytes32
     logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM]
     random: Bytes32  # 'difficulty' in the yellow paper
     block_number: uint64  # 'number' in the yellow paper
@@ -191,7 +191,7 @@ class ExecutionPayloadHeader(Container):
     parent_hash: Hash32
     fee_recipient: ExecutionAddress
     state_root: Bytes32
-    receipt_root: Bytes32
+    receipts_root: Bytes32
     logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM]
     random: Bytes32
     block_number: uint64
@@ -307,17 +307,17 @@ def slash_validator(state: BeaconState,
 The implementation-dependent `ExecutionEngine` protocol encapsulates the execution sub-system logic via:
 
 * a state object `self.execution_state` of type `ExecutionState`
-* a state transition function `self.execute_payload` which applies changes to the `self.execution_state`
+* a notification function `self.notify_new_payload` which may apply changes to the `self.execution_state`
 
-*Note*: `execute_payload` is a function accessed through the `EXECUTION_ENGINE` module which instantiates the `ExecutionEngine` protocol.
+*Note*: `notify_new_payload` is a function accessed through the `EXECUTION_ENGINE` module which instantiates the `ExecutionEngine` protocol.
 
 The body of this function is implementation dependent.
 The Engine API may be used to implement this and similarly defined functions via an external execution engine.
 
-#### `execute_payload`
+#### `notify_new_payload`
 
 ```python
-def execute_payload(self: ExecutionEngine, execution_payload: ExecutionPayload) -> bool:
+def notify_new_payload(self: ExecutionEngine, execution_payload: ExecutionPayload) -> bool:
     """
     Return ``True`` if and only if ``execution_payload`` is valid with respect to ``self.execution_state``.
     """
@@ -353,13 +353,13 @@ def process_execution_payload(state: BeaconState, payload: ExecutionPayload, exe
     # Verify timestamp
     assert payload.timestamp == compute_timestamp_at_slot(state, state.slot)
     # Verify the execution payload is valid
-    assert execution_engine.execute_payload(payload)
+    assert execution_engine.notify_new_payload(payload)
     # Cache execution payload header
     state.latest_execution_payload_header = ExecutionPayloadHeader(
         parent_hash=payload.parent_hash,
         fee_recipient=payload.fee_recipient,
         state_root=payload.state_root,
-        receipt_root=payload.receipt_root,
+        receipts_root=payload.receipts_root,
         logs_bloom=payload.logs_bloom,
         random=payload.random,
         block_number=payload.block_number,

specs/bellatrix/p2p-interface.md

@@ -29,6 +29,7 @@ Readers should understand the Phase 0 and Altair documents and use them as a bas
     - [Why was the max gossip message size increased at Bellatrix?](#why-was-the-max-gossip-message-size-increased-at-bellatrix)
   - [Req/Resp](#reqresp)
     - [Why was the max chunk response size increased at Bellatrix?](#why-was-the-max-chunk-response-size-increased-at-bellatrix)
+    - [Why allow invalid payloads on the P2P network?](#why-allow-invalid-payloads-on-the-p2p-network)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 <!-- /TOC -->
@@ -85,13 +86,26 @@ The *type* of the payload of this topic changes to the (modified) `SignedBeaconB
 Specifically, this type changes with the addition of `execution_payload` to the inner `BeaconBlockBody`.
 See Bellatrix [state transition document](./beacon-chain.md#beaconblockbody) for further details.
 
+Blocks with execution enabled will be permitted to propagate regardless of the
+validity of the execution payload. This prevents network segregation between
+[optimistic](/sync/optimistic.md) and non-optimistic nodes.
+
 In addition to the gossip validations for this topic from prior specifications,
 the following validations MUST pass before forwarding the `signed_beacon_block` on the network.
 Alias `block = signed_beacon_block.message`, `execution_payload = block.body.execution_payload`.
 - If the execution is enabled for the block -- i.e. `is_execution_enabled(state, block.body)`
   then validate the following:
-  - _[REJECT]_ The block's execution payload timestamp is correct with respect to the slot
-    -- i.e. `execution_payload.timestamp == compute_timestamp_at_slot(state, block.slot)`.
+    - _[REJECT]_ The block's execution payload timestamp is correct with respect to the slot
+       -- i.e. `execution_payload.timestamp == compute_timestamp_at_slot(state, block.slot)`.
+    - If `exection_payload` verification of block's parent by an execution node is *not* complete:
+    	- [REJECT] The block's parent (defined by `block.parent_root`) passes all
+    	  validation (excluding execution node verification of the `block.body.execution_payload`).
+    - otherwise:
+    	- [IGNORE] The block's parent (defined by `block.parent_root`) passes all
+    	  validation (including execution node verification of the `block.body.execution_payload`).
+
+The following gossip validation from prior specifications MUST NOT be applied if the execution is enabled for the block -- i.e. `is_execution_enabled(state, block.body)`:
+  - [REJECT] The block's parent (defined by `block.parent_root`) passes validation.
 
 ### Transitioning the gossip
 
@@ -100,6 +114,14 @@ details on how to handle transitioning gossip topics for Bellatrix.
 
 ## The Req/Resp domain
 
+Non-faulty, [optimistic](/sync/optimistic.md) nodes may send blocks which
+result in an INVALID response from an execution engine. To prevent network
+segregation between optimistic and non-optimistic nodes, transmission of an
+INVALID execution payload via the Req/Resp domain SHOULD NOT cause a node to be
+down-scored or disconnected. Transmission of a block which is invalid due to
+any consensus layer rules (i.e., *not* execution layer rules) MAY result in
+down-scoring or disconnection.
+
 ### Messages
 
 #### BeaconBlocksByRange v2
@@ -181,3 +203,26 @@ valid block sizes in the range of gas limits expected in the medium term.
 
 As with both gossip and req/rsp maximum values, type-specific limits should
 always by simultaneously respected.
+
+### Why allow invalid payloads on the P2P network?
+
+The specification allows blocks with invalid execution payloads to propagate across
+gossip and via RPC calls. The reasoning for this is as follows:
+
+1. Optimistic nodes must listen to block gossip to obtain a view of the head of
+   the chain.
+2. Therefore, optimistic nodes must propagate gossip blocks. Otherwise, they'd
+   be censoring.
+3. If optimistic nodes will propagate blocks via gossip, then they must respond
+   to requests for the parent via RPC.
+4. Therefore, optimistic nodes must send optimistic blocks via RPC.
+
+So, to prevent network segregation from optimistic nodes inadvertently sending
+invalid execution payloads, nodes should never downscore/disconnect nodes due to such invalid
+payloads. This does open the network to some DoS attacks from invalid execution
+payloads, but the scope of actors is limited to validators who can put those
+payloads in valid (and slashable) beacon blocks. Therefore, it is argued that
+the DoS risk introduced in tolerable.
+
+More complicated schemes are possible that could restrict invalid payloads from
+RPC. However, it's not clear that complexity is warranted.