From 9374766f32aabcee67f5e117ecb158071ea6ffb7 Mon Sep 17 00:00:00 2001 From: Potuz Date: Mon, 7 Aug 2023 11:22:37 -0300 Subject: [PATCH 01/57] init --- specs/_features/epbs/beacon-chain.md | 331 +++++++++++++++++++++++++++ 1 file changed, 331 insertions(+) create mode 100644 specs/_features/epbs/beacon-chain.md diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md new file mode 100644 index 0000000000..8d293e7bfc --- /dev/null +++ b/specs/_features/epbs/beacon-chain.md @@ -0,0 +1,331 @@ +# ePBS -- The Beacon Chain + +## Table of contents + + + + + + + + +## Introduction + +This is the beacon chain specification of the enshrined proposer builder separation feature. + +*Note:* This specification is built upon [Deneb](../../deneb/beacon-chain.md) and is under active development. + +This feature adds new staked consensus participants called *Builders* and new honest validators duties called *payload timeliness attestations*. The slot is divided in **four** intervals as opposed to the current three. Honest validators gather *signed bids* from builders and submit their consensus blocks (a `SignedBlindedBeaconBlock`) at the beginning of the slot. At the start of the second interval, honest validators submit attestations just as they do previous to this feature). At the start of the third interval, aggregators aggregate these attestations (exactly as before this feature) and the honest builder reveals the full payload. At the start of the fourth interval, some honest validators selected to be members of the new **Payload Timeliness Committee** attest to the presence of the builder's payload. + +At any given slot, the status of the blockchain's head may be either +- A *full* block from a previous slot (eg. the current slot's proposer did not submit its block). +- An *empty* block from the current slot (eg. the proposer submitted a timely block, but the builder did not reveal the payload on time). +- A full block for the current slot (both the proposer and the builder revealed on time). + +For a further introduction please refer to this [ethresear.ch article](https://ethresear.ch/t/payload-timeliness-committee-ptc-an-epbs-design/16054) + +## Preset + +### Misc + +| Name | Value | +| - | - | +| `PTC_SIZE` | `uint64(2**9)` (=512) | + +### Domain types + +| Name | Value | +| - | - | +| `DOMAIN_BEACON_BUILDER` | `DomainType('0x0B000000')` | + +### State list lengths + +| Name | Value | Unit | Duration | +| - | - | :-: | :-: | +| `BUILDER_REGISTRY_LIMIT` | `uint64(2**20)` (=1,048,576) | builders | + +### Gwei values + +| Name | Value | +| - | - | +| `BUILDER_MIN_BALANCE` | `Gwei(2**10 * 10**9)` = (1,024,000,000,000) | + +### Incentivization weights + +| Name | Value | +| - | - | +| `PTC_PENALTY_WEIGHT` | `uint64(2)` | + +### Execution +| Name | Value | +| - | - | +| MAX_TRANSACTIONS_PER_INCLUSION_LIST | `2**4` (=16) | + +## Containers + +### New Containers + +#### `Builder` + +``` python +class Builder(Container): + pubkey: BLSPubkey + withdrawal_address: ExecutionAddress # Commitment to pubkey for withdrawals + effective_balance: Gwei # Balance at stake + exit_epoch: Epoch + withdrawable_epoch: Epoch # When builder can withdraw funds +``` + +#### `SignedExecutionPayloadHeader` + +```python +class SignedExecutionPayloadHeader(Container): + message: ExecutionPayloadHeader + signature: BLSSignature +``` + +#### `ExecutionPayloadEnvelope` + +```python +class ExecutionPayloadEnvelope(Container): + payload: ExecutionPayload + state_root: Root +``` + +#### `SignedExecutionPayloadEnvelope` + +```python +class SignedExecutionPayloadEnvelope(Container): + message: ExecutionPayloadEnvelope + signature: BLSSignature +``` + +### Modified Containers + +#### `ExecutionPayload` + +```python +class ExecutionPayload(Container): + # Execution block header fields + parent_hash: Hash32 + fee_recipient: ExecutionAddress # 'beneficiary' in the yellow paper + state_root: Bytes32 + receipts_root: Bytes32 + logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM] + prev_randao: Bytes32 # 'difficulty' in the yellow paper + block_number: uint64 # 'number' in the yellow paper + gas_limit: uint64 + gas_used: uint64 + timestamp: uint64 + extra_data: ByteList[MAX_EXTRA_DATA_BYTES] + base_fee_per_gas: uint256 + # Extra payload fields + block_hash: Hash32 # Hash of execution block + transactions: List[Transaction, MAX_TRANSACTIONS_PER_PAYLOAD] + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] + builder_index: uint64 # [New in ePBS] +``` + +#### `ExecutionPayloadHeader` + +```python +class ExecutionPayloadHeader(Container): + # Execution block header fields + parent_hash: Hash32 + fee_recipient: ExecutionAddress + state_root: Bytes32 + receipts_root: Bytes32 + logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM] + prev_randao: Bytes32 + block_number: uint64 + gas_limit: uint64 + gas_used: uint64 + timestamp: uint64 + extra_data: ByteList[MAX_EXTRA_DATA_BYTES] + base_fee_per_gas: uint256 + # Extra payload fields + block_hash: Hash32 # Hash of execution block + transactions_root: Root + withdrawals_root: Root + builder_index: uint64 # [New in ePBS] +``` + +#### `BeaconBlockBody` + +```python +class BeaconBlockBody(Container): + randao_reveal: BLSSignature + eth1_data: Eth1Data # Eth1 data vote + graffiti: Bytes32 # Arbitrary data + # Operations + proposer_slashings: List[ProposerSlashing, MAX_PROPOSER_SLASHINGS] + attester_slashings: List[AttesterSlashing, MAX_ATTESTER_SLASHINGS] + attestations: List[Attestation, MAX_ATTESTATIONS] + deposits: List[Deposit, MAX_DEPOSITS] + voluntary_exits: List[SignedVoluntaryExit, MAX_VOLUNTARY_EXITS] + sync_aggregate: SyncAggregate + execution_payload_header: SignedExecutionPayloadHeader # [Modified in ePBS] + bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES] + tx_inclusion_list: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] +``` + + +#### `BeaconState` +*Note*: the beacon state is modified to store a signed latest execution payload header and it adds a registry of builders, their balances and two transaction inclusion lists. + +```python +class BeaconState(Container): + # Versioning + genesis_time: uint64 + genesis_validators_root: Root + slot: Slot + fork: Fork + # History + latest_block_header: BeaconBlockHeader + block_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT] + state_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT] + historical_roots: List[Root, HISTORICAL_ROOTS_LIMIT] # Frozen in Capella, replaced by historical_summaries + # Eth1 + eth1_data: Eth1Data + eth1_data_votes: List[Eth1Data, EPOCHS_PER_ETH1_VOTING_PERIOD * SLOTS_PER_EPOCH] + eth1_deposit_index: uint64 + # Registry + validators: List[Validator, VALIDATOR_REGISTRY_LIMIT] + balances: List[Gwei, VALIDATOR_REGISTRY_LIMIT] + # Randomness + randao_mixes: Vector[Bytes32, EPOCHS_PER_HISTORICAL_VECTOR] + # Slashings + slashings: Vector[Gwei, EPOCHS_PER_SLASHINGS_VECTOR] # Per-epoch sums of slashed effective balances + # Participation + previous_epoch_participation: List[ParticipationFlags, VALIDATOR_REGISTRY_LIMIT] + current_epoch_participation: List[ParticipationFlags, VALIDATOR_REGISTRY_LIMIT] + # Finality + justification_bits: Bitvector[JUSTIFICATION_BITS_LENGTH] # Bit set for every recent justified epoch + previous_justified_checkpoint: Checkpoint + current_justified_checkpoint: Checkpoint + finalized_checkpoint: Checkpoint + # Inactivity + inactivity_scores: List[uint64, VALIDATOR_REGISTRY_LIMIT] + # Sync + current_sync_committee: SyncCommittee + next_sync_committee: SyncCommittee + # Execution + latest_execution_payload_header: ExecutionPayloadHeader + # Withdrawals + next_withdrawal_index: WithdrawalIndex + next_withdrawal_validator_index: ValidatorIndex + # Deep history valid from Capella onwards + historical_summaries: List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT] + # PBS + builders: List[Builder, BUILDER_REGISTRY_LIMIT] # [New in ePBS] + builder_balances: List[Gwei, BUILDER_REGISTRY_LIMIT] # [New in ePBS] + previous_tx_inclusion_list: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] + current_tx_inclusion_list: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] + current_signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] +``` + +## Beacon chain state transition function + +*Note*: state transition is fundamentally modified in ePBS. The full state transition is broken in two parts, first importing a signed block and then importing an execution payload. + +The post-state corresponding to a pre-state `state` and a signed block `signed_block` is defined as `state_transition(state, signed_block)`. State transitions that trigger an unhandled exception (e.g. a failed `assert` or an out-of-range list access) are considered invalid. State transitions that cause a `uint64` overflow or underflow are also considered invalid. + +The post-state corresponding to a pre-state `state` and a signed execution payload `signed_execution_payload` is defined as `process_execution_payload(state, signed_execution_payload)`. State transitions that trigger an unhandled exception (e.g. a failed `assert` or an out-of-range list access) are considered invalid. State transitions that cause a `uint64` overflow or underflow are also considered invalid. + +### Block processing + +*Note*: the function `process_block` is modified to only process the consensus block. The full state-transition process is broken into separate functions, one to process a `BeaconBlock` and another to process a `SignedExecutionPayload`. + +```python +def process_block(state: BeaconState, block: BeaconBlock) -> None: + process_block_header(state, block) + process_execution_payload_header(state, block.body.execution_payload_header) # [Modified in ePBS] + # Removed process_withdrawal in ePBS is processed during payload processing [Modified in ePBS] + process_randao(state, block.body) + process_eth1_data(state, block.body) + process_operations(state, block.body) # [Modified in ePBS] + process_sync_aggregate(state, block.body.sync_aggregate) + process_tx_inclusion_list(state, block) # [New in ePBS] +``` + +#### New `update_tx_inclusion_lists` + +```python +def update_tx_inclusion_lists(state: BeaconState, payload: ExecutionPayload) -> None: + old_transactions = payload.transactions[:len(state.previous_tx_inclusion_list)] + assert state.previous_tx_inclusion_list == old_transactions + + new_transactions = payload.transactions[len(state.previous_tx_inclusion_list):] + state.previous_tx_inclusion_list = [tx for tx in state.current_tx_inclusion_list if x not in new_transactions] + + #TODO: check validity of the IL for the next block, requires engine changes +``` +#### New `verify_execution_payload_header_signature` + +```python +def verify_execution_payload_header_signature(state: BeaconState, signed_header: SignedExecutionPayloadHeader) -> bool: + builder = state.builders[signed_header.message.builder_index] + signing_root = compute_signing_root(signed_header.message, get_domain(state, DOMAIN_BEACON_BUILDER)) + return bls.Verify(builder.pubkey, signing_root, signed_header.signature) +``` + +#### New `verify_execution_payload_signature` + +```python +def verify_execution_envelope_signature(state: BeaconState, signed_envelope: SignedExecutionPayloadEnvelope) -> bool: + builder = state.builders[signed_envelope.message.payload.builder_index] + signing_root = compute_signing_root(signed_envelope.message, get_domain(state, DOMAIN_BEACON_BUILDER)) + return bls.Verify(builder.pubkey, signing_root, signed_envelope.signature) +``` + +#### New `process_execution_payload_header` + +```python +def process_execution_payload_header(state: BeaconState, signed_header: SignedExecutionPayloadHeader) -> None: + assert verify_execution_payload_header_signature(state, signed_header) + header = signed_header.message + # Verify consistency of the parent hash with respect to the previous execution payload header + assert header.parent_hash == state.latest_execution_payload_header.block_hash + # Verify prev_randao + assert header.prev_randao == get_randao_mix(state, get_current_epoch(state)) + # Verify timestamp + assert header.timestamp == compute_timestamp_at_slot(state, state.slot) + # Cache execution payload header + state.current_signed_execution_payload_header = signed_header +``` + +#### Modified `process_execution_payload` +*Note*: `process_execution_payload` is now an independent check in state transition. It is called when importing a signed execution payload proposed by the builder of the current slot. + +TODO: Deal with the case when the payload becomes invalid because of the forward inclusion list. + +```python +def process_execution_payload(state: BeaconState, signed_envelope: SignedExecutionPayloadEnvelope, execution_engine: ExecutionEngine) -> None: + # Verify signature [New in ePBS] + assert verify_execution_envelope_signature(state, signed_envelope) + payload = signed_envelope.message.payload + # Verify consistency with the committed header + hash = hash_tree_root(payload) + previous_hash = hash_tree_root(state.current_signed_execution_payload_header.message) + assert hash == previous_hash + # Verify and update the proposers inclusion lists + update_tx_inclusion_lists(state, payload) + # Verify the execution payload is valid + assert execution_engine.verify_and_notify_new_payload(NewPayloadRequest(execution_payload=payload)) + # Process Withdrawals in the payload + process_withdrawals(state, payload) + # Cache the execution payload header + state.latest_execution_payload_header = state.current_signed_execution_payload_header.message + # Verify the state root + assert signed_envelope.message.state_root == hash_tree_root(state) +``` + +#### New `process_tx_inclusion_list` + +```python +def process_tx_inclusion_list(state: BeaconState, block: BeaconBlock) -> None: + # TODO: cap gas usage, comunicate with the engine. + state.previous_tx_inclusion_list = state.current_tx_inclusion_list + state.current_tx_inclusion_list = block.body.tx_inclusion_list +``` + From 82cf8dda3c5ce16ab3b6061ebc5e779cd5e2905c Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 8 Aug 2023 11:54:34 -0300 Subject: [PATCH 02/57] Add engine methods and check IL validity --- specs/_features/epbs/beacon-chain.md | 94 ++++++++++++++++++++++++---- 1 file changed, 81 insertions(+), 13 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 8d293e7bfc..7383485ddb 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -60,6 +60,7 @@ For a further introduction please refer to this [ethresear.ch article](https://e | Name | Value | | - | - | | MAX_TRANSACTIONS_PER_INCLUSION_LIST | `2**4` (=16) | +| MAX_GAS_PER_INCLUSION_LIST | `2**20` (=1,048,576) | ## Containers @@ -124,6 +125,7 @@ class ExecutionPayload(Container): transactions: List[Transaction, MAX_TRANSACTIONS_PER_PAYLOAD] withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] builder_index: uint64 # [New in ePBS] + value: Gwei # [New in ePBS] ``` #### `ExecutionPayloadHeader` @@ -148,6 +150,7 @@ class ExecutionPayloadHeader(Container): transactions_root: Root withdrawals_root: Root builder_index: uint64 # [New in ePBS] + value: Gwei # [New in ePBS] ``` #### `BeaconBlockBody` @@ -232,41 +235,101 @@ The post-state corresponding to a pre-state `state` and a signed block `signed_b The post-state corresponding to a pre-state `state` and a signed execution payload `signed_execution_payload` is defined as `process_execution_payload(state, signed_execution_payload)`. State transitions that trigger an unhandled exception (e.g. a failed `assert` or an out-of-range list access) are considered invalid. State transitions that cause a `uint64` overflow or underflow are also considered invalid. +### Execution engine + +#### Request data + +##### New `NewInclusionListRequest` + +```python +@dataclass +class NewInclusionListRequest(object): + inclusion_list: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] +``` + +#### Engine APIs + +#### Modified `notify_new_payload` +*Note*: the function notify new payload is modified to raise an exception if the payload is not valid, and to return the list of transactions that remain valid in the inclusion list + +```python +def notify_new_payload(self: ExecutionEngine, execution_payload: ExecutionPayload) -> List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST]: + """ + Raise an exception if ``execution_payload`` is not valid with respect to ``self.execution_state``. + Returns the list of transactions in the inclusion list that remain valid after executing the payload. That is + it is guaranteed that the transactions returned in the list can be executed in the exact order starting from the + current ``self.execution_state``. + """ + ... +``` + +#### Modified `verify_and_notify_new_payload` +*Note*: the function `verify_and_notify_new_payload` is modified so that it returns the list of transactions that remain valid in the forward inclusion list. It raises an exception if the payload is not valid. + +```python +def verify_and_notify_new_payload(self: ExecutionEngine, + new_payload_request: NewPayloadRequest) -> List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST]: + """ + Raise an exception if ``execution_payload`` is not valid with respect to ``self.execution_state``. + Returns the list of transactions in the inclusion list that remain valid after executing the payload. That is + it is guaranteed that the transactions returned in the list can be executed in the exact order starting from the + current ``self.execution_state``. + """ + assert self.is_valid_block_hash(new_payload_request.execution_payload) + return self.notify_new_payload(new_payload_request.execution_payload) +``` + +#### New `notify_new_inclusion_list` + +```python +def notify_new_inclusion_list(self: ExecutionEngine, + inclusion_list_request: NewInclusionListRequest) -> bool: + """ + Return ``True`` if and only if the transactions in the inclusion list can be succesfully executed + starting from the current ``self.execution_state`` and that they consume less or equal than + ```MAX_GAS_PER_INCLUSION_LIST``. + """ + ... +``` + ### Block processing *Note*: the function `process_block` is modified to only process the consensus block. The full state-transition process is broken into separate functions, one to process a `BeaconBlock` and another to process a `SignedExecutionPayload`. +Notice that `process_tx_inclusion_list` needs to be processed before the payload header since the former requires to check the last committed payload header. + + ```python def process_block(state: BeaconState, block: BeaconBlock) -> None: process_block_header(state, block) + process_tx_inclusion_list(state, block, EXECUTION_ENGINE) # [New in ePBS] process_execution_payload_header(state, block.body.execution_payload_header) # [Modified in ePBS] # Removed process_withdrawal in ePBS is processed during payload processing [Modified in ePBS] process_randao(state, block.body) process_eth1_data(state, block.body) process_operations(state, block.body) # [Modified in ePBS] process_sync_aggregate(state, block.body.sync_aggregate) - process_tx_inclusion_list(state, block) # [New in ePBS] ``` #### New `update_tx_inclusion_lists` ```python -def update_tx_inclusion_lists(state: BeaconState, payload: ExecutionPayload) -> None: +def update_tx_inclusion_lists(state: BeaconState, payload: ExecutionPayload, engine: ExecutionEngine, inclusion_list: List[Transaction, MAX_TRANSACTION_PER_INCLUSION_LIST]) -> None: old_transactions = payload.transactions[:len(state.previous_tx_inclusion_list)] assert state.previous_tx_inclusion_list == old_transactions - new_transactions = payload.transactions[len(state.previous_tx_inclusion_list):] - state.previous_tx_inclusion_list = [tx for tx in state.current_tx_inclusion_list if x not in new_transactions] - - #TODO: check validity of the IL for the next block, requires engine changes + state.previous_tx_inclusion_list = inclusion_list ``` + #### New `verify_execution_payload_header_signature` ```python def verify_execution_payload_header_signature(state: BeaconState, signed_header: SignedExecutionPayloadHeader) -> bool: builder = state.builders[signed_header.message.builder_index] signing_root = compute_signing_root(signed_header.message, get_domain(state, DOMAIN_BEACON_BUILDER)) - return bls.Verify(builder.pubkey, signing_root, signed_header.signature) + if not bls.Verify(builder.pubkey, signing_root, signed_header.signature): + return False + return ``` #### New `verify_execution_payload_signature` @@ -308,10 +371,10 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti hash = hash_tree_root(payload) previous_hash = hash_tree_root(state.current_signed_execution_payload_header.message) assert hash == previous_hash - # Verify and update the proposers inclusion lists - update_tx_inclusion_lists(state, payload) # Verify the execution payload is valid - assert execution_engine.verify_and_notify_new_payload(NewPayloadRequest(execution_payload=payload)) + inclusion_list = execution_engine.verify_and_notify_new_payload(NewPayloadRequest(execution_payload=payload)) + # Verify and update the proposers inclusion lists + update_tx_inclusion_lists(state, payload, inclusion_list) # Process Withdrawals in the payload process_withdrawals(state, payload) # Cache the execution payload header @@ -323,9 +386,14 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti #### New `process_tx_inclusion_list` ```python -def process_tx_inclusion_list(state: BeaconState, block: BeaconBlock) -> None: - # TODO: cap gas usage, comunicate with the engine. +def process_tx_inclusion_list(state: BeaconState, block: BeaconBlock, execution_engine: ExecutionEngine) -> None: + inclusion_list = block.body.tx_inclusion_list + # Verify that the list is empty if the parent consensus block did not contain a payload + if state.current_signed_execution_payload_header.message != state.latest_execution_payload_header: + assert not inclusion_list + return + assert notify_new_inclusion_list(execution_engine, inclusion_list) state.previous_tx_inclusion_list = state.current_tx_inclusion_list - state.current_tx_inclusion_list = block.body.tx_inclusion_list + state.current_tx_inclusion_list = inclusion_list ``` From 4cff1ae1d7e68ece3902c07e32082a2ed5e0be98 Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 8 Aug 2023 13:07:15 -0300 Subject: [PATCH 03/57] add small design notes for ILs --- specs/_features/epbs/design.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 specs/_features/epbs/design.md diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md new file mode 100644 index 0000000000..1cc3d86aa9 --- /dev/null +++ b/specs/_features/epbs/design.md @@ -0,0 +1,17 @@ +# ePBS design notes + +## Inclusion lists + +ePBS introduces forward inclusion lists for proposers to guarantee censor resistanship of the network. They are implemented as follows + +- Proposer for slot N submits a signed block that includes some transactions to be included at the beginning of slot N+1. +- Validators for slot N will consider the block invalid if those transactions are not executable at the start of slot N (this makes it impossible to put transactions that are only valid at slot N+1 for example, but still the proposer can protect from unbundling/replaying by binding the transaction to only be valid up to N+1 for example, because the block for N has already been committed by the builder). +- The IL is put in the `BeaconState`. +- The builder for slot N reveals its payload. This payload also contains a `beacon_state_root`. Validators for this slot will remove from the IL any transaction that was already executed during slot N (for example the builder may have independently included this transaction) or that became invalid because of other transactions from the same sender that appeared during N. They also check that the resulting `BeaconState` has the same `beacon_state_root` committed to by the builder. The upshot of this step is that the IL in the beacon state contains transactions that are guaranteed to be valid and executable during the beginning of N+1. +- The proposer for N+1 produces a block with its own IL for N+2. The builder for N+1 reveals its payload, and validators deem it invalid if the first transactions do not agree with the corresponding IL exactly. + +**Note:** in the event that the payload for the canonical block in slot N is not revealed, then the IL for slot N remains valid, the proposer for slot N+1 is not allowed to include a new IL. + +There are some concerns about proposers using IL for data availability, since the CL will have to keep the blocks somewhere to reconstruct the beacon state. There is a nice design by @vbuterin that instead of committing the IL to state, allows the txs to go on a sidecar together with a signed summary. The builder then needs to include the signed summary and a block that satisfies it. This design trades complexity for safety under the free DA issue of the above. + + From e6b24a3584cfa7e46b50df289e1708d903636d33 Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 8 Aug 2023 13:26:12 -0300 Subject: [PATCH 04/57] check for gas usage after payload insertion Also notice that the DA issue only happens if all the transactions are invalidated in the same slot --- specs/_features/epbs/beacon-chain.md | 4 +++- specs/_features/epbs/design.md | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 7383485ddb..d1611f0c88 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -273,7 +273,9 @@ def verify_and_notify_new_payload(self: ExecutionEngine, Raise an exception if ``execution_payload`` is not valid with respect to ``self.execution_state``. Returns the list of transactions in the inclusion list that remain valid after executing the payload. That is it is guaranteed that the transactions returned in the list can be executed in the exact order starting from the - current ``self.execution_state``. + current ``self.execution_state``. This check also includes that the transactions still use less than + ``MAX_GAS_PER_INCLUSION_LIST``, since the gas usage may have been different if the transaction was + executed before or after slot N """ assert self.is_valid_block_hash(new_payload_request.execution_payload) return self.notify_new_payload(new_payload_request.execution_payload) diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 1cc3d86aa9..a89c1dddef 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -12,6 +12,6 @@ ePBS introduces forward inclusion lists for proposers to guarantee censor resist **Note:** in the event that the payload for the canonical block in slot N is not revealed, then the IL for slot N remains valid, the proposer for slot N+1 is not allowed to include a new IL. -There are some concerns about proposers using IL for data availability, since the CL will have to keep the blocks somewhere to reconstruct the beacon state. There is a nice design by @vbuterin that instead of committing the IL to state, allows the txs to go on a sidecar together with a signed summary. The builder then needs to include the signed summary and a block that satisfies it. This design trades complexity for safety under the free DA issue of the above. +There are some concerns about proposers using IL for data availability, since the CL will have to keep the blocks somewhere to reconstruct the beacon state. A proposer may freely include a IL in a block by including transactions and invalidating them all in the payload for the same slot N. There is a nice design by @vbuterin that instead of committing the IL to state, allows the txs to go on a sidecar together with a signed summary. The builder then needs to include the signed summary and a block that satisfies it. This design trades complexity for safety under the free DA issue of the above. From f701bf8314faa1d3f2790123831993a4abfb3e07 Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 8 Aug 2023 15:05:04 -0300 Subject: [PATCH 05/57] Add helpers that change on ePBS --- specs/_features/epbs/beacon-chain.md | 172 ++++++++++++++++++++++++++- specs/_features/epbs/design.md | 7 ++ 2 files changed, 175 insertions(+), 4 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index d1611f0c88..ffcf96ee4d 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -24,6 +24,14 @@ At any given slot, the status of the blockchain's head may be either For a further introduction please refer to this [ethresear.ch article](https://ethresear.ch/t/payload-timeliness-committee-ptc-an-epbs-design/16054) +## Configuration + +### Time parameters + +| Name | Value | Unit | Duration | +| - | - | :-: | :-: | +| `SECONDS_PER_SLOT` | `uint64(16)` | seconds | 16 seconds # (Modified in ePBS) | + ## Preset ### Misc @@ -64,7 +72,7 @@ For a further introduction please refer to this [ethresear.ch article](https://e ## Containers -### New Containers +### New containers #### `Builder` @@ -73,6 +81,7 @@ class Builder(Container): pubkey: BLSPubkey withdrawal_address: ExecutionAddress # Commitment to pubkey for withdrawals effective_balance: Gwei # Balance at stake + slashed: boolean exit_epoch: Epoch withdrawable_epoch: Epoch # When builder can withdraw funds ``` @@ -101,7 +110,7 @@ class SignedExecutionPayloadEnvelope(Container): signature: BLSSignature ``` -### Modified Containers +### Modified containers #### `ExecutionPayload` @@ -226,6 +235,163 @@ class BeaconState(Container): current_tx_inclusion_list: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] current_signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] ``` +## Helper functions + +### Predicates + +#### Modified `is_active_builder` + +```python +def is_active_builder(builder: Builder, epoch: Epoch) -> bool: + return epoch < builder.exit_epoch +``` +### Misc + +#### Modified `compute_proposer_index` +*Note*: `compute_proposer_index` is modified to account for builders being validators + +TODO: actually do the sampling proportional to effective balance + +### Beacon state accessors + +#### Modified `get_active_validator_indices` + +```python +def get_active_validator_indices(state: BeaconState, epoch: Epoch) -> Sequence[ValidatorIndex]: + """ + Return the sequence of active validator indices at ``epoch``. + """ + builder_indices = [ValidatorIndex(len(state.validators) + i) for i,b in enumerate(state.builders) if is_active_builder(b,epoch)] + return [ValidatorIndex(i) for i, v in enumerate(state.validators) if is_active_validator(v, epoch)] + builder_indices +``` + +#### New `get_effective_balance` + +```python +def get_effective_balance(state: BeaconState, index: ValidatorIndex) -> Gwei: + """ + Return the effective balance for the validator or the builder indexed by ``index`` + """ + if index < len(state.validators): + return state.validators[index].effective_balance + return state.builders[index-len(state.validators)].effective_balance +``` + +#### Modified `get_total_balance` + +```python +def get_total_balance(state: BeaconState, indices: Set[ValidatorIndex]) -> Gwei: + """ + Return the combined effective balance of the ``indices``. + ``EFFECTIVE_BALANCE_INCREMENT`` Gwei minimum to avoid divisions by zero. + Math safe up to ~10B ETH, after which this overflows uint64. + """ + return Gwei(max(EFFECTIVE_BALANCE_INCREMENT, sum([get_effective_balance(state, index) for index in indices]))) +``` + +### Beacon state mutators + +#### Modified `increase_balance` + +```python +def increase_balance(state: BeaconState, index: ValidatorIndex, delta: Gwei) -> None: + """ + Increase the validator balance at index ``index`` by ``delta``. + """ + if index < len(state.validators): + state.balances[index] += delta + return + state.builder_balances[index-len(state.validators)] += delta +``` + +#### Modified `decrease_balance` + +```python +def decrease_balance(state: BeaconState, index: ValidatorIndex, delta: Gwei) -> None: + """ + Decrease the validator balance at index ``index`` by ``delta``, with underflow protection. + """ + if index < len(state.validators) + state.balances[index] = 0 if delta > state.balances[index] else state.balances[index] - delta + return + index -= len(state.validators) + state.builder_balances[index] = 0 if delta > state.builder_balances[index] else state.builder_balances[index] - delta +``` + +#### Modified `initiate_validator_exit` + +```python +def initiate_validator_exit(state: BeaconState, index: ValidatorIndex) -> None: + """ + Initiate the exit of the validator with index ``index``. + """ + # Notice that the local variable ``validator`` may refer to a builder. Also that it continues defined outside + # its declaration scope. This is valid Python. + if index < len(state.validators): + validator = state.validators[index] + else: + validator = state.builders[index - len(state.validators)] + + # Return if validator already initiated exit + if validator.exit_epoch != FAR_FUTURE_EPOCH: + return + + # Compute exit queue epoch + exit_epochs = [v.exit_epoch for v in state.validators + state.builders if v.exit_epoch != FAR_FUTURE_EPOCH] + exit_queue_epoch = max(exit_epochs + [compute_activation_exit_epoch(get_current_epoch(state))]) + exit_queue_churn = len([v for v in state.validators + state.builders if v.exit_epoch == exit_queue_epoch]) + if exit_queue_churn >= get_validator_churn_limit(state): + exit_queue_epoch += Epoch(1) + + # Set validator exit epoch and withdrawable epoch + validator.exit_epoch = exit_queue_epoch + validator.withdrawable_epoch = Epoch(validator.exit_epoch + MIN_VALIDATOR_WITHDRAWABILITY_DELAY) # TODO: Do we want to differentiate builders here? +``` + +#### New `proposer_slashing_amount` + +```python +def proposer_slashing_amount(slashed_index: ValidatorIndex, effective_balance: Gwei): + return min(MAX_EFFECTIVE_BALANCE, effective_balance) // MIN_SLASHING_PENALTY_QUOTIENT +``` + +#### Modified `slash_validator` + +```python +def slash_validator(state: BeaconState, + slashed_index: ValidatorIndex, + proposer_slashing: bool, + whistleblower_index: ValidatorIndex=None) -> None: + """ + Slash the validator with index ``slashed_index``. + """ + epoch = get_current_epoch(state) + initiate_validator_exit(state, slashed_index) + # Notice that the local variable ``validator`` may refer to a builder. Also that it continues defined outside + # its declaration scope. This is valid Python. + if index < len(state.validators): + validator = state.validators[slashed_index] + else: + validator = state.builders[slashed_index - len(state.validators)] + validator.slashed = True + validator.withdrawable_epoch = max(validator.withdrawable_epoch, Epoch(epoch + EPOCHS_PER_SLASHINGS_VECTOR)) + state.slashings[epoch % EPOCHS_PER_SLASHINGS_VECTOR] += validator.effective_balance + if proposer_slashing: + decrease_balance(state, slashed_index, proposer_slashing_amount(slashed_index, validator.effective_balance)) + else: + decrease_balance(state, slashed_index, validator.effective_balance // MIN_SLASHING_PENALTY_QUOTIENT) + + # Apply proposer and whistleblower rewards + proposer_index = get_beacon_proposer_index(state) + if whistleblower_index is None: + whistleblower_index = proposer_index + whistleblower_reward = Gwei(max(MAX_EFFECTIVE_BALANCE, validator.effective_balance) // WHISTLEBLOWER_REWARD_QUOTIENT) + proposer_reward = Gwei(whistleblower_reward // PROPOSER_REWARD_QUOTIENT) + increase_balance(state, proposer_index, proposer_reward) + increase_balance(state, whistleblower_index, Gwei(whistleblower_reward - proposer_reward)) +``` + + ## Beacon chain state transition function @@ -362,8 +528,6 @@ def process_execution_payload_header(state: BeaconState, signed_header: SignedEx #### Modified `process_execution_payload` *Note*: `process_execution_payload` is now an independent check in state transition. It is called when importing a signed execution payload proposed by the builder of the current slot. -TODO: Deal with the case when the payload becomes invalid because of the forward inclusion list. - ```python def process_execution_payload(state: BeaconState, signed_envelope: SignedExecutionPayloadEnvelope, execution_engine: ExecutionEngine) -> None: # Verify signature [New in ePBS] diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index a89c1dddef..9b91edafc2 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -14,4 +14,11 @@ ePBS introduces forward inclusion lists for proposers to guarantee censor resist There are some concerns about proposers using IL for data availability, since the CL will have to keep the blocks somewhere to reconstruct the beacon state. A proposer may freely include a IL in a block by including transactions and invalidating them all in the payload for the same slot N. There is a nice design by @vbuterin that instead of committing the IL to state, allows the txs to go on a sidecar together with a signed summary. The builder then needs to include the signed summary and a block that satisfies it. This design trades complexity for safety under the free DA issue of the above. +## Builders +There is a new entity `Builder` that is a glorified validator required to have a higher stake and required to sign when producing execution payloads. + +- There is a new list in the `BeaconState` that contains all the registered builders +- Builders are also validators (otherwise their staked capital depreciates) +- We onboard builders by simply turning validators into builders if they achieve the necessary minimum balance (this way we avoid two forks to onboard builders and keep the same deposit flow, avoid builders to skip the entry churn) +- The unit `ValidatorIndex` is used for both indexing validators and builders, after all, builders are validators. Throughout the code, we often see checks of the form `index < len(state.validators)`, thus we consider a `ValidatorIndex(len(state.validators))` to correspond to the first builder, that is `state.builders[0]`. From 97b2f400d56afc27b9c49fa990d4791ea485bc27 Mon Sep 17 00:00:00 2001 From: Potuz Date: Wed, 9 Aug 2023 08:23:57 -0300 Subject: [PATCH 06/57] switch to gas limit instead of usage as pointed by V. Buterin --- specs/_features/epbs/beacon-chain.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index ffcf96ee4d..e93e80e8e9 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -392,7 +392,6 @@ def slash_validator(state: BeaconState, ``` - ## Beacon chain state transition function *Note*: state transition is fundamentally modified in ePBS. The full state transition is broken in two parts, first importing a signed block and then importing an execution payload. @@ -439,9 +438,7 @@ def verify_and_notify_new_payload(self: ExecutionEngine, Raise an exception if ``execution_payload`` is not valid with respect to ``self.execution_state``. Returns the list of transactions in the inclusion list that remain valid after executing the payload. That is it is guaranteed that the transactions returned in the list can be executed in the exact order starting from the - current ``self.execution_state``. This check also includes that the transactions still use less than - ``MAX_GAS_PER_INCLUSION_LIST``, since the gas usage may have been different if the transaction was - executed before or after slot N + current ``self.execution_state``. """ assert self.is_valid_block_hash(new_payload_request.execution_payload) return self.notify_new_payload(new_payload_request.execution_payload) @@ -454,7 +451,7 @@ def notify_new_inclusion_list(self: ExecutionEngine, inclusion_list_request: NewInclusionListRequest) -> bool: """ Return ``True`` if and only if the transactions in the inclusion list can be succesfully executed - starting from the current ``self.execution_state`` and that they consume less or equal than + starting from the current ``self.execution_state`` and that their total gas limit is less or equal than ```MAX_GAS_PER_INCLUSION_LIST``. """ ... From 0bf73825aee786e15d199da1a071dda504114cd7 Mon Sep 17 00:00:00 2001 From: Potuz Date: Wed, 9 Aug 2023 11:27:55 -0300 Subject: [PATCH 07/57] Added more changed helpers Notice that the whole thing is broken by the construct if index < len(validators) else. --- specs/_features/epbs/beacon-chain.md | 219 ++++++++++++++++++++++++++- 1 file changed, 218 insertions(+), 1 deletion(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index e93e80e8e9..32d00895cf 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -231,6 +231,8 @@ class BeaconState(Container): # PBS builders: List[Builder, BUILDER_REGISTRY_LIMIT] # [New in ePBS] builder_balances: List[Gwei, BUILDER_REGISTRY_LIMIT] # [New in ePBS] + previous_epoch_builder_participation: List[ParticipationFlags, BUILDER_REGISTRY_LIMIT] # [New in ePBS] + current_epoch_builder_participation: List[ParticipationFlags, BUILDER_REGISTRY_LIMIT] # [New in ePBS] previous_tx_inclusion_list: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] current_tx_inclusion_list: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] current_signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] @@ -239,12 +241,51 @@ class BeaconState(Container): ### Predicates -#### Modified `is_active_builder` +#### New `is_active_builder` ```python def is_active_builder(builder: Builder, epoch: Epoch) -> bool: return epoch < builder.exit_epoch ``` + +#### New `is_slashable_builder` + +```python +def is_slashable_builder(builder: Builder, epoch: Epoch) -> bool: + """ + Check if ``builder`` is slashable. + """ + return (not validator.slashed) and (epoch < builder.withdrawable_epoch) +``` + +#### New `is_active_validator_at_index` + +```python +def is_active_validator_at_index(state: BeaconState, index: ValidatorIndex, epoch: Epoch) -> bool: + if index < len(state.validators): + return is_active_validator(state.validators[index], epoch) + return is_active_builder(state.builders[index-len(state.validators)], epoch) +``` + +#### Modified `is_valid_indexed_attestation` + +```python +def is_valid_indexed_attestation(state: BeaconState, indexed_attestation: IndexedAttestation) -> bool: + """ + Check if ``indexed_attestation`` is not empty, has sorted and unique indices and has a valid aggregate signature. + """ + # Verify indices are sorted and unique + indices = indexed_attestation.attesting_indices + if len(indices) == 0 or not indices == sorted(set(indices)): + return False + # Verify aggregate signature + pubkeys = [state.validators[i].pubkey if i < len(state.validators) else state.builders[i - len(state.validators)].pubkey for i in indices] + domain = get_domain(state, DOMAIN_BEACON_ATTESTER, indexed_attestation.data.target.epoch) + signing_root = compute_signing_root(indexed_attestation.data, domain) + return bls.FastAggregateVerify(pubkeys, signing_root, indexed_attestation.signature) +``` + + ### Misc #### Modified `compute_proposer_index` @@ -289,6 +330,96 @@ def get_total_balance(state: BeaconState, indices: Set[ValidatorIndex]) -> Gwei: return Gwei(max(EFFECTIVE_BALANCE_INCREMENT, sum([get_effective_balance(state, index) for index in indices]))) ``` +#### Modified `get_next_sync_committee_indices` + +*TODO*: make the shuffling actually weighted by the builder's effective balance + +```python +def get_next_sync_committee_indices(state: BeaconState) -> Sequence[ValidatorIndex]: + """ + Return the sync committee indices, with possible duplicates, for the next sync committee. + """ + epoch = Epoch(get_current_epoch(state) + 1) + + MAX_RANDOM_BYTE = 2**8 - 1 + active_validator_indices = get_active_validator_indices(state, epoch) + active_validator_count = uint64(len(active_validator_indices)) + seed = get_seed(state, epoch, DOMAIN_SYNC_COMMITTEE) + i = 0 + sync_committee_indices: List[ValidatorIndex] = [] + while len(sync_committee_indices) < SYNC_COMMITTEE_SIZE: + shuffled_index = compute_shuffled_index(uint64(i % active_validator_count), active_validator_count, seed) + candidate_index = active_validator_indices[shuffled_index] + random_byte = hash(seed + uint_to_bytes(uint64(i // 32)))[i % 32] + if candidate_index >= len(state.validators): + sync_commitee_indices.append(candidate_index) + else: + effective_balance = state.validators[candidate_index].effective_balance + if effective_balance * MAX_RANDOM_BYTE >= MAX_EFFECTIVE_BALANCE * random_byte: + sync_committee_indices.append(candidate_index) + i += 1 + return sync_committee_indices +``` + +#### Modified `get_next_sync_committee` + +```python +def get_next_sync_committee(state: BeaconState) -> SyncCommittee: + """ + Return the next sync committee, with possible pubkey duplicates. + """ + indices = get_next_sync_committee_indices(state) + pubkeys = [state.validators[index].pubkey if index < len(state.validators) else state.builders[index-len(state.validators)] for index in indices] + aggregate_pubkey = eth_aggregate_pubkeys(pubkeys) + return SyncCommittee(pubkeys=pubkeys, aggregate_pubkey=aggregate_pubkey) +``` + +#### Modified `get_unslashed_participating_indices` + +```python +def get_unslashed_participating_indices(state: BeaconState, flag_index: int, epoch: Epoch) -> Set[ValidatorIndex]: + """ + Return the set of validator indices that are both active and unslashed for the given ``flag_index`` and ``epoch``. + """ + assert epoch in (get_previous_epoch(state), get_current_epoch(state)) + if epoch == get_current_epoch(state): + epoch_participation = state.current_epoch_participation + epoch_builder_participation = state.current_epoch_builder_participation + else: + epoch_participation = state.previous_epoch_participation + epoch_builder_participation = state.previous_epoch_builder_participation + active_validator_indices = get_active_validator_indices(state, epoch) + participating_indices = [i for i in active_validator_indices if (has_flag(epoch_participation[i], flag_index) if i < len(state.validators) else has_flag(epoch_builder_participation[i-len(state.validators)], flag_index))] + return set(filter(lambda index: not state.validators[index].slashed if index < len(state.validators) else not state.builders[index-len(state.validators)].slashed, participating_indices)) +``` + +#### Modified `get_flag_index_deltas` + +```python +def get_flag_index_deltas(state: BeaconState, flag_index: int) -> Tuple[Sequence[Gwei], Sequence[Gwei]]: + """ + Return the deltas for a given ``flag_index`` by scanning through the participation flags. + """ + rewards = [Gwei(0)] * (len(state.validators) + len(state.builders)) + penalties = [Gwei(0)] * (len(state.validators) + len(state.builders) + previous_epoch = get_previous_epoch(state) + unslashed_participating_indices = get_unslashed_participating_indices(state, flag_index, previous_epoch) + weight = PARTICIPATION_FLAG_WEIGHTS[flag_index] + unslashed_participating_balance = get_total_balance(state, unslashed_participating_indices) + unslashed_participating_increments = unslashed_participating_balance // EFFECTIVE_BALANCE_INCREMENT + active_increments = get_total_active_balance(state) // EFFECTIVE_BALANCE_INCREMENT + for index in get_eligible_validator_indices(state): + base_reward = get_base_reward(state, index) + if index in unslashed_participating_indices: + if not is_in_inactivity_leak(state): + reward_numerator = base_reward * weight * unslashed_participating_increments + rewards[index] += Gwei(reward_numerator // (active_increments * WEIGHT_DENOMINATOR)) + elif flag_index != TIMELY_HEAD_FLAG_INDEX: + penalties[index] += Gwei(base_reward * weight // WEIGHT_DENOMINATOR) + return rewards, penalties +``` + + ### Beacon state mutators #### Modified `increase_balance` @@ -400,6 +531,92 @@ The post-state corresponding to a pre-state `state` and a signed block `signed_b The post-state corresponding to a pre-state `state` and a signed execution payload `signed_execution_payload` is defined as `process_execution_payload(state, signed_execution_payload)`. State transitions that trigger an unhandled exception (e.g. a failed `assert` or an out-of-range list access) are considered invalid. State transitions that cause a `uint64` overflow or underflow are also considered invalid. +### Modified `verify_block_signature` + +```python +def verify_block_signature(state: BeaconState, signed_block: SignedBeaconBlock) -> bool: + index = signed_block.message.proposer_index + if index < len(state.validators): + proposer = state.validators[index] + else: + proposer = state.builders[index-len(state.validators)] + signing_root = compute_signing_root(signed_block.message, get_domain(state, DOMAIN_BEACON_PROPOSER)) + return bls.Verify(proposer.pubkey, signing_root, signed_block.signature) +``` + +### Epoch processing + +#### Modified `process_epoch` + +```python +def process_epoch(state: BeaconState) -> None: + process_justification_and_finalization(state) + process_inactivity_updates(state) + process_rewards_and_penalties(state) + process_registry_updates(state) + process_slashings(state) + process_eth1_data_reset(state) + process_effective_balance_updates(state) + process_slashings_reset(state) + process_randao_mixes_reset(state) + process_historical_summaries_update(state) + process_participation_flag_updates(state) # [Modified in ePBS] + process_sync_committee_updates(state) + process_builder_updates(state) # [New in ePBS] +``` + +#### Modified `process_participation_flag_updates` + +```python +def process_participation_flag_updates(state: BeaconState) -> None: + state.previous_epoch_participation = state.current_epoch_participation + state.current_epoch_participation = [ParticipationFlags(0b0000_0000) for _ in range(len(state.validators))] + state.previous_epoch_builder_participation = state.current_epoch_builder_participation + state.current_epoch_builder_participation = [ParticipationFlags(0b0000_0000) for _ in range(len(state.builders))] +``` + +#### Rewards and penalties + +##### Helpers + +*Note*: the function `get_base_reward` is modified to account for builders. + +```python +def get_base_reward(state: BeaconState, index: ValidatorIndex) -> Gwei: + """ + Return the base reward for the validator defined by ``index`` with respect to the current ``state``. + """ + if index < len(state.validators): + validator = state.validators[index] + else: + validator = state.builders[index-len(state.validators)] + increments = validator.effective_balance // EFFECTIVE_BALANCE_INCREMENT + return Gwei(increments * get_base_reward_per_increment(state)) +``` + +*Note*: The function `is_active_validator_at_index` is new + +```python +def is_active_validator_at_index(state: BeaconState, index: ValidatorIndex) -> Bool: + if index < len(state.validators): + validator = state.validators[index] + else: + validator = state + +``` + +*Note*: The function `get_eligible_validator_indices` is modified to account for builders. + +```python +def get_eligible_validator_indices(state: BeaconState) -> Sequence[ValidatorIndex]: + previous_epoch = get_previous_epoch(state) + return [ + ValidatorIndex(index) for index, v in enumerate(state.validators + state.builders) + if is_active_validator(v, previous_epoch) or (v.slashed and previous_epoch + 1 < v.withdrawable_epoch) + ] +``` + + ### Execution engine #### Request data From 28e6f9f239d0c383a8d44fc3e40e530cc9d79b33 Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 22 Aug 2023 13:18:51 -0300 Subject: [PATCH 08/57] change design notes --- specs/_features/epbs/design.md | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 9b91edafc2..cd76fc8431 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -2,23 +2,22 @@ ## Inclusion lists -ePBS introduces forward inclusion lists for proposers to guarantee censor resistanship of the network. They are implemented as follows +ePBS introduces forward inclusion lists for proposers to guarantee censor resistanship of the network. We follow the design described in [this post](https://ethresear.ch/t/no-free-lunch-a-new-inclusion-list-design/16389). -- Proposer for slot N submits a signed block that includes some transactions to be included at the beginning of slot N+1. -- Validators for slot N will consider the block invalid if those transactions are not executable at the start of slot N (this makes it impossible to put transactions that are only valid at slot N+1 for example, but still the proposer can protect from unbundling/replaying by binding the transaction to only be valid up to N+1 for example, because the block for N has already been committed by the builder). -- The IL is put in the `BeaconState`. -- The builder for slot N reveals its payload. This payload also contains a `beacon_state_root`. Validators for this slot will remove from the IL any transaction that was already executed during slot N (for example the builder may have independently included this transaction) or that became invalid because of other transactions from the same sender that appeared during N. They also check that the resulting `BeaconState` has the same `beacon_state_root` committed to by the builder. The upshot of this step is that the IL in the beacon state contains transactions that are guaranteed to be valid and executable during the beginning of N+1. -- The proposer for N+1 produces a block with its own IL for N+2. The builder for N+1 reveals its payload, and validators deem it invalid if the first transactions do not agree with the corresponding IL exactly. +- Proposer for slot N submits a signed block and in parallel broadcasts pairs of `summaries` and `transactions` to be included at the beginning of slot N+1. `transactions` are just list of transactions that this proposer wants included at the most at the beginning of N+1. `Summaries` are lists consisting on addresses sending those transactions and their gas limits. The summaries are signed, the transactions aren't. An honest proposer is allowed to send many of these pairs that aren't committed to its beacon block so no double proposing slashing is involved. +- Validators for slot N will consider the block for validation only if they have seen at least one pair (summary, transactions). They will consider the block invalid if those transactions are not executable at the start of slot N and if they don't have at least 12.5% higher `maxFeePerGas` than the current slot's `maxFeePerGas`. +- The builder for slot N reveals its payload together with a signed summary of the proposer of slot N-1. The payload is considered only valid if the following applies + - Let k >= 0 be the minimum such that tx[0],...,tx[k-1], the first `k` transactions of the payload of slot N, satisfy some entry in the summary and `tx[k]` does not satisfy any entry. + - There exist transactions in the payload for N-1 that satisfy all the remaining entries in the summary. + - The payload is executable, that is, it's valid from the execution layer perspective. -**Note:** in the event that the payload for the canonical block in slot N is not revealed, then the IL for slot N remains valid, the proposer for slot N+1 is not allowed to include a new IL. +**Note:** in the event that the payload for the canonical block in slot N is not revealed, then the summaries and transactions list for slot N-1 remains valid, the honest proposer for slot N+1 is not allowed to submit a new IL and any such message will be ignored. The builder for N+1 still has to satisfy the summary of N-1. If there are k slots in a row that are missing payloads, the next full slot will still need to satisfy the inclusion list for N-1. -There are some concerns about proposers using IL for data availability, since the CL will have to keep the blocks somewhere to reconstruct the beacon state. A proposer may freely include a IL in a block by including transactions and invalidating them all in the payload for the same slot N. There is a nice design by @vbuterin that instead of committing the IL to state, allows the txs to go on a sidecar together with a signed summary. The builder then needs to include the signed summary and a block that satisfies it. This design trades complexity for safety under the free DA issue of the above. ## Builders There is a new entity `Builder` that is a glorified validator required to have a higher stake and required to sign when producing execution payloads. -- There is a new list in the `BeaconState` that contains all the registered builders -- Builders are also validators (otherwise their staked capital depreciates) -- We onboard builders by simply turning validators into builders if they achieve the necessary minimum balance (this way we avoid two forks to onboard builders and keep the same deposit flow, avoid builders to skip the entry churn) -- The unit `ValidatorIndex` is used for both indexing validators and builders, after all, builders are validators. Throughout the code, we often see checks of the form `index < len(state.validators)`, thus we consider a `ValidatorIndex(len(state.validators))` to correspond to the first builder, that is `state.builders[0]`. +- Builders are also validators (otherwise their staked capital depreciates). +- We onboard builders by simply turning validators into builders if they achieve the necessary minimum balance (this way we avoid two forks to onboard builders and keep the same deposit flow, avoid builders to skip the entry churn), we change their withdrawal prefix to be distinguished from normal validators. +- We need to include several changes from the [MaxEB PR](https://github.com/michaelneuder/consensus-specs/pull/3) in order to account with builders having an increased balance that would otherwise depreciate. From 48415b10c4a3aa0407e4e38bc50b93dcb00df83a Mon Sep 17 00:00:00 2001 From: Potuz Date: Fri, 25 Aug 2023 08:12:20 -0300 Subject: [PATCH 09/57] minor changes --- specs/_features/epbs/beacon-chain.md | 112 +++++++++++++++------------ 1 file changed, 62 insertions(+), 50 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 32d00895cf..077c16ed7b 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -24,6 +24,15 @@ At any given slot, the status of the blockchain's head may be either For a further introduction please refer to this [ethresear.ch article](https://ethresear.ch/t/payload-timeliness-committee-ptc-an-epbs-design/16054) +## Constants + +### Withdrawal prefixes + +| Name | Value | +| - | - | +| `BUILDER_WITHDRAWAL_PREFIX` | `Bytes1('0x0b')` # (New in ePBS) | + + ## Configuration ### Time parameters @@ -44,19 +53,22 @@ For a further introduction please refer to this [ethresear.ch article](https://e | Name | Value | | - | - | -| `DOMAIN_BEACON_BUILDER` | `DomainType('0x0B000000')` | - -### State list lengths - -| Name | Value | Unit | Duration | -| - | - | :-: | :-: | -| `BUILDER_REGISTRY_LIMIT` | `uint64(2**20)` (=1,048,576) | builders | +| `DOMAIN_BEACON_BUILDER` | `DomainType('0x0B000000')` # (New in ePBS)| ### Gwei values | Name | Value | | - | - | -| `BUILDER_MIN_BALANCE` | `Gwei(2**10 * 10**9)` = (1,024,000,000,000) | +| `BUILDER_MIN_BALANCE` | `Gwei(2**10 * 10**9)` = (1,024,000,000,000) # (New in ePBS)| +| `MIN_ACTIVATION_BALANCE` | `Gwei(2**5 * 10**9)` (= 32,000,000,000) # (New in ePBS)| +| `EFFECTIVE_BALANCE_INCREMENT` | `Gwei(2**0 * 10**9)` (= 1,000,000,000) # (New in ePBS)| +| `MAX_EFFECTIVE_BALANCE` | `Gwei(2**11 * 10**9)` = (2,048,000,000,000) # (Modified in ePBS) | + +### Rewards and penalties + +| Name | Value | +| - | - | +| `PROPOSER_EQUIVOCATION_PENALTY_FACTOR` | `uint64(2**2)` (= 4) # (New in ePBS)| ### Incentivization weights @@ -68,24 +80,12 @@ For a further introduction please refer to this [ethresear.ch article](https://e | Name | Value | | - | - | | MAX_TRANSACTIONS_PER_INCLUSION_LIST | `2**4` (=16) | -| MAX_GAS_PER_INCLUSION_LIST | `2**20` (=1,048,576) | +| MAX_GAS_PER_INCLUSION_LIST | `2**21` (=2,097,152) | ## Containers ### New containers -#### `Builder` - -``` python -class Builder(Container): - pubkey: BLSPubkey - withdrawal_address: ExecutionAddress # Commitment to pubkey for withdrawals - effective_balance: Gwei # Balance at stake - slashed: boolean - exit_epoch: Epoch - withdrawable_epoch: Epoch # When builder can withdraw funds -``` - #### `SignedExecutionPayloadHeader` ```python @@ -99,6 +99,8 @@ class SignedExecutionPayloadHeader(Container): ```python class ExecutionPayloadEnvelope(Container): payload: ExecutionPayload + builder_index: ValidatorIndex + value: Gwei state_root: Root ``` @@ -110,10 +112,44 @@ class SignedExecutionPayloadEnvelope(Container): signature: BLSSignature ``` +#### `InclusionListSummaryEntry` + +```python +class InclusionListSummaryEntry(Container): + address: ExecutionAddress + gas_limit: uint64 +``` + +#### `InclusionListSummary` + +```python +class InclusionListSummary(Container) + proposer_index: ValidatorIndex + summary: List[InclusionListSummaryEntry, MAX_TRANSACTIONS_PER_INCLUSION_LIST] +``` + +#### `SignedInclusionListSummary` + +```python +class SignedInclusionListSummary(Container): + message: InclusionListSummary + signature: BLSSignature +``` + +#### `InclusionList` + +```python +class InclusionList(Container) + summary: SignedInclusionListSummary + transactions: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] +``` + ### Modified containers #### `ExecutionPayload` +**Note:** The `ExecutionPayload` is modified to contain the builder's index and the bid value. It also contains a transaction inclusion list summary signed by the corresponding beacon block proposer and the list of indices of transactions in the parent block that have to be excluded from the inclusion list summary because they were satisfied in the previous slot. + ```python class ExecutionPayload(Container): # Execution block header fields @@ -133,12 +169,14 @@ class ExecutionPayload(Container): block_hash: Hash32 # Hash of execution block transactions: List[Transaction, MAX_TRANSACTIONS_PER_PAYLOAD] withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] - builder_index: uint64 # [New in ePBS] - value: Gwei # [New in ePBS] + inclusion_list_summary: SignedInclusionListSummary # [New in ePBS] + inclusion_list_exclusions: List[uint64, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] ``` #### `ExecutionPayloadHeader` +**Note:** The `ExecutionPayloadHeader` is modified to include the builder's index and the bid's value. + ```python class ExecutionPayloadHeader(Container): # Execution block header fields @@ -162,28 +200,8 @@ class ExecutionPayloadHeader(Container): value: Gwei # [New in ePBS] ``` -#### `BeaconBlockBody` - -```python -class BeaconBlockBody(Container): - randao_reveal: BLSSignature - eth1_data: Eth1Data # Eth1 data vote - graffiti: Bytes32 # Arbitrary data - # Operations - proposer_slashings: List[ProposerSlashing, MAX_PROPOSER_SLASHINGS] - attester_slashings: List[AttesterSlashing, MAX_ATTESTER_SLASHINGS] - attestations: List[Attestation, MAX_ATTESTATIONS] - deposits: List[Deposit, MAX_DEPOSITS] - voluntary_exits: List[SignedVoluntaryExit, MAX_VOLUNTARY_EXITS] - sync_aggregate: SyncAggregate - execution_payload_header: SignedExecutionPayloadHeader # [Modified in ePBS] - bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES] - tx_inclusion_list: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] -``` - - #### `BeaconState` -*Note*: the beacon state is modified to store a signed latest execution payload header and it adds a registry of builders, their balances and two transaction inclusion lists. +*Note*: the beacon state is modified to store a signed latest execution payload header. ```python class BeaconState(Container): @@ -229,12 +247,6 @@ class BeaconState(Container): # Deep history valid from Capella onwards historical_summaries: List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT] # PBS - builders: List[Builder, BUILDER_REGISTRY_LIMIT] # [New in ePBS] - builder_balances: List[Gwei, BUILDER_REGISTRY_LIMIT] # [New in ePBS] - previous_epoch_builder_participation: List[ParticipationFlags, BUILDER_REGISTRY_LIMIT] # [New in ePBS] - current_epoch_builder_participation: List[ParticipationFlags, BUILDER_REGISTRY_LIMIT] # [New in ePBS] - previous_tx_inclusion_list: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] - current_tx_inclusion_list: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] current_signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] ``` ## Helper functions From 8b69b9bf2a22d5489f23e6bd48f7b9f033356d94 Mon Sep 17 00:00:00 2001 From: Potuz Date: Fri, 25 Aug 2023 15:21:19 -0300 Subject: [PATCH 10/57] First forkchoice changes. - Added handlers for execution payloads and checks inclusion lists availability on consensus blocks --- specs/_features/epbs/beacon-chain.md | 195 +++++++++------------------ specs/_features/epbs/fork-choice.md | 173 ++++++++++++++++++++++++ 2 files changed, 235 insertions(+), 133 deletions(-) create mode 100644 specs/_features/epbs/fork-choice.md diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 077c16ed7b..ebe75c1691 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -64,6 +64,12 @@ For a further introduction please refer to this [ethresear.ch article](https://e | `EFFECTIVE_BALANCE_INCREMENT` | `Gwei(2**0 * 10**9)` (= 1,000,000,000) # (New in ePBS)| | `MAX_EFFECTIVE_BALANCE` | `Gwei(2**11 * 10**9)` = (2,048,000,000,000) # (Modified in ePBS) | +### Time parameters + +| Name | Value | Unit | Duration | +| - | - | :-: | :-: | +| `MIN_SLOTS_FOR_INCLUSION_LISTS_REQUESTS` | `uint64(2)` | slots | 32 seconds # (New in ePBS) | + ### Rewards and penalties | Name | Value | @@ -99,8 +105,7 @@ class SignedExecutionPayloadHeader(Container): ```python class ExecutionPayloadEnvelope(Container): payload: ExecutionPayload - builder_index: ValidatorIndex - value: Gwei + beacon_block_root: Root state_root: Root ``` @@ -146,6 +151,28 @@ class InclusionList(Container) ### Modified containers +#### `BeaconBlockBody` +**Note:** The Beacon Block body is modified to contain a Signed `ExecutionPayloadHeader`. The containers `BeaconBlock` and `SignedBeaconBlock` are modified indirectly. + +```python +class BeaconBlockBody(Container): + randao_reveal: BLSSignature + eth1_data: Eth1Data # Eth1 data vote + graffiti: Bytes32 # Arbitrary data + # Operations + proposer_slashings: List[ProposerSlashing, MAX_PROPOSER_SLASHINGS] + attester_slashings: List[AttesterSlashing, MAX_ATTESTER_SLASHINGS] + attestations: List[Attestation, MAX_ATTESTATIONS] + deposits: List[Deposit, MAX_DEPOSITS] + voluntary_exits: List[SignedVoluntaryExit, MAX_VOLUNTARY_EXITS] + sync_aggregate: SyncAggregate + # Execution + # Removed execution_payload [ Removed in ePBS] + signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] + bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES] + blob_kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK] # [New in Deneb:EIP4844] +``` + #### `ExecutionPayload` **Note:** The `ExecutionPayload` is modified to contain the builder's index and the bid value. It also contains a transaction inclusion list summary signed by the corresponding beacon block proposer and the list of indices of transactions in the parent block that have to be excluded from the inclusion list summary because they were satisfied in the previous slot. @@ -169,6 +196,8 @@ class ExecutionPayload(Container): block_hash: Hash32 # Hash of execution block transactions: List[Transaction, MAX_TRANSACTIONS_PER_PAYLOAD] withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] + builder_index: ValidatorIndex # [New in ePBS] + value: Gwei # [New in ePBS] inclusion_list_summary: SignedInclusionListSummary # [New in ePBS] inclusion_list_exclusions: List[uint64, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] ``` @@ -196,8 +225,10 @@ class ExecutionPayloadHeader(Container): block_hash: Hash32 # Hash of execution block transactions_root: Root withdrawals_root: Root - builder_index: uint64 # [New in ePBS] + builder_index: ValidatorIndex # [New in ePBS] value: Gwei # [New in ePBS] + inclusion_list_summary_root: Root # [New in ePBS] + inclusion_list_exclusions_root: Root # [New in ePBS] ``` #### `BeaconState` @@ -572,63 +603,11 @@ def process_epoch(state: BeaconState) -> None: process_slashings_reset(state) process_randao_mixes_reset(state) process_historical_summaries_update(state) - process_participation_flag_updates(state) # [Modified in ePBS] + process_participation_flag_updates(state) process_sync_committee_updates(state) process_builder_updates(state) # [New in ePBS] ``` -#### Modified `process_participation_flag_updates` - -```python -def process_participation_flag_updates(state: BeaconState) -> None: - state.previous_epoch_participation = state.current_epoch_participation - state.current_epoch_participation = [ParticipationFlags(0b0000_0000) for _ in range(len(state.validators))] - state.previous_epoch_builder_participation = state.current_epoch_builder_participation - state.current_epoch_builder_participation = [ParticipationFlags(0b0000_0000) for _ in range(len(state.builders))] -``` - -#### Rewards and penalties - -##### Helpers - -*Note*: the function `get_base_reward` is modified to account for builders. - -```python -def get_base_reward(state: BeaconState, index: ValidatorIndex) -> Gwei: - """ - Return the base reward for the validator defined by ``index`` with respect to the current ``state``. - """ - if index < len(state.validators): - validator = state.validators[index] - else: - validator = state.builders[index-len(state.validators)] - increments = validator.effective_balance // EFFECTIVE_BALANCE_INCREMENT - return Gwei(increments * get_base_reward_per_increment(state)) -``` - -*Note*: The function `is_active_validator_at_index` is new - -```python -def is_active_validator_at_index(state: BeaconState, index: ValidatorIndex) -> Bool: - if index < len(state.validators): - validator = state.validators[index] - else: - validator = state - -``` - -*Note*: The function `get_eligible_validator_indices` is modified to account for builders. - -```python -def get_eligible_validator_indices(state: BeaconState) -> Sequence[ValidatorIndex]: - previous_epoch = get_previous_epoch(state) - return [ - ValidatorIndex(index) for index, v in enumerate(state.validators + state.builders) - if is_active_validator(v, previous_epoch) or (v.slashed and previous_epoch + 1 < v.withdrawable_epoch) - ] -``` - - ### Execution engine #### Request data @@ -638,41 +617,11 @@ def get_eligible_validator_indices(state: BeaconState) -> Sequence[ValidatorInde ```python @dataclass class NewInclusionListRequest(object): - inclusion_list: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] + inclusion_list: InclusionList ``` #### Engine APIs -#### Modified `notify_new_payload` -*Note*: the function notify new payload is modified to raise an exception if the payload is not valid, and to return the list of transactions that remain valid in the inclusion list - -```python -def notify_new_payload(self: ExecutionEngine, execution_payload: ExecutionPayload) -> List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST]: - """ - Raise an exception if ``execution_payload`` is not valid with respect to ``self.execution_state``. - Returns the list of transactions in the inclusion list that remain valid after executing the payload. That is - it is guaranteed that the transactions returned in the list can be executed in the exact order starting from the - current ``self.execution_state``. - """ - ... -``` - -#### Modified `verify_and_notify_new_payload` -*Note*: the function `verify_and_notify_new_payload` is modified so that it returns the list of transactions that remain valid in the forward inclusion list. It raises an exception if the payload is not valid. - -```python -def verify_and_notify_new_payload(self: ExecutionEngine, - new_payload_request: NewPayloadRequest) -> List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST]: - """ - Raise an exception if ``execution_payload`` is not valid with respect to ``self.execution_state``. - Returns the list of transactions in the inclusion list that remain valid after executing the payload. That is - it is guaranteed that the transactions returned in the list can be executed in the exact order starting from the - current ``self.execution_state``. - """ - assert self.is_valid_block_hash(new_payload_request.execution_payload) - return self.notify_new_payload(new_payload_request.execution_payload) -``` - #### New `notify_new_inclusion_list` ```python @@ -680,8 +629,8 @@ def notify_new_inclusion_list(self: ExecutionEngine, inclusion_list_request: NewInclusionListRequest) -> bool: """ Return ``True`` if and only if the transactions in the inclusion list can be succesfully executed - starting from the current ``self.execution_state`` and that their total gas limit is less or equal than - ```MAX_GAS_PER_INCLUSION_LIST``. + starting from the current ``self.execution_state``, their total gas limit is less or equal that + ```MAX_GAS_PER_INCLUSION_LIST``, And the transactions in the list of transactions correspond to the signed summary """ ... ``` @@ -690,13 +639,10 @@ def notify_new_inclusion_list(self: ExecutionEngine, *Note*: the function `process_block` is modified to only process the consensus block. The full state-transition process is broken into separate functions, one to process a `BeaconBlock` and another to process a `SignedExecutionPayload`. -Notice that `process_tx_inclusion_list` needs to be processed before the payload header since the former requires to check the last committed payload header. - ```python def process_block(state: BeaconState, block: BeaconBlock) -> None: process_block_header(state, block) - process_tx_inclusion_list(state, block, EXECUTION_ENGINE) # [New in ePBS] process_execution_payload_header(state, block.body.execution_payload_header) # [Modified in ePBS] # Removed process_withdrawal in ePBS is processed during payload processing [Modified in ePBS] process_randao(state, block.body) @@ -705,34 +651,14 @@ def process_block(state: BeaconState, block: BeaconBlock) -> None: process_sync_aggregate(state, block.body.sync_aggregate) ``` -#### New `update_tx_inclusion_lists` - -```python -def update_tx_inclusion_lists(state: BeaconState, payload: ExecutionPayload, engine: ExecutionEngine, inclusion_list: List[Transaction, MAX_TRANSACTION_PER_INCLUSION_LIST]) -> None: - old_transactions = payload.transactions[:len(state.previous_tx_inclusion_list)] - assert state.previous_tx_inclusion_list == old_transactions - - state.previous_tx_inclusion_list = inclusion_list -``` - #### New `verify_execution_payload_header_signature` ```python def verify_execution_payload_header_signature(state: BeaconState, signed_header: SignedExecutionPayloadHeader) -> bool: - builder = state.builders[signed_header.message.builder_index] + # Check the signature + builder = state.validators[signed_header.message.builder_index] signing_root = compute_signing_root(signed_header.message, get_domain(state, DOMAIN_BEACON_BUILDER)) - if not bls.Verify(builder.pubkey, signing_root, signed_header.signature): - return False - return -``` - -#### New `verify_execution_payload_signature` - -```python -def verify_execution_envelope_signature(state: BeaconState, signed_envelope: SignedExecutionPayloadEnvelope) -> bool: - builder = state.builders[signed_envelope.message.payload.builder_index] - signing_root = compute_signing_root(signed_envelope.message, get_domain(state, DOMAIN_BEACON_BUILDER)) - return bls.Verify(builder.pubkey, signing_root, signed_envelope.signature) + return bls.Verify(builder.pubkey, signing_root, signed_header.signature) ``` #### New `process_execution_payload_header` @@ -740,7 +666,11 @@ def verify_execution_envelope_signature(state: BeaconState, signed_envelope: Sig ```python def process_execution_payload_header(state: BeaconState, signed_header: SignedExecutionPayloadHeader) -> None: assert verify_execution_payload_header_signature(state, signed_header) + # Check that the builder has funds to cover the bid header = signed_header.message + builder_index = header.builder_index + if state.balances[builder_index] < header.value: + return false # Verify consistency of the parent hash with respect to the previous execution payload header assert header.parent_hash == state.latest_execution_payload_header.block_hash # Verify prev_randao @@ -751,6 +681,15 @@ def process_execution_payload_header(state: BeaconState, signed_header: SignedEx state.current_signed_execution_payload_header = signed_header ``` +#### New `verify_execution_payload_signature` + +```python +def verify_execution_envelope_signature(state: BeaconState, signed_envelope: SignedExecutionPayloadEnvelope) -> bool: + builder = state.validators[signed_envelope.message.payload.builder_index] + signing_root = compute_signing_root(signed_envelope.message, get_domain(state, DOMAIN_BEACON_BUILDER)) + return bls.Verify(builder.pubkey, signing_root, signed_envelope.signature) +``` + #### Modified `process_execution_payload` *Note*: `process_execution_payload` is now an independent check in state transition. It is called when importing a signed execution payload proposed by the builder of the current slot. @@ -764,9 +703,14 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti previous_hash = hash_tree_root(state.current_signed_execution_payload_header.message) assert hash == previous_hash # Verify the execution payload is valid - inclusion_list = execution_engine.verify_and_notify_new_payload(NewPayloadRequest(execution_payload=payload)) - # Verify and update the proposers inclusion lists - update_tx_inclusion_lists(state, payload, inclusion_list) + versioned_hashes = [kzg_commitment_to_versioned_hash(commitment) for commitment in body.blob_kzg_commitments] + assert execution_engine.verify_and_notify_new_payload( + NewPayloadRequest( + execution_payload=payload, + versioned_hashes=versioned_hashes, + parent_beacon_block_root=state.latest_block_header.parent_root, + ) + ) # Process Withdrawals in the payload process_withdrawals(state, payload) # Cache the execution payload header @@ -774,18 +718,3 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti # Verify the state root assert signed_envelope.message.state_root == hash_tree_root(state) ``` - -#### New `process_tx_inclusion_list` - -```python -def process_tx_inclusion_list(state: BeaconState, block: BeaconBlock, execution_engine: ExecutionEngine) -> None: - inclusion_list = block.body.tx_inclusion_list - # Verify that the list is empty if the parent consensus block did not contain a payload - if state.current_signed_execution_payload_header.message != state.latest_execution_payload_header: - assert not inclusion_list - return - assert notify_new_inclusion_list(execution_engine, inclusion_list) - state.previous_tx_inclusion_list = state.current_tx_inclusion_list - state.current_tx_inclusion_list = inclusion_list -``` - diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md new file mode 100644 index 0000000000..6b20ebfec9 --- /dev/null +++ b/specs/_features/epbs/fork-choice.md @@ -0,0 +1,173 @@ +# ePBS -- Fork Choice + +## Table of contents + + + + + + + +## Introduction + +This is the modification of the fork choice accompanying the ePBS upgrade. + +## Helpers + +### `verify_inclusion_list` +*[New in ePBS]* + +```python +def verify_inclusion_list(state: BeaconState, block: BeaconBlock, inclusion_list: InclusionList, execution_engine: ExecutionEngine) -> bool: + """ + returns true if the inclusion list is valid. + """ + # Check that the inclusion list corresponds to the block proposer + signed_summary = inclusion_list.summary + proposer_index = signed_summary.message.proposer_index + assert block.proposer_index == proposer_index + + # TODO: These checks will also be performed by the EL surely so we can probably remove them from here. + # Check the summary and transaction list lengths + summary = signed_summary.message.summary + assert len(summary) <= MAX_TRANSACTIONS_PER_INCLUSION_LIST + assert len(inclusion_list.transactions) == len(summary) + + # TODO: These checks will also be performed by the EL surely so we can probably remove them from here. + # Check that the total gas limit is bounded + total_gas_limit = sum( entry.gas_limit for entry in summary) + assert total_gas_limit <= MAX_GAS_PER_INCLUSION_LIST + + # Check that the signature is correct + # TODO: do we need a new domain? + signing_root = compute_signing_root(signed_summary.message, get_domain(state, DOMAIN_BEACON_PROPOSER)) + proposer = state.validators[proposer_index] + assert bls.Verify(proposer.pubkey, signing_root, signed_summary.signature) + + # Check that the inclusion list is valid + return execution_engine.notify_new_inclusion_list(inclusion_list) +``` + +### `is_inclusion_list_available` +*[New in ePBS]* + +```python +def is_inclusion_list_available(state: BeaconState, block: BeaconBlock) -> bool: + """ + Returns whether one inclusion list for the corresponding block was seen in full and has been validated. + There is one exception if the parent consensus block did not contain an exceution payload, in which case + We return true early + + `retrieve_inclusion_list` is implementation and context dependent + It returns one inclusion list that was broadcasted during the given slot by the given proposer. + Note: the p2p network does not guarantee sidecar retrieval outside of + `MIN_SLOTS_FOR_INCLUSION_LISTS_REQUESTS` + """ + # Verify that the list is empty if the parent consensus block did not contain a payload + if state.current_signed_execution_payload_header.message != state.latest_execution_payload_header: + return true + + # verify the inclusion list + inclusion_list = retrieve_inclusion_list(block.slot, block.proposer_index) + return verify_inclusion_list(state, block, inclusion_list, EXECUTION_ENGINE) +``` + + +## Updated fork-choice handlers + +### `on_block` + +*Note*: The handler `on_block` is modified to consider the pre `state` of the given consensus beacon block depending not only on the parent block root, but also on the parent blockhash. There is also the addition of the inclusion list availability check. + +```python +def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: + """ + Run ``on_block`` upon receiving a new block. + """ + block = signed_block.message + # Parent block must be known + assert block.parent_root in store.block_states + + # Check if this blocks builds on empty or full parent block + parent_block = store.blocks[block.parent_root] + parent_signed_payload_header = parent_block.body.signed_execution_payload_header + parent_payload_hash = paernt_signed_payload_header.message.block_hash + current_signed_payload_header = block.body.signed_execution_payload_header + current_payload_parent_hash = current_signed_payload_header.message.parent_hash + # Make a copy of the state to avoid mutability issues + if current_payload_parent_hash == parent_payload_hash: + assert block.parent_root in store.execution_payload_states + state = copy(store.execution_payload_states[block.parent_root]) + else: + state = copy(store.block_states[block.parent_root]) + + # Blocks cannot be in the future. If they are, their consideration must be delayed until they are in the past. + current_slot = get_current_slot(store) + assert current_slot >= block.slot + + # Check that block is later than the finalized epoch slot (optimization to reduce calls to get_ancestor) + finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) + assert block.slot > finalized_slot + # Check block is a descendant of the finalized block at the checkpoint finalized slot + finalized_checkpoint_block = get_checkpoint_block( + store, + block.parent_root, + store.finalized_checkpoint.epoch, + ) + assert store.finalized_checkpoint.root == finalized_checkpoint_block + + # Check if blob data is available + # If not, this block MAY be queued and subsequently considered when blob data becomes available + assert is_data_available(hash_tree_root(block), block.body.blob_kzg_commitments) + + # Check if there is a valid inclusion list. + # This check is performed only if the block's slot is within the visibility window + # If not, this block MAY be queued and subsequently considered when a valid inclusion list becomes available + if block.slot + MIN_SLOTS_FOR_INCLUSION_LISTS_REQUESTS >= current_slot: + assert is_inclusion_list_available(state, block) + + # Check the block is valid and compute the post-state + block_root = hash_tree_root(block) + state_transition(state, signed_block, True) + + # Add new block to the store + store.blocks[block_root] = block + # Add new state for this block to the store + store.block_states[block_root] = state + + # Add proposer score boost if the block is timely + time_into_slot = (store.time - store.genesis_time) % SECONDS_PER_SLOT + is_before_attesting_interval = time_into_slot < SECONDS_PER_SLOT // INTERVALS_PER_SLOT + if get_current_slot(store) == block.slot and is_before_attesting_interval: + store.proposer_boost_root = hash_tree_root(block) + + # Update checkpoints in store if necessary + update_checkpoints(store, state.current_justified_checkpoint, state.finalized_checkpoint) + + # Eagerly compute unrealized justification and finality. + compute_pulled_up_tip(store, block_root) +``` + +## New fork-choice handlers + +### `on_execution_payload` + +```python +def on_excecution_payload(store: Store, signed_envelope_: SignedExecutionPayloadEnvelope) -> None: + """ + Run ``on_execution_payload`` upon receiving a new execution payload. + """ + beacon_block_root = signed_envelope.beacon_block_root + # The corresponding beacon block root needs to be known + assert beacon_block_root in store.block_states + + # Make a copy of the state to avoid mutability issues + state = copy(store.block_states[beacon_block_root]) + + # Process the execution payload + process_execution_payload(state, signed_envelope, EXECUTION_ENGINE) + + #Add new state for this payload to the store + store.execution_payload_states[beacon_block_root] = state +``` + From 2a6400a2543b58f4a57c9b9e51ee538335fc1e83 Mon Sep 17 00:00:00 2001 From: Potuz Date: Fri, 25 Aug 2023 17:01:02 -0300 Subject: [PATCH 11/57] don't modify verify_block_signature --- specs/_features/epbs/beacon-chain.md | 13 ------------- specs/_features/epbs/fork-choice.md | 3 +-- 2 files changed, 1 insertion(+), 15 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index ebe75c1691..1b217afc4b 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -574,19 +574,6 @@ The post-state corresponding to a pre-state `state` and a signed block `signed_b The post-state corresponding to a pre-state `state` and a signed execution payload `signed_execution_payload` is defined as `process_execution_payload(state, signed_execution_payload)`. State transitions that trigger an unhandled exception (e.g. a failed `assert` or an out-of-range list access) are considered invalid. State transitions that cause a `uint64` overflow or underflow are also considered invalid. -### Modified `verify_block_signature` - -```python -def verify_block_signature(state: BeaconState, signed_block: SignedBeaconBlock) -> bool: - index = signed_block.message.proposer_index - if index < len(state.validators): - proposer = state.validators[index] - else: - proposer = state.builders[index-len(state.validators)] - signing_root = compute_signing_root(signed_block.message, get_domain(state, DOMAIN_BEACON_PROPOSER)) - return bls.Verify(proposer.pubkey, signing_root, signed_block.signature) -``` - ### Epoch processing #### Modified `process_epoch` diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 6b20ebfec9..91774a6b2a 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -169,5 +169,4 @@ def on_excecution_payload(store: Store, signed_envelope_: SignedExecutionPayload #Add new state for this payload to the store store.execution_payload_states[beacon_block_root] = state -``` - +``` From c852a4e49eb49ec3d2548d0d40f2c53391536062 Mon Sep 17 00:00:00 2001 From: Potuz Date: Fri, 25 Aug 2023 17:03:27 -0300 Subject: [PATCH 12/57] remove clutter --- specs/_features/epbs/beacon-chain.md | 287 +-------------------------- 1 file changed, 1 insertion(+), 286 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 1b217afc4b..802bc926c5 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -170,7 +170,7 @@ class BeaconBlockBody(Container): # Removed execution_payload [ Removed in ePBS] signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES] - blob_kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK] # [New in Deneb:EIP4844] + blob_kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK] ``` #### `ExecutionPayload` @@ -280,291 +280,6 @@ class BeaconState(Container): # PBS current_signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] ``` -## Helper functions - -### Predicates - -#### New `is_active_builder` - -```python -def is_active_builder(builder: Builder, epoch: Epoch) -> bool: - return epoch < builder.exit_epoch -``` - -#### New `is_slashable_builder` - -```python -def is_slashable_builder(builder: Builder, epoch: Epoch) -> bool: - """ - Check if ``builder`` is slashable. - """ - return (not validator.slashed) and (epoch < builder.withdrawable_epoch) -``` - -#### New `is_active_validator_at_index` - -```python -def is_active_validator_at_index(state: BeaconState, index: ValidatorIndex, epoch: Epoch) -> bool: - if index < len(state.validators): - return is_active_validator(state.validators[index], epoch) - return is_active_builder(state.builders[index-len(state.validators)], epoch) -``` - -#### Modified `is_valid_indexed_attestation` - -```python -def is_valid_indexed_attestation(state: BeaconState, indexed_attestation: IndexedAttestation) -> bool: - """ - Check if ``indexed_attestation`` is not empty, has sorted and unique indices and has a valid aggregate signature. - """ - # Verify indices are sorted and unique - indices = indexed_attestation.attesting_indices - if len(indices) == 0 or not indices == sorted(set(indices)): - return False - # Verify aggregate signature - pubkeys = [state.validators[i].pubkey if i < len(state.validators) else state.builders[i - len(state.validators)].pubkey for i in indices] - domain = get_domain(state, DOMAIN_BEACON_ATTESTER, indexed_attestation.data.target.epoch) - signing_root = compute_signing_root(indexed_attestation.data, domain) - return bls.FastAggregateVerify(pubkeys, signing_root, indexed_attestation.signature) -``` - - -### Misc - -#### Modified `compute_proposer_index` -*Note*: `compute_proposer_index` is modified to account for builders being validators - -TODO: actually do the sampling proportional to effective balance - -### Beacon state accessors - -#### Modified `get_active_validator_indices` - -```python -def get_active_validator_indices(state: BeaconState, epoch: Epoch) -> Sequence[ValidatorIndex]: - """ - Return the sequence of active validator indices at ``epoch``. - """ - builder_indices = [ValidatorIndex(len(state.validators) + i) for i,b in enumerate(state.builders) if is_active_builder(b,epoch)] - return [ValidatorIndex(i) for i, v in enumerate(state.validators) if is_active_validator(v, epoch)] + builder_indices -``` - -#### New `get_effective_balance` - -```python -def get_effective_balance(state: BeaconState, index: ValidatorIndex) -> Gwei: - """ - Return the effective balance for the validator or the builder indexed by ``index`` - """ - if index < len(state.validators): - return state.validators[index].effective_balance - return state.builders[index-len(state.validators)].effective_balance -``` - -#### Modified `get_total_balance` - -```python -def get_total_balance(state: BeaconState, indices: Set[ValidatorIndex]) -> Gwei: - """ - Return the combined effective balance of the ``indices``. - ``EFFECTIVE_BALANCE_INCREMENT`` Gwei minimum to avoid divisions by zero. - Math safe up to ~10B ETH, after which this overflows uint64. - """ - return Gwei(max(EFFECTIVE_BALANCE_INCREMENT, sum([get_effective_balance(state, index) for index in indices]))) -``` - -#### Modified `get_next_sync_committee_indices` - -*TODO*: make the shuffling actually weighted by the builder's effective balance - -```python -def get_next_sync_committee_indices(state: BeaconState) -> Sequence[ValidatorIndex]: - """ - Return the sync committee indices, with possible duplicates, for the next sync committee. - """ - epoch = Epoch(get_current_epoch(state) + 1) - - MAX_RANDOM_BYTE = 2**8 - 1 - active_validator_indices = get_active_validator_indices(state, epoch) - active_validator_count = uint64(len(active_validator_indices)) - seed = get_seed(state, epoch, DOMAIN_SYNC_COMMITTEE) - i = 0 - sync_committee_indices: List[ValidatorIndex] = [] - while len(sync_committee_indices) < SYNC_COMMITTEE_SIZE: - shuffled_index = compute_shuffled_index(uint64(i % active_validator_count), active_validator_count, seed) - candidate_index = active_validator_indices[shuffled_index] - random_byte = hash(seed + uint_to_bytes(uint64(i // 32)))[i % 32] - if candidate_index >= len(state.validators): - sync_commitee_indices.append(candidate_index) - else: - effective_balance = state.validators[candidate_index].effective_balance - if effective_balance * MAX_RANDOM_BYTE >= MAX_EFFECTIVE_BALANCE * random_byte: - sync_committee_indices.append(candidate_index) - i += 1 - return sync_committee_indices -``` - -#### Modified `get_next_sync_committee` - -```python -def get_next_sync_committee(state: BeaconState) -> SyncCommittee: - """ - Return the next sync committee, with possible pubkey duplicates. - """ - indices = get_next_sync_committee_indices(state) - pubkeys = [state.validators[index].pubkey if index < len(state.validators) else state.builders[index-len(state.validators)] for index in indices] - aggregate_pubkey = eth_aggregate_pubkeys(pubkeys) - return SyncCommittee(pubkeys=pubkeys, aggregate_pubkey=aggregate_pubkey) -``` - -#### Modified `get_unslashed_participating_indices` - -```python -def get_unslashed_participating_indices(state: BeaconState, flag_index: int, epoch: Epoch) -> Set[ValidatorIndex]: - """ - Return the set of validator indices that are both active and unslashed for the given ``flag_index`` and ``epoch``. - """ - assert epoch in (get_previous_epoch(state), get_current_epoch(state)) - if epoch == get_current_epoch(state): - epoch_participation = state.current_epoch_participation - epoch_builder_participation = state.current_epoch_builder_participation - else: - epoch_participation = state.previous_epoch_participation - epoch_builder_participation = state.previous_epoch_builder_participation - active_validator_indices = get_active_validator_indices(state, epoch) - participating_indices = [i for i in active_validator_indices if (has_flag(epoch_participation[i], flag_index) if i < len(state.validators) else has_flag(epoch_builder_participation[i-len(state.validators)], flag_index))] - return set(filter(lambda index: not state.validators[index].slashed if index < len(state.validators) else not state.builders[index-len(state.validators)].slashed, participating_indices)) -``` - -#### Modified `get_flag_index_deltas` - -```python -def get_flag_index_deltas(state: BeaconState, flag_index: int) -> Tuple[Sequence[Gwei], Sequence[Gwei]]: - """ - Return the deltas for a given ``flag_index`` by scanning through the participation flags. - """ - rewards = [Gwei(0)] * (len(state.validators) + len(state.builders)) - penalties = [Gwei(0)] * (len(state.validators) + len(state.builders) - previous_epoch = get_previous_epoch(state) - unslashed_participating_indices = get_unslashed_participating_indices(state, flag_index, previous_epoch) - weight = PARTICIPATION_FLAG_WEIGHTS[flag_index] - unslashed_participating_balance = get_total_balance(state, unslashed_participating_indices) - unslashed_participating_increments = unslashed_participating_balance // EFFECTIVE_BALANCE_INCREMENT - active_increments = get_total_active_balance(state) // EFFECTIVE_BALANCE_INCREMENT - for index in get_eligible_validator_indices(state): - base_reward = get_base_reward(state, index) - if index in unslashed_participating_indices: - if not is_in_inactivity_leak(state): - reward_numerator = base_reward * weight * unslashed_participating_increments - rewards[index] += Gwei(reward_numerator // (active_increments * WEIGHT_DENOMINATOR)) - elif flag_index != TIMELY_HEAD_FLAG_INDEX: - penalties[index] += Gwei(base_reward * weight // WEIGHT_DENOMINATOR) - return rewards, penalties -``` - - -### Beacon state mutators - -#### Modified `increase_balance` - -```python -def increase_balance(state: BeaconState, index: ValidatorIndex, delta: Gwei) -> None: - """ - Increase the validator balance at index ``index`` by ``delta``. - """ - if index < len(state.validators): - state.balances[index] += delta - return - state.builder_balances[index-len(state.validators)] += delta -``` - -#### Modified `decrease_balance` - -```python -def decrease_balance(state: BeaconState, index: ValidatorIndex, delta: Gwei) -> None: - """ - Decrease the validator balance at index ``index`` by ``delta``, with underflow protection. - """ - if index < len(state.validators) - state.balances[index] = 0 if delta > state.balances[index] else state.balances[index] - delta - return - index -= len(state.validators) - state.builder_balances[index] = 0 if delta > state.builder_balances[index] else state.builder_balances[index] - delta -``` - -#### Modified `initiate_validator_exit` - -```python -def initiate_validator_exit(state: BeaconState, index: ValidatorIndex) -> None: - """ - Initiate the exit of the validator with index ``index``. - """ - # Notice that the local variable ``validator`` may refer to a builder. Also that it continues defined outside - # its declaration scope. This is valid Python. - if index < len(state.validators): - validator = state.validators[index] - else: - validator = state.builders[index - len(state.validators)] - - # Return if validator already initiated exit - if validator.exit_epoch != FAR_FUTURE_EPOCH: - return - - # Compute exit queue epoch - exit_epochs = [v.exit_epoch for v in state.validators + state.builders if v.exit_epoch != FAR_FUTURE_EPOCH] - exit_queue_epoch = max(exit_epochs + [compute_activation_exit_epoch(get_current_epoch(state))]) - exit_queue_churn = len([v for v in state.validators + state.builders if v.exit_epoch == exit_queue_epoch]) - if exit_queue_churn >= get_validator_churn_limit(state): - exit_queue_epoch += Epoch(1) - - # Set validator exit epoch and withdrawable epoch - validator.exit_epoch = exit_queue_epoch - validator.withdrawable_epoch = Epoch(validator.exit_epoch + MIN_VALIDATOR_WITHDRAWABILITY_DELAY) # TODO: Do we want to differentiate builders here? -``` - -#### New `proposer_slashing_amount` - -```python -def proposer_slashing_amount(slashed_index: ValidatorIndex, effective_balance: Gwei): - return min(MAX_EFFECTIVE_BALANCE, effective_balance) // MIN_SLASHING_PENALTY_QUOTIENT -``` - -#### Modified `slash_validator` - -```python -def slash_validator(state: BeaconState, - slashed_index: ValidatorIndex, - proposer_slashing: bool, - whistleblower_index: ValidatorIndex=None) -> None: - """ - Slash the validator with index ``slashed_index``. - """ - epoch = get_current_epoch(state) - initiate_validator_exit(state, slashed_index) - # Notice that the local variable ``validator`` may refer to a builder. Also that it continues defined outside - # its declaration scope. This is valid Python. - if index < len(state.validators): - validator = state.validators[slashed_index] - else: - validator = state.builders[slashed_index - len(state.validators)] - validator.slashed = True - validator.withdrawable_epoch = max(validator.withdrawable_epoch, Epoch(epoch + EPOCHS_PER_SLASHINGS_VECTOR)) - state.slashings[epoch % EPOCHS_PER_SLASHINGS_VECTOR] += validator.effective_balance - if proposer_slashing: - decrease_balance(state, slashed_index, proposer_slashing_amount(slashed_index, validator.effective_balance)) - else: - decrease_balance(state, slashed_index, validator.effective_balance // MIN_SLASHING_PENALTY_QUOTIENT) - - # Apply proposer and whistleblower rewards - proposer_index = get_beacon_proposer_index(state) - if whistleblower_index is None: - whistleblower_index = proposer_index - whistleblower_reward = Gwei(max(MAX_EFFECTIVE_BALANCE, validator.effective_balance) // WHISTLEBLOWER_REWARD_QUOTIENT) - proposer_reward = Gwei(whistleblower_reward // PROPOSER_REWARD_QUOTIENT) - increase_balance(state, proposer_index, proposer_reward) - increase_balance(state, whistleblower_index, Gwei(whistleblower_reward - proposer_reward)) -``` - ## Beacon chain state transition function From 05022af389903bfe5a855100e4ec883b4b5abf6c Mon Sep 17 00:00:00 2001 From: Potuz Date: Sat, 26 Aug 2023 08:23:18 -0300 Subject: [PATCH 13/57] Check IL compatibility with parent block hash --- specs/_features/epbs/beacon-chain.md | 28 ++++++++++++++++++++++------ specs/_features/epbs/fork-choice.md | 18 +++++++++++------- 2 files changed, 33 insertions(+), 13 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 802bc926c5..cc93060a6a 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -47,7 +47,7 @@ For a further introduction please refer to this [ethresear.ch article](https://e | Name | Value | | - | - | -| `PTC_SIZE` | `uint64(2**9)` (=512) | +| `PTC_SIZE` | `uint64(2**9)` (=512) # (New in ePBS) | ### Domain types @@ -80,13 +80,13 @@ For a further introduction please refer to this [ethresear.ch article](https://e | Name | Value | | - | - | -| `PTC_PENALTY_WEIGHT` | `uint64(2)` | +| `PTC_PENALTY_WEIGHT` | `uint64(2)` # (New in ePBS)| ### Execution | Name | Value | | - | - | -| MAX_TRANSACTIONS_PER_INCLUSION_LIST | `2**4` (=16) | -| MAX_GAS_PER_INCLUSION_LIST | `2**21` (=2,097,152) | +| MAX_TRANSACTIONS_PER_INCLUSION_LIST | `2**4` (=16) # (New in ePBS) | +| MAX_GAS_PER_INCLUSION_LIST | `2**21` (=2,097,152) # (New in ePBS) | ## Containers @@ -130,6 +130,7 @@ class InclusionListSummaryEntry(Container): ```python class InclusionListSummary(Container) proposer_index: ValidatorIndex + parent_block_hash: Hash32 summary: List[InclusionListSummaryEntry, MAX_TRANSACTIONS_PER_INCLUSION_LIST] ``` @@ -281,6 +282,20 @@ class BeaconState(Container): current_signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] ``` +## Helper functions + +### Predicates + +#### `is_builder` + +```python +def is_builder(validator: Validator) -> bool: + """ + Check if `validator` is a registered builder + """ + return validator.withdrawal_credentials[0] == BUILDER_WITHDRAWAL_PREFIX +``` + ## Beacon chain state transition function *Note*: state transition is fundamentally modified in ePBS. The full state transition is broken in two parts, first importing a signed block and then importing an execution payload. @@ -331,8 +346,9 @@ def notify_new_inclusion_list(self: ExecutionEngine, inclusion_list_request: NewInclusionListRequest) -> bool: """ Return ``True`` if and only if the transactions in the inclusion list can be succesfully executed - starting from the current ``self.execution_state``, their total gas limit is less or equal that - ```MAX_GAS_PER_INCLUSION_LIST``, And the transactions in the list of transactions correspond to the signed summary + starting from the execution state corresponding to the `parent_block_hash` in the inclusion list + summary. The execution engine also checks that the total gas limit is less or equal that + ```MAX_GAS_PER_INCLUSION_LIST``, and the transactions in the list of transactions correspond to the signed summary """ ... ``` diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 91774a6b2a..50b44bd297 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -27,6 +27,16 @@ def verify_inclusion_list(state: BeaconState, block: BeaconBlock, inclusion_list proposer_index = signed_summary.message.proposer_index assert block.proposer_index == proposer_index + # Check that the signature is correct + # TODO: do we need a new domain? + signing_root = compute_signing_root(signed_summary.message, get_domain(state, DOMAIN_BEACON_PROPOSER)) + proposer = state.validators[proposer_index] + assert bls.Verify(proposer.pubkey, signing_root, signed_summary.signature) + + # Check that the parent_hash corresponds to the state's last execution payload header + parent_hash = signed_summary.message.parent_block_hash + assert parent_hash == state.latest_execution_payload_header.block_hash + # TODO: These checks will also be performed by the EL surely so we can probably remove them from here. # Check the summary and transaction list lengths summary = signed_summary.message.summary @@ -37,13 +47,7 @@ def verify_inclusion_list(state: BeaconState, block: BeaconBlock, inclusion_list # Check that the total gas limit is bounded total_gas_limit = sum( entry.gas_limit for entry in summary) assert total_gas_limit <= MAX_GAS_PER_INCLUSION_LIST - - # Check that the signature is correct - # TODO: do we need a new domain? - signing_root = compute_signing_root(signed_summary.message, get_domain(state, DOMAIN_BEACON_PROPOSER)) - proposer = state.validators[proposer_index] - assert bls.Verify(proposer.pubkey, signing_root, signed_summary.signature) - + # Check that the inclusion list is valid return execution_engine.notify_new_inclusion_list(inclusion_list) ``` From 59048f9699189fa54b9e1483a73aa06d9f8621c4 Mon Sep 17 00:00:00 2001 From: Potuz Date: Sat, 26 Aug 2023 08:27:49 -0300 Subject: [PATCH 14/57] Do not broadcast the parent_block_hash but use it as parameter to the EL --- specs/_features/epbs/beacon-chain.md | 2 +- specs/_features/epbs/fork-choice.md | 7 ++----- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index cc93060a6a..3e65363b32 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -130,7 +130,6 @@ class InclusionListSummaryEntry(Container): ```python class InclusionListSummary(Container) proposer_index: ValidatorIndex - parent_block_hash: Hash32 summary: List[InclusionListSummaryEntry, MAX_TRANSACTIONS_PER_INCLUSION_LIST] ``` @@ -335,6 +334,7 @@ def process_epoch(state: BeaconState) -> None: @dataclass class NewInclusionListRequest(object): inclusion_list: InclusionList + parent_block_hash: Hash32 ``` #### Engine APIs diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 50b44bd297..7762f209b0 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -33,10 +33,6 @@ def verify_inclusion_list(state: BeaconState, block: BeaconBlock, inclusion_list proposer = state.validators[proposer_index] assert bls.Verify(proposer.pubkey, signing_root, signed_summary.signature) - # Check that the parent_hash corresponds to the state's last execution payload header - parent_hash = signed_summary.message.parent_block_hash - assert parent_hash == state.latest_execution_payload_header.block_hash - # TODO: These checks will also be performed by the EL surely so we can probably remove them from here. # Check the summary and transaction list lengths summary = signed_summary.message.summary @@ -49,7 +45,8 @@ def verify_inclusion_list(state: BeaconState, block: BeaconBlock, inclusion_list assert total_gas_limit <= MAX_GAS_PER_INCLUSION_LIST # Check that the inclusion list is valid - return execution_engine.notify_new_inclusion_list(inclusion_list) + return execution_engine.notify_new_inclusion_list(NewInclusionListRequest( + inclusion_list=inclusion_list, parent_block_hash = state.latest_execution_payload_header.block_hash)) ``` ### `is_inclusion_list_available` From 188d09d94a0860599b949cb5a4c5a5d904584327 Mon Sep 17 00:00:00 2001 From: Potuz Date: Sat, 26 Aug 2023 09:15:55 -0300 Subject: [PATCH 15/57] process payment immediately --- specs/_features/epbs/beacon-chain.md | 8 +++++--- specs/_features/epbs/design.md | 7 ++++++- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 3e65363b32..c72b19a03b 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -384,11 +384,13 @@ def verify_execution_payload_header_signature(state: BeaconState, signed_header: ```python def process_execution_payload_header(state: BeaconState, signed_header: SignedExecutionPayloadHeader) -> None: assert verify_execution_payload_header_signature(state, signed_header) - # Check that the builder has funds to cover the bid + # Check that the builder has funds to cover the bid and transfer the funds header = signed_header.message builder_index = header.builder_index - if state.balances[builder_index] < header.value: - return false + amount = header.value + assert state.balances[builder_index] >= amount: + decrease_balance(state, builder_index, amount) + increase_balance(state, proposer_index, amount) # Verify consistency of the parent hash with respect to the previous execution payload header assert header.parent_hash == state.latest_execution_payload_header.block_hash # Verify prev_randao diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index cd76fc8431..6d9cb5fb79 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -16,8 +16,13 @@ ePBS introduces forward inclusion lists for proposers to guarantee censor resist ## Builders -There is a new entity `Builder` that is a glorified validator required to have a higher stake and required to sign when producing execution payloads. +There is a new entity `Builder` that is a glorified validator (they are simply validators with a different withdrawal prefix `0x0b`) required to have a higher stake and required to sign when producing execution payloads. - Builders are also validators (otherwise their staked capital depreciates). - We onboard builders by simply turning validators into builders if they achieve the necessary minimum balance (this way we avoid two forks to onboard builders and keep the same deposit flow, avoid builders to skip the entry churn), we change their withdrawal prefix to be distinguished from normal validators. - We need to include several changes from the [MaxEB PR](https://github.com/michaelneuder/consensus-specs/pull/3) in order to account with builders having an increased balance that would otherwise depreciate. + +## Builder Payments + +Payments are processed unconditionally when processing the signed execution payload header. There are cases to study for possible same-slot unbundling even by an equivocation. Same slot unbundling can happen if the proposer equivocates, and propagates his equivocation after seeing the reveal of the builder which happens at 8 seconds. The next proposer has to build on full which can only happen by being dishonest. Honest validators will vote for the previous block not letting the attack succeed. The honest builder does not lose his bid as the block is reorged. + From ea645a1a3158c59e384af49f193ea1de1467fa01 Mon Sep 17 00:00:00 2001 From: Potuz Date: Sat, 26 Aug 2023 11:10:51 -0300 Subject: [PATCH 16/57] Deal with withdrawals --- specs/_features/epbs/beacon-chain.md | 101 +++++++++++++++++++++++++-- specs/_features/epbs/design.md | 13 ++++ 2 files changed, 110 insertions(+), 4 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index c72b19a03b..3753eed2be 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -171,6 +171,7 @@ class BeaconBlockBody(Container): signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES] blob_kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK] + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] # [New in ePBS] ``` #### `ExecutionPayload` @@ -328,6 +329,15 @@ def process_epoch(state: BeaconState) -> None: #### Request data +##### New `NewWithdrawalsRequest` + +```python +@dataclass +class NewWithdrawalsRequest(object): + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] + parent_block_hash: Hash32 +``` + ##### New `NewInclusionListRequest` ```python @@ -339,6 +349,19 @@ class NewInclusionListRequest(object): #### Engine APIs +#### New `notify_withdrawals` + +TODO: Can we send this with FCU as parameters instead of a new engine method reorg resistant? We need to remove withdrawals from the payload attributes now. + +```python +def notify_withdrawals(self: ExecutionEngine, withdrawals: NewWithdrawalsRequest) -> None + """ + This call informs the EL that the next payload which is a grandchild of the current ``parent_block_hash`` + needs to include the listed withdrawals that have been already fulfilled in the CL + """ + ... +``` + #### New `notify_new_inclusion_list` ```python @@ -355,20 +378,92 @@ def notify_new_inclusion_list(self: ExecutionEngine, ### Block processing -*Note*: the function `process_block` is modified to only process the consensus block. The full state-transition process is broken into separate functions, one to process a `BeaconBlock` and another to process a `SignedExecutionPayload`. +*Note*: the function `process_block` is modified to only process the consensus block. The full state-transition process is broken into separate functions, one to process a `BeaconBlock` and another to process a `SignedExecutionPayload`. Notice that withdrawals are now included in the beacon block, they are processed before the execution payload header as this header may affect validator balances. ```python def process_block(state: BeaconState, block: BeaconBlock) -> None: process_block_header(state, block) + process_withdrawals(state, block.body) [Modified in ePBS] process_execution_payload_header(state, block.body.execution_payload_header) # [Modified in ePBS] - # Removed process_withdrawal in ePBS is processed during payload processing [Modified in ePBS] process_randao(state, block.body) process_eth1_data(state, block.body) process_operations(state, block.body) # [Modified in ePBS] process_sync_aggregate(state, block.body.sync_aggregate) ``` +#### Modified `get_expected_withdrawals` +**Note:** the function `get_expected_withdrawals` is modified to return no withdrawals if the parent block was empty. +TODO: Still need to include the MaxEB changes + +```python +def get_expected_withdrawals(state: BeaconState) -> Sequence[Withdrawal]: + ## return early if the parent block was empty + withdrawals: List[Withdrawal] = [] + if state.current_signed_execution_payload_header.message != state.latest_execution_payload_header: + return withdrawals + epoch = get_current_epoch(state) + withdrawal_index = state.next_withdrawal_index + validator_index = state.next_withdrawal_validator_index + bound = min(len(state.validators), MAX_VALIDATORS_PER_WITHDRAWALS_SWEEP) + for _ in range(bound): + validator = state.validators[validator_index] + balance = state.balances[validator_index] + if is_fully_withdrawable_validator(validator, balance, epoch): + withdrawals.append(Withdrawal( + index=withdrawal_index, + validator_index=validator_index, + address=ExecutionAddress(validator.withdrawal_credentials[12:]), + amount=balance, + )) + withdrawal_index += WithdrawalIndex(1) + elif is_partially_withdrawable_validator(validator, balance): + withdrawals.append(Withdrawal( + index=withdrawal_index, + validator_index=validator_index, + address=ExecutionAddress(validator.withdrawal_credentials[12:]), + amount=balance - MAX_EFFECTIVE_BALANCE, + )) + withdrawal_index += WithdrawalIndex(1) + if len(withdrawals) == MAX_WITHDRAWALS_PER_PAYLOAD: + break + validator_index = ValidatorIndex((validator_index + 1) % len(state.validators)) + return withdrawals +``` + +#### Modified `process_withdrawals` +**Note:** TODO: This is modified to take a `BeaconBlockBody`. Still need to include the MaxEB changes + +```python +def process_withdrawals(state: BeaconState, body: BeaconBlockBody) -> None: + withdrawals = body.withdrawals + expected_withdrawals = get_expected_withdrawals(state) + assert len(withdrawals) == len(expected_withdrawals) + + for expected_withdrawal, withdrawal in zip(expected_withdrawals, withdrawals): + assert withdrawal == expected_withdrawal + decrease_balance(state, withdrawal.validator_index, withdrawal.amount) + + # Update the next withdrawal index if this block contained withdrawals + if len(expected_withdrawals) != 0: + latest_withdrawal = expected_withdrawals[-1] + state.next_withdrawal_index = WithdrawalIndex(latest_withdrawal.index + 1) + + # Update the next validator index to start the next withdrawal sweep + if len(expected_withdrawals) == MAX_WITHDRAWALS_PER_PAYLOAD: + # Next sweep starts after the latest withdrawal's validator index + next_validator_index = ValidatorIndex((expected_withdrawals[-1].validator_index + 1) % len(state.validators)) + state.next_withdrawal_validator_index = next_validator_index + else: + # Advance sweep by the max length of the sweep if there was not a full set of withdrawals + next_index = state.next_withdrawal_validator_index + MAX_VALIDATORS_PER_WITHDRAWALS_SWEEP + next_validator_index = ValidatorIndex(next_index % len(state.validators)) + state.next_withdrawal_validator_index = next_validator_index + # Inform the EL of the processed withdrawals + hash = body.signed_execution_payload_header.message.parent_block_hash + execution_engine.notify_withdrawals(NewWithdrawalsRequest(withdrawals=withdrawals, parent_block_hash = hash)) +``` + #### New `verify_execution_payload_header_signature` ```python @@ -431,8 +526,6 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti parent_beacon_block_root=state.latest_block_header.parent_root, ) ) - # Process Withdrawals in the payload - process_withdrawals(state, payload) # Cache the execution payload header state.latest_execution_payload_header = state.current_signed_execution_payload_header.message # Verify the state root diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 6d9cb5fb79..71f420360e 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -26,3 +26,16 @@ There is a new entity `Builder` that is a glorified validator (they are simply v Payments are processed unconditionally when processing the signed execution payload header. There are cases to study for possible same-slot unbundling even by an equivocation. Same slot unbundling can happen if the proposer equivocates, and propagates his equivocation after seeing the reveal of the builder which happens at 8 seconds. The next proposer has to build on full which can only happen by being dishonest. Honest validators will vote for the previous block not letting the attack succeed. The honest builder does not lose his bid as the block is reorged. +## Withdrawals + +Withdrawals cannot be fulfilled at the same time in the CL and the EL now: if they are included in the Execution payload for slot N, the consensus block for the same slot may have invalidated the validator balances. The new mechanism is as follows +- Proposer for block N includes the withdrawals in his CL block and they are immediately processed in the CL +- The execution payload for N mints ETH in the EL for the already burnt ETH in the CL during slot N-1. +- If the slot N-1 was empty, the proposer for N does not advance withdrawals and does not include any new withdrawals, so that each EL block mints a maximum of `MAX_WITHDRAWALS_PER_PAYLOAD` withdrawals. +- There is a new engine endpoint that notifies the EL of the next withdrawals that need to be included. The EL needs to verify the validity that the execution payload includes those withdrawals in the next blockhash. + +### Examples + +(N-2, Full) -- (N-1: Full, withdrawals 0--15) -- (N: Empty, withdrawals 16--31) -- (N+1, does not send withdrawals, Full: excecutes withdrawals 0--15 since it is a child of the blockhash from N-1, thus grandchild of the blockhash of N-2) -- (N+2, withdrawals 32--47, Full: excecutes withdrawals 16--31) + +(N-2, Full) - (N-1: Full withdrawals 0--15) -- (N: Empty, withdrawals 16--31) -- (N+1: Empty, no withdrawals) -- (N+2: no withdrawals, Full: executes withdrawals 0--15). From 92ffab64408b546d222634a60dc403d4e0d69e75 Mon Sep 17 00:00:00 2001 From: Potuz Date: Sat, 26 Aug 2023 13:41:39 -0300 Subject: [PATCH 17/57] fix withdrawals, check only in the CL --- specs/_features/epbs/beacon-chain.md | 85 ++++++---------------------- specs/_features/epbs/design.md | 12 +--- 2 files changed, 19 insertions(+), 78 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 3753eed2be..537fd97b6a 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -171,7 +171,6 @@ class BeaconBlockBody(Container): signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES] blob_kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK] - withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] # [New in ePBS] ``` #### `ExecutionPayload` @@ -280,6 +279,7 @@ class BeaconState(Container): historical_summaries: List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT] # PBS current_signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] + last_withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] # [New in ePBS] ``` ## Helper functions @@ -349,19 +349,6 @@ class NewInclusionListRequest(object): #### Engine APIs -#### New `notify_withdrawals` - -TODO: Can we send this with FCU as parameters instead of a new engine method reorg resistant? We need to remove withdrawals from the payload attributes now. - -```python -def notify_withdrawals(self: ExecutionEngine, withdrawals: NewWithdrawalsRequest) -> None - """ - This call informs the EL that the next payload which is a grandchild of the current ``parent_block_hash`` - needs to include the listed withdrawals that have been already fulfilled in the CL - """ - ... -``` - #### New `notify_new_inclusion_list` ```python @@ -384,7 +371,7 @@ def notify_new_inclusion_list(self: ExecutionEngine, ```python def process_block(state: BeaconState, block: BeaconBlock) -> None: process_block_header(state, block) - process_withdrawals(state, block.body) [Modified in ePBS] + process_withdrawals(state) [Modified in ePBS] process_execution_payload_header(state, block.body.execution_payload_header) # [Modified in ePBS] process_randao(state, block.body) process_eth1_data(state, block.body) @@ -392,76 +379,34 @@ def process_block(state: BeaconState, block: BeaconBlock) -> None: process_sync_aggregate(state, block.body.sync_aggregate) ``` -#### Modified `get_expected_withdrawals` -**Note:** the function `get_expected_withdrawals` is modified to return no withdrawals if the parent block was empty. -TODO: Still need to include the MaxEB changes +#### Modified `process_withdrawals` +**Note:** TODO: This is modified to take only the State as parameter as they are deterministic. Still need to include the MaxEB changes ```python -def get_expected_withdrawals(state: BeaconState) -> Sequence[Withdrawal]: +def process_withdrawals(state: BeaconState) -> None: ## return early if the parent block was empty - withdrawals: List[Withdrawal] = [] if state.current_signed_execution_payload_header.message != state.latest_execution_payload_header: - return withdrawals - epoch = get_current_epoch(state) - withdrawal_index = state.next_withdrawal_index - validator_index = state.next_withdrawal_validator_index - bound = min(len(state.validators), MAX_VALIDATORS_PER_WITHDRAWALS_SWEEP) - for _ in range(bound): - validator = state.validators[validator_index] - balance = state.balances[validator_index] - if is_fully_withdrawable_validator(validator, balance, epoch): - withdrawals.append(Withdrawal( - index=withdrawal_index, - validator_index=validator_index, - address=ExecutionAddress(validator.withdrawal_credentials[12:]), - amount=balance, - )) - withdrawal_index += WithdrawalIndex(1) - elif is_partially_withdrawable_validator(validator, balance): - withdrawals.append(Withdrawal( - index=withdrawal_index, - validator_index=validator_index, - address=ExecutionAddress(validator.withdrawal_credentials[12:]), - amount=balance - MAX_EFFECTIVE_BALANCE, - )) - withdrawal_index += WithdrawalIndex(1) - if len(withdrawals) == MAX_WITHDRAWALS_PER_PAYLOAD: - break - validator_index = ValidatorIndex((validator_index + 1) % len(state.validators)) - return withdrawals -``` - -#### Modified `process_withdrawals` -**Note:** TODO: This is modified to take a `BeaconBlockBody`. Still need to include the MaxEB changes - -```python -def process_withdrawals(state: BeaconState, body: BeaconBlockBody) -> None: - withdrawals = body.withdrawals - expected_withdrawals = get_expected_withdrawals(state) - assert len(withdrawals) == len(expected_withdrawals) - - for expected_withdrawal, withdrawal in zip(expected_withdrawals, withdrawals): - assert withdrawal == expected_withdrawal + return + withdrawals = get_expected_withdrawals(state) + state.last_withdrawals = withdrawals + for withdrawal in withdrawals: decrease_balance(state, withdrawal.validator_index, withdrawal.amount) # Update the next withdrawal index if this block contained withdrawals - if len(expected_withdrawals) != 0: - latest_withdrawal = expected_withdrawals[-1] + if len(withdrawals) != 0: + latest_withdrawal = withdrawals[-1] state.next_withdrawal_index = WithdrawalIndex(latest_withdrawal.index + 1) # Update the next validator index to start the next withdrawal sweep - if len(expected_withdrawals) == MAX_WITHDRAWALS_PER_PAYLOAD: + if len(withdrawals) == MAX_WITHDRAWALS_PER_PAYLOAD: # Next sweep starts after the latest withdrawal's validator index - next_validator_index = ValidatorIndex((expected_withdrawals[-1].validator_index + 1) % len(state.validators)) + next_validator_index = ValidatorIndex((withdrawals[-1].validator_index + 1) % len(state.validators)) state.next_withdrawal_validator_index = next_validator_index else: # Advance sweep by the max length of the sweep if there was not a full set of withdrawals next_index = state.next_withdrawal_validator_index + MAX_VALIDATORS_PER_WITHDRAWALS_SWEEP next_validator_index = ValidatorIndex(next_index % len(state.validators)) state.next_withdrawal_validator_index = next_validator_index - # Inform the EL of the processed withdrawals - hash = body.signed_execution_payload_header.message.parent_block_hash - execution_engine.notify_withdrawals(NewWithdrawalsRequest(withdrawals=withdrawals, parent_block_hash = hash)) ``` #### New `verify_execution_payload_header_signature` @@ -517,6 +462,10 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti hash = hash_tree_root(payload) previous_hash = hash_tree_root(state.current_signed_execution_payload_header.message) assert hash == previous_hash + # Verify the withdrawals + assert len(state.last_withrawals) == len(payload.withdrawals) + for withdrawal, payload_withdrawal in zip(state.last_withdrawals, payload.withdrawals): + assert withdrawal == payload_withdrawal # Verify the execution payload is valid versioned_hashes = [kzg_commitment_to_versioned_hash(commitment) for commitment in body.blob_kzg_commitments] assert execution_engine.verify_and_notify_new_payload( diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 71f420360e..25cb62179c 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -28,14 +28,6 @@ Payments are processed unconditionally when processing the signed execution payl ## Withdrawals -Withdrawals cannot be fulfilled at the same time in the CL and the EL now: if they are included in the Execution payload for slot N, the consensus block for the same slot may have invalidated the validator balances. The new mechanism is as follows -- Proposer for block N includes the withdrawals in his CL block and they are immediately processed in the CL -- The execution payload for N mints ETH in the EL for the already burnt ETH in the CL during slot N-1. -- If the slot N-1 was empty, the proposer for N does not advance withdrawals and does not include any new withdrawals, so that each EL block mints a maximum of `MAX_WITHDRAWALS_PER_PAYLOAD` withdrawals. -- There is a new engine endpoint that notifies the EL of the next withdrawals that need to be included. The EL needs to verify the validity that the execution payload includes those withdrawals in the next blockhash. +Withdrawals are deterministic on the beacon state, so on a consensus layer block processing, they are immediately processed, then later when the payload appears we verify that the withdrawals in the payload agree with the already fulfilled withdrawals in the CL. -### Examples - -(N-2, Full) -- (N-1: Full, withdrawals 0--15) -- (N: Empty, withdrawals 16--31) -- (N+1, does not send withdrawals, Full: excecutes withdrawals 0--15 since it is a child of the blockhash from N-1, thus grandchild of the blockhash of N-2) -- (N+2, withdrawals 32--47, Full: excecutes withdrawals 16--31) - -(N-2, Full) - (N-1: Full withdrawals 0--15) -- (N: Empty, withdrawals 16--31) -- (N+1: Empty, no withdrawals) -- (N+2: no withdrawals, Full: executes withdrawals 0--15). +So when importing the CL block for slot N, we process the expected withdrawals at that slot. We save the list of paid withdrawals to the beacon state. When the payload for slot N appears, we check that the withdrawals correspond to the saved withdrawals. If the payload does not appear, the saved withdrawals remain, so any future payload has to include those. From 25fb473c0770f65051d23e4c350ba1fe6e242d74 Mon Sep 17 00:00:00 2001 From: Potuz Date: Sun, 27 Aug 2023 08:15:49 -0300 Subject: [PATCH 18/57] remove unused request dataclass --- specs/_features/epbs/beacon-chain.md | 9 --------- 1 file changed, 9 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 537fd97b6a..ac9d6436ee 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -329,15 +329,6 @@ def process_epoch(state: BeaconState) -> None: #### Request data -##### New `NewWithdrawalsRequest` - -```python -@dataclass -class NewWithdrawalsRequest(object): - withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] - parent_block_hash: Hash32 -``` - ##### New `NewInclusionListRequest` ```python From 51847e25e2278eb6985019989e5ae5fda4abf2d8 Mon Sep 17 00:00:00 2001 From: Potuz Date: Sun, 27 Aug 2023 08:24:06 -0300 Subject: [PATCH 19/57] pass the propsoer_index to process_execution_payload_header --- specs/_features/epbs/beacon-chain.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index ac9d6436ee..49a9e0e49b 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -363,7 +363,7 @@ def notify_new_inclusion_list(self: ExecutionEngine, def process_block(state: BeaconState, block: BeaconBlock) -> None: process_block_header(state, block) process_withdrawals(state) [Modified in ePBS] - process_execution_payload_header(state, block.body.execution_payload_header) # [Modified in ePBS] + process_execution_payload_header(state, block) # [Modified in ePBS] process_randao(state, block.body) process_eth1_data(state, block.body) process_operations(state, block.body) # [Modified in ePBS] @@ -413,7 +413,8 @@ def verify_execution_payload_header_signature(state: BeaconState, signed_header: #### New `process_execution_payload_header` ```python -def process_execution_payload_header(state: BeaconState, signed_header: SignedExecutionPayloadHeader) -> None: +def process_execution_payload_header(state: BeaconState, block: BeaconBlock) -> None: + signed_header = block.body.signed_execution_payload_header assert verify_execution_payload_header_signature(state, signed_header) # Check that the builder has funds to cover the bid and transfer the funds header = signed_header.message @@ -421,7 +422,7 @@ def process_execution_payload_header(state: BeaconState, signed_header: SignedEx amount = header.value assert state.balances[builder_index] >= amount: decrease_balance(state, builder_index, amount) - increase_balance(state, proposer_index, amount) + increase_balance(state, block.proposer_index, amount) # Verify consistency of the parent hash with respect to the previous execution payload header assert header.parent_hash == state.latest_execution_payload_header.block_hash # Verify prev_randao From 95e4eb2313949cc20d9396a3262dbadf12dbb9f9 Mon Sep 17 00:00:00 2001 From: Potuz Date: Sun, 27 Aug 2023 08:29:36 -0300 Subject: [PATCH 20/57] run doctoc --- specs/_features/epbs/beacon-chain.md | 45 ++++++++++++++++++++++++++++ specs/_features/epbs/fork-choice.md | 9 ++++++ 2 files changed, 54 insertions(+) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 49a9e0e49b..7b45aeea40 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -6,6 +6,51 @@ +- [Introduction](#introduction) +- [Constants](#constants) + - [Withdrawal prefixes](#withdrawal-prefixes) +- [Configuration](#configuration) + - [Time parameters](#time-parameters) +- [Preset](#preset) + - [Misc](#misc) + - [Domain types](#domain-types) + - [Gwei values](#gwei-values) + - [Time parameters](#time-parameters-1) + - [Rewards and penalties](#rewards-and-penalties) + - [Incentivization weights](#incentivization-weights) + - [Execution](#execution) +- [Containers](#containers) + - [New containers](#new-containers) + - [`SignedExecutionPayloadHeader`](#signedexecutionpayloadheader) + - [`ExecutionPayloadEnvelope`](#executionpayloadenvelope) + - [`SignedExecutionPayloadEnvelope`](#signedexecutionpayloadenvelope) + - [`InclusionListSummaryEntry`](#inclusionlistsummaryentry) + - [`InclusionListSummary`](#inclusionlistsummary) + - [`SignedInclusionListSummary`](#signedinclusionlistsummary) + - [`InclusionList`](#inclusionlist) + - [Modified containers](#modified-containers) + - [`BeaconBlockBody`](#beaconblockbody) + - [`ExecutionPayload`](#executionpayload) + - [`ExecutionPayloadHeader`](#executionpayloadheader) + - [`BeaconState`](#beaconstate) +- [Helper functions](#helper-functions) + - [Predicates](#predicates) + - [`is_builder`](#is_builder) +- [Beacon chain state transition function](#beacon-chain-state-transition-function) + - [Epoch processing](#epoch-processing) + - [Modified `process_epoch`](#modified-process_epoch) + - [Execution engine](#execution-engine) + - [Request data](#request-data) + - [New `NewInclusionListRequest`](#new-newinclusionlistrequest) + - [Engine APIs](#engine-apis) + - [New `notify_new_inclusion_list`](#new-notify_new_inclusion_list) + - [Block processing](#block-processing) + - [Modified `process_withdrawals`](#modified-process_withdrawals) + - [New `verify_execution_payload_header_signature`](#new-verify_execution_payload_header_signature) + - [New `process_execution_payload_header`](#new-process_execution_payload_header) + - [New `verify_execution_payload_signature`](#new-verify_execution_payload_signature) + - [Modified `process_execution_payload`](#modified-process_execution_payload) + diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 7762f209b0..7532476e12 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -5,6 +5,15 @@ +- [Introduction](#introduction) +- [Helpers](#helpers) + - [`verify_inclusion_list`](#verify_inclusion_list) + - [`is_inclusion_list_available`](#is_inclusion_list_available) +- [Updated fork-choice handlers](#updated-fork-choice-handlers) + - [`on_block`](#on_block) +- [New fork-choice handlers](#new-fork-choice-handlers) + - [`on_execution_payload`](#on_execution_payload) + From cbe061815025847b700b3a9a21ab428fd44dcc42 Mon Sep 17 00:00:00 2001 From: Potuz Date: Sun, 27 Aug 2023 15:13:16 -0300 Subject: [PATCH 21/57] only cache the withdrawals root in the state --- specs/_features/epbs/beacon-chain.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 7b45aeea40..a20874d41e 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -222,6 +222,7 @@ class BeaconBlockBody(Container): **Note:** The `ExecutionPayload` is modified to contain the builder's index and the bid value. It also contains a transaction inclusion list summary signed by the corresponding beacon block proposer and the list of indices of transactions in the parent block that have to be excluded from the inclusion list summary because they were satisfied in the previous slot. +TODO: `builder_index` and `value` do not need to be in the payload sent to the engine, but they need to be in the header committed to the state. Either we move them out here to the envelope and we add them to back when comparing with the committed header, or we keep as here and we will be sending 16 extra bytes to the EL that are ignored. ```python class ExecutionPayload(Container): # Execution block header fields @@ -324,7 +325,7 @@ class BeaconState(Container): historical_summaries: List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT] # PBS current_signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] - last_withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] # [New in ePBS] + last_withdrawals_root: Root # [New in ePBS] ``` ## Helper functions @@ -424,7 +425,7 @@ def process_withdrawals(state: BeaconState) -> None: if state.current_signed_execution_payload_header.message != state.latest_execution_payload_header: return withdrawals = get_expected_withdrawals(state) - state.last_withdrawals = withdrawals + state.last_withdrawals_root = hash_tree_root(withdrawals) for withdrawal in withdrawals: decrease_balance(state, withdrawal.validator_index, withdrawal.amount) @@ -468,6 +469,8 @@ def process_execution_payload_header(state: BeaconState, block: BeaconBlock) -> assert state.balances[builder_index] >= amount: decrease_balance(state, builder_index, amount) increase_balance(state, block.proposer_index, amount) + # Verify the withdrawals_root against the state cached ones + assert header.withdrawals_root == state.last_withdrawals_root # Verify consistency of the parent hash with respect to the previous execution payload header assert header.parent_hash == state.latest_execution_payload_header.block_hash # Verify prev_randao @@ -499,10 +502,6 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti hash = hash_tree_root(payload) previous_hash = hash_tree_root(state.current_signed_execution_payload_header.message) assert hash == previous_hash - # Verify the withdrawals - assert len(state.last_withrawals) == len(payload.withdrawals) - for withdrawal, payload_withdrawal in zip(state.last_withdrawals, payload.withdrawals): - assert withdrawal == payload_withdrawal # Verify the execution payload is valid versioned_hashes = [kzg_commitment_to_versioned_hash(commitment) for commitment in body.blob_kzg_commitments] assert execution_engine.verify_and_notify_new_payload( From ef11fb4e4fe19e4909b3e8c1a058b81aac7c75d6 Mon Sep 17 00:00:00 2001 From: Potuz Date: Sun, 27 Aug 2023 21:23:18 -0300 Subject: [PATCH 22/57] ptc rewards pass 1 --- specs/_features/epbs/beacon-chain.md | 207 ++++++++++++++++++++++++++- specs/_features/epbs/fork-choice.md | 19 ++- 2 files changed, 223 insertions(+), 3 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index a20874d41e..dd5d400ed8 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -98,7 +98,8 @@ For a further introduction please refer to this [ethresear.ch article](https://e | Name | Value | | - | - | -| `DOMAIN_BEACON_BUILDER` | `DomainType('0x0B000000')` # (New in ePBS)| +| `DOMAIN_BEACON_BUILDER` | `DomainType('0x1B000000')` # (New in ePBS)| +| `DOMAIN_PTC_ATTESTER` | `DomainType('0x0C000000')` # (New in ePBS)| ### Gwei values @@ -137,6 +138,41 @@ For a further introduction please refer to this [ethresear.ch article](https://e ### New containers +#### `PayloadAttestationData` + +```python +class PayloadAttestationData(Container): + beacon_block_root: Root + payload_present: bool +``` + +#### `PayloadAttestation` + +```python +class PayloadAttestation(Container): + aggregation_bits: BitVector[PTC_SIZE] + data: PayloadAttestationData + signature: BLSSignature +``` + +#### `PayloadAttestationMessage` + +```python +class PayloadAttestationMessage(Container): + validator_index: ValidatorIndex + data: PayloadAttestationData + signature: BLSSignature +``` + +#### `IndexedPayloadAttestation` + +```python +class IndexedPayloadAttestation(Container): + attesting_indices: List[ValidatorIndex, PTC_SIZE] + data: PayloadAttestationData + signature: BLSSignature +``` + #### `SignedExecutionPayloadHeader` ```python @@ -342,6 +378,68 @@ def is_builder(validator: Validator) -> bool: return validator.withdrawal_credentials[0] == BUILDER_WITHDRAWAL_PREFIX ``` +#### `is_valid_indexed_payload_attestation` + +```python +def is_valid_indexed_payload_attestation(state: BeaconState, indexed_payload_attestation: IndexedPayloadAttestation) -> bool: + """ + Check if ``indexed_payload_attestation`` is not empty, has sorted and unique indices and has a valid aggregate signature. + """ + # Verify indices are sorted and unique + indices = indexed_payload_attestation.attesting_indices + if len(indices) == 0 or not indices == sorted(set(indices)): + return False + # Verify aggregate signature + pubkeys = [state.validators[i].pubkey for i in indices] + domain = get_domain(state, DOMAIN_PTC_ATTESTER, None) + signing_root = compute_signing_root(indexed_payload_attestation.data, domain) + return bls.FastAggregateVerify(pubkeys, signing_root, indexed_payload_attestation.signature) +``` + +### Beacon State accessors + +#### `get_ptc` + +```python +def get_ptc(state: BeaconState, slot: Slot) -> Vector[ValidatorIndex, PTC_SIZE]: + """ + Get the ptc committee for the give ``slot`` + """ + beacon_committee = get_beacon_committee(state, slot, 0)[:PTC_SIZE] + validator_indices = [idx for idx in beacon_committee if not is_builder(idx)] + return validator_indices[:PTC_SIZE] +``` + +#### `get_payload_attesting_indices` + +```python +def get_payload_attesting_indices(state: BeaconState, + slot: Slot, payload_attestation: PayloadAttestation) -> Set[ValidatorIndex]: + """ + Return the set of attesting indices corresponding to ``payload_attestation``. + """ + ptc = get_ptc(state, slot) + return set(index for i, index in enumerate(ptc) if payload_attestation.aggregation_bits[i]) +``` + + +#### `get_indexed_payload_attestation` + +```python +def get_indexed_payload_attestation(state: BeaconState, + slot: Slot, payload_attestation: PayloadAttestation) -> IndexedPayloadAttestation: + """ + Return the indexed payload attestation corresponding to ``payload_attestation``. + """ + attesting_indices = get_payload_attesting_indices(state, slot, payload_attestation) + + return IndexedPayloadAttestation( + attesting_indices=sorted(attesting_indices), + data=payload_attestation.data, + signature=payload_attestation.signature, + ) +``` + ## Beacon chain state transition function *Note*: state transition is fundamentally modified in ePBS. The full state transition is broken in two parts, first importing a signed block and then importing an execution payload. @@ -415,6 +513,111 @@ def process_block(state: BeaconState, block: BeaconBlock) -> None: process_operations(state, block.body) # [Modified in ePBS] process_sync_aggregate(state, block.body.sync_aggregate) ``` +#### Modified `process_operations` + +**Note:** `process_operations` is modified to process PTC attestations + +```python +def process_operations(state: BeaconState, body: BeaconBlockBody) -> None: + # Verify that outstanding deposits are processed up to the maximum number of deposits + assert len(body.deposits) == min(MAX_DEPOSITS, state.eth1_data.deposit_count - state.eth1_deposit_index) + + def for_ops(operations: Sequence[Any], fn: Callable[[BeaconState, Any], None]) -> None: + for operation in operations: + fn(state, operation) + + for_ops(body.proposer_slashings, process_proposer_slashing) + for_ops(body.attester_slashings, process_attester_slashing) + for_ops(body.attestations, process_attestation) + for_ops(body.deposits, process_deposit) + for_ops(body.voluntary_exits, process_voluntary_exit) + for_ops(body.bls_to_execution_changes, process_bls_to_execution_change) + for_ops(body.payload_attestations, process_payload_attestation) # [New in ePBS] +``` + +#### Modified `process_attestation` + +*Note*: The function `process_attestation` is modified to ignore attestations from the ptc + +```python +def process_attestation(state: BeaconState, attestation: Attestation) -> None: + data = attestation.data + assert data.target.epoch in (get_previous_epoch(state), get_current_epoch(state)) + assert data.target.epoch == compute_epoch_at_slot(data.slot) + assert data.slot + MIN_ATTESTATION_INCLUSION_DELAY <= state.slot <= data.slot + SLOTS_PER_EPOCH + assert data.index < get_committee_count_per_slot(state, data.target.epoch) + + committee = get_beacon_committee(state, data.slot, data.index) + assert len(attestation.aggregation_bits) == len(committee) + + # Participation flag indices + participation_flag_indices = get_attestation_participation_flag_indices(state, data, state.slot - data.slot) + + # Verify signature + assert is_valid_indexed_attestation(state, get_indexed_attestation(state, attestation)) + + # Update epoch participation flags + if data.target.epoch == get_current_epoch(state): + epoch_participation = state.current_epoch_participation + else: + epoch_participation = state.previous_epoch_participation + + ptc = get_ptc(state, data.slot) + attesting_indices = [i for i in get_attesting_indices(state, data, attestation.aggregation_bits) if i not in ptc] + proposer_reward_numerator = 0 + for index in attesting_indices + for flag_index, weight in enumerate(PARTICIPATION_FLAG_WEIGHTS): + if flag_index in participation_flag_indices and not has_flag(epoch_participation[index], flag_index): + epoch_participation[index] = add_flag(epoch_participation[index], flag_index) + proposer_reward_numerator += get_base_reward(state, index) * weight + + # Reward proposer + proposer_reward_denominator = (WEIGHT_DENOMINATOR - PROPOSER_WEIGHT) * WEIGHT_DENOMINATOR // PROPOSER_WEIGHT + proposer_reward = Gwei(proposer_reward_numerator // proposer_reward_denominator) + increase_balance(state, get_beacon_proposer_index(state), proposer_reward) +``` + + +##### Payload Attestations + +```python +def process_payload_attestation(state: BeaconState, payload_attestation: PayloadAttestation) -> None: + ## Check that the attestation is for the parent beacon block + data = payload_attestation.data + assert data.beacon_block_root == state.latest_block_header.parent_root + ## Check that the attestation is for the previous slot + assert state.slot > 0 + assert data.beacon_block_root == state.block_roots[(state.slot - 1) % SLOTS_PER_HISTORICAL_ROOT] + + #Verify signature + indexed_payload_attestation = get_indexed_payload_attestation(state, state.slot - 1, payload_attestation) + assert is_valid_indexed_payload_attestation(state, indexed_payload_attestation) + + ptc = get_ptc(state, state.slot - 1) + if slot % SLOTS_PER_EPOCH == 0: + epoch_participation = state.previous_epoch_participation + else: + epoch_participation = state.current_epoch_participation + + # Return early if the attestation is for the wrong payload status + latest_payload_timestamp = state.latest_execution_payload_header.timestamp + present_timestamp = compute_timestamp_at_slot(state, state.slot - 1) + payload_was_present = latest_payload_timestamp == present_timestamp + if data.payload_present != payload_was_present: + return + # Reward the proposer and set all the participation flags + proposer_reward_numerator = 0 + for index in indexed_payload_attestation.attesting_indices: + for flag_index, weight in enumerate(PARTICIPATION_FLAG_WEIGHTS): + if not has_flag(epoch_participation[index], flag_index): + epoch_participation[index] = add_flag(epoch_participation[index], flag_index) + proposer_reward_numerator += get_base_reward(state, index) * weight + + # Reward proposer + proposer_reward_denominator = (WEIGHT_DENOMINATOR - PROPOSER_WEIGHT) * WEIGHT_DENOMINATOR // PROPOSER_WEIGHT + proposer_reward = Gwei(proposer_reward_numerator // proposer_reward_denominator) + increase_balance(state, get_beacon_proposer_index(state), proposer_reward) +``` #### Modified `process_withdrawals` **Note:** TODO: This is modified to take only the State as parameter as they are deterministic. Still need to include the MaxEB changes @@ -422,7 +625,7 @@ def process_block(state: BeaconState, block: BeaconBlock) -> None: ```python def process_withdrawals(state: BeaconState) -> None: ## return early if the parent block was empty - if state.current_signed_execution_payload_header.message != state.latest_execution_payload_header: + state.current_signed_execution_payload_header.message != state.latest_execution_payload_header: return withdrawals = get_expected_withdrawals(state) state.last_withdrawals_root = hash_tree_root(withdrawals) diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 7532476e12..58d8705fae 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -50,7 +50,7 @@ def verify_inclusion_list(state: BeaconState, block: BeaconBlock, inclusion_list # TODO: These checks will also be performed by the EL surely so we can probably remove them from here. # Check that the total gas limit is bounded - total_gas_limit = sum( entry.gas_limit for entry in summary) + total_gas_limit = sum( entry.gas_limit for entry in summary ) assert total_gas_limit <= MAX_GAS_PER_INCLUSION_LIST # Check that the inclusion list is valid @@ -82,6 +82,13 @@ def is_inclusion_list_available(state: BeaconState, block: BeaconBlock) -> bool: return verify_inclusion_list(state, block, inclusion_list, EXECUTION_ENGINE) ``` +### `validate_on_payload_attestation` + +```python +def validate_ptc_from_block(store: Store, payload_attestation: PayloadAttestation) -> None: + # The beacon block root must be known + assert payload_attestation.data.beacon_block_root in store.blocks +``` ## Updated fork-choice handlers @@ -180,3 +187,13 @@ def on_excecution_payload(store: Store, signed_envelope_: SignedExecutionPayload #Add new state for this payload to the store store.execution_payload_states[beacon_block_root] = state ``` + +### `on_payload_attestation` + +```python +def on_payload_attestation(store: Store, ptc_attestation: PayloadAttestation) -> None + """ + Run ``on_payload_attestation`` upon receiving a new ``payload_attestation`` from either within a ``BeaconBlock`` + or directly on the wire. + """ + # From 7014513e1efbedca3c625307f10855eb8d290bc2 Mon Sep 17 00:00:00 2001 From: Potuz Date: Mon, 28 Aug 2023 12:40:56 -0300 Subject: [PATCH 23/57] only pass relevant IL info to the EL --- specs/_features/epbs/beacon-chain.md | 3 ++- specs/_features/epbs/fork-choice.md | 4 +++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index dd5d400ed8..18b85ddeab 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -478,7 +478,8 @@ def process_epoch(state: BeaconState) -> None: ```python @dataclass class NewInclusionListRequest(object): - inclusion_list: InclusionList + inclusion_list: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] + summary: List[InclusionListSummaryEntry, MAX_TRANSACTIONS_PER_INCLUSION_LIST] parent_block_hash: Hash32 ``` diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 58d8705fae..d064446b9a 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -55,7 +55,9 @@ def verify_inclusion_list(state: BeaconState, block: BeaconBlock, inclusion_list # Check that the inclusion list is valid return execution_engine.notify_new_inclusion_list(NewInclusionListRequest( - inclusion_list=inclusion_list, parent_block_hash = state.latest_execution_payload_header.block_hash)) + inclusion_list=inclusion_list.transactions, + summary=inclusion_list.summary.message.summary, + parent_block_hash = state.latest_execution_payload_header.block_hash)) ``` ### `is_inclusion_list_available` From cbee247f8fd0515b028b01f419138dfc6655699a Mon Sep 17 00:00:00 2001 From: Potuz Date: Mon, 28 Aug 2023 16:47:32 -0300 Subject: [PATCH 24/57] Use envelopes instead of adding builder info to the payload Move the builder id and other unnecessary information outside of the payload. --- specs/_features/epbs/beacon-chain.md | 87 ++++++++++++++++++---------- specs/_features/epbs/design.md | 5 ++ specs/_features/epbs/fork-choice.md | 50 +++++++++++----- 3 files changed, 98 insertions(+), 44 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 18b85ddeab..9744798219 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -180,13 +180,31 @@ class SignedExecutionPayloadHeader(Container): message: ExecutionPayloadHeader signature: BLSSignature ``` +#### `ExecutionPayloadHeaderEnvelope` +```python +class ExecutionPayloadHeaderEnvelope(Container): + header: ExecutionPayloadHeader + builder_index: ValidatorIndex + value: Gwei +``` + +#### `SignedExecutionPayloadHeaderEnvelope` + +```python +class SignedExecutionPayloadHeaderEnvelope(Container): + message: ExecutionPayloadHeaderEnvelope + signature: BLSSignature +``` + #### `ExecutionPayloadEnvelope` ```python class ExecutionPayloadEnvelope(Container): payload: ExecutionPayload + builder_index: ValidatorIndex beacon_block_root: Root + blob_kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK] state_root: Root ``` @@ -248,17 +266,16 @@ class BeaconBlockBody(Container): voluntary_exits: List[SignedVoluntaryExit, MAX_VOLUNTARY_EXITS] sync_aggregate: SyncAggregate # Execution - # Removed execution_payload [ Removed in ePBS] - signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] + # Removed execution_payload [Removed in ePBS] + # Removed blob_kzg_commitments [Removed in ePBS] + signed_execution_payload_header_envelope: SignedExecutionPayloadHeaderEnvelope # [New in ePBS] bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES] - blob_kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK] ``` #### `ExecutionPayload` -**Note:** The `ExecutionPayload` is modified to contain the builder's index and the bid value. It also contains a transaction inclusion list summary signed by the corresponding beacon block proposer and the list of indices of transactions in the parent block that have to be excluded from the inclusion list summary because they were satisfied in the previous slot. +**Note:** The `ExecutionPayload` is modified to contain a transaction inclusion list summary signed by the corresponding beacon block proposer and the list of indices of transactions in the parent block that have to be excluded from the inclusion list summary because they were satisfied in the previous slot. -TODO: `builder_index` and `value` do not need to be in the payload sent to the engine, but they need to be in the header committed to the state. Either we move them out here to the envelope and we add them to back when comparing with the committed header, or we keep as here and we will be sending 16 extra bytes to the EL that are ignored. ```python class ExecutionPayload(Container): # Execution block header fields @@ -278,15 +295,15 @@ class ExecutionPayload(Container): block_hash: Hash32 # Hash of execution block transactions: List[Transaction, MAX_TRANSACTIONS_PER_PAYLOAD] withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] - builder_index: ValidatorIndex # [New in ePBS] - value: Gwei # [New in ePBS] + blob_gas_used: uint64 + excess_blob_gas: uint64 inclusion_list_summary: SignedInclusionListSummary # [New in ePBS] inclusion_list_exclusions: List[uint64, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] ``` #### `ExecutionPayloadHeader` -**Note:** The `ExecutionPayloadHeader` is modified to include the builder's index and the bid's value. +**Note:** The `ExecutionPayloadHeader` is modified to account for the transactions inclusion lists. ```python class ExecutionPayloadHeader(Container): @@ -307,8 +324,8 @@ class ExecutionPayloadHeader(Container): block_hash: Hash32 # Hash of execution block transactions_root: Root withdrawals_root: Root - builder_index: ValidatorIndex # [New in ePBS] - value: Gwei # [New in ePBS] + blob_gas_used: uint64 + excess_blob_gas: uint64 inclusion_list_summary_root: Root # [New in ePBS] inclusion_list_exclusions_root: Root # [New in ePBS] ``` @@ -360,7 +377,7 @@ class BeaconState(Container): # Deep history valid from Capella onwards historical_summaries: List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT] # PBS - current_signed_execution_payload_header: SignedExecutionPayloadHeader # [New in ePBS] + signed_execution_payload_header_envelope: SignedExecutionPayloadHeaderEnvelop # [New in ePBS] last_withdrawals_root: Root # [New in ePBS] ``` @@ -508,12 +525,13 @@ def notify_new_inclusion_list(self: ExecutionEngine, def process_block(state: BeaconState, block: BeaconBlock) -> None: process_block_header(state, block) process_withdrawals(state) [Modified in ePBS] - process_execution_payload_header(state, block) # [Modified in ePBS] + process_execution_payload_header(state, block) # [Modified in ePBS, removed process_execution_payload] process_randao(state, block.body) process_eth1_data(state, block.body) process_operations(state, block.body) # [Modified in ePBS] process_sync_aggregate(state, block.body.sync_aggregate) ``` + #### Modified `process_operations` **Note:** `process_operations` is modified to process PTC attestations @@ -626,7 +644,7 @@ def process_payload_attestation(state: BeaconState, payload_attestation: Payload ```python def process_withdrawals(state: BeaconState) -> None: ## return early if the parent block was empty - state.current_signed_execution_payload_header.message != state.latest_execution_payload_header: + state.signed_execution_payload_header_envelope.message.header != state.latest_execution_payload_header: return withdrawals = get_expected_withdrawals(state) state.last_withdrawals_root = hash_tree_root(withdrawals) @@ -650,26 +668,27 @@ def process_withdrawals(state: BeaconState) -> None: state.next_withdrawal_validator_index = next_validator_index ``` -#### New `verify_execution_payload_header_signature` +#### New `verify_execution_payload_header_envelope_signature` ```python -def verify_execution_payload_header_signature(state: BeaconState, signed_header: SignedExecutionPayloadHeader) -> bool: +def verify_execution_payload_header_envelope_signature(state: BeaconState, + signed_header_envelope: SignedExecutionPayloadHeaderEnvelope) -> bool: # Check the signature - builder = state.validators[signed_header.message.builder_index] - signing_root = compute_signing_root(signed_header.message, get_domain(state, DOMAIN_BEACON_BUILDER)) - return bls.Verify(builder.pubkey, signing_root, signed_header.signature) + builder = state.validators[signed_header_envelope.message.builder_index] + signing_root = compute_signing_root(signed_header_envelope.message, get_domain(state, DOMAIN_BEACON_BUILDER)) + return bls.Verify(builder.pubkey, signing_root, signed_header_envelope.signature) ``` #### New `process_execution_payload_header` ```python def process_execution_payload_header(state: BeaconState, block: BeaconBlock) -> None: - signed_header = block.body.signed_execution_payload_header - assert verify_execution_payload_header_signature(state, signed_header) + signed_header_envelope = block.body.signed_execution_payload_header_envelope + assert verify_execution_payload_header_envelope_signature(state, signed_header_envelope) # Check that the builder has funds to cover the bid and transfer the funds - header = signed_header.message - builder_index = header.builder_index - amount = header.value + envelope = signed_header_envelope.message + builder_index = envelope.builder_index + amount = envelope.value assert state.balances[builder_index] >= amount: decrease_balance(state, builder_index, amount) increase_balance(state, block.proposer_index, amount) @@ -681,15 +700,15 @@ def process_execution_payload_header(state: BeaconState, block: BeaconBlock) -> assert header.prev_randao == get_randao_mix(state, get_current_epoch(state)) # Verify timestamp assert header.timestamp == compute_timestamp_at_slot(state, state.slot) - # Cache execution payload header - state.current_signed_execution_payload_header = signed_header + # Cache execution payload header envelope + state.signed_execution_payload_header_envelope = signed_header_envelope ``` #### New `verify_execution_payload_signature` ```python def verify_execution_envelope_signature(state: BeaconState, signed_envelope: SignedExecutionPayloadEnvelope) -> bool: - builder = state.validators[signed_envelope.message.payload.builder_index] + builder = state.validators[signed_envelope.message.builder_index] signing_root = compute_signing_root(signed_envelope.message, get_domain(state, DOMAIN_BEACON_BUILDER)) return bls.Verify(builder.pubkey, signing_root, signed_envelope.signature) ``` @@ -701,13 +720,19 @@ def verify_execution_envelope_signature(state: BeaconState, signed_envelope: Sig def process_execution_payload(state: BeaconState, signed_envelope: SignedExecutionPayloadEnvelope, execution_engine: ExecutionEngine) -> None: # Verify signature [New in ePBS] assert verify_execution_envelope_signature(state, signed_envelope) - payload = signed_envelope.message.payload + envelope = signed_envelope.message + payload = envelope.payload + # Verify consistency with the beacon block + assert envelope.beacon_block_root == hash_tree_root(state.latest_block_header) # Verify consistency with the committed header hash = hash_tree_root(payload) - previous_hash = hash_tree_root(state.current_signed_execution_payload_header.message) + commited_envelope = state.signed_execution_payload_header_envelope.message + previous_hash = hash_tree_root(committed_envelope.payload) assert hash == previous_hash + # Verify consistency with the envelope + assert envelope.builder_index == committed_envelope.builder_index # Verify the execution payload is valid - versioned_hashes = [kzg_commitment_to_versioned_hash(commitment) for commitment in body.blob_kzg_commitments] + versioned_hashes = [kzg_commitment_to_versioned_hash(commitment) for commitment in envelope.blob_kzg_commitments] assert execution_engine.verify_and_notify_new_payload( NewPayloadRequest( execution_payload=payload, @@ -716,7 +741,7 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti ) ) # Cache the execution payload header - state.latest_execution_payload_header = state.current_signed_execution_payload_header.message + state.latest_execution_payload_header = committed_envelope.payload # Verify the state root - assert signed_envelope.message.state_root == hash_tree_root(state) + assert envelope.state_root == hash_tree_root(state) ``` diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 25cb62179c..34c9b5f7f9 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -31,3 +31,8 @@ Payments are processed unconditionally when processing the signed execution payl Withdrawals are deterministic on the beacon state, so on a consensus layer block processing, they are immediately processed, then later when the payload appears we verify that the withdrawals in the payload agree with the already fulfilled withdrawals in the CL. So when importing the CL block for slot N, we process the expected withdrawals at that slot. We save the list of paid withdrawals to the beacon state. When the payload for slot N appears, we check that the withdrawals correspond to the saved withdrawals. If the payload does not appear, the saved withdrawals remain, so any future payload has to include those. + +## Blobs + +- KZG Commitments are now sent on the Execution Payload envelope broadcasted by the EL and the EL block can only be valid if the data is available. +- Blobs themselves may be broadcasted by the builder below as soon as it sees the beacon block if he sees it's safe. diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index d064446b9a..b665c927a6 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -23,6 +23,28 @@ This is the modification of the fork choice accompanying the ePBS upgrade. ## Helpers +### Modified `Store` +**Note:** `Store` is modified to track the intermediate states of "empty" consensus blocks, that is, those consensus blocks for which the corresponding execution payload has not been revealed or has not been included on chain. + +```python +@dataclass +class Store(object): + time: uint64 + genesis_time: uint64 + justified_checkpoint: Checkpoint + finalized_checkpoint: Checkpoint + unrealized_justified_checkpoint: Checkpoint + unrealized_finalized_checkpoint: Checkpoint + proposer_boost_root: Root + equivocating_indices: Set[ValidatorIndex] + blocks: Dict[Root, BeaconBlock] = field(default_factory=dict) + block_states: Dict[Root, BeaconState] = field(default_factory=dict) + checkpoint_states: Dict[Checkpoint, BeaconState] = field(default_factory=dict) + latest_messages: Dict[ValidatorIndex, LatestMessage] = field(default_factory=dict) + unrealized_justifications: Dict[Root, Checkpoint] = field(default_factory=dict) + execution_payload_states: Dict[Root, BeaconState] = field(default_factory=dict) # [New in ePBS] +``` + ### `verify_inclusion_list` *[New in ePBS]* @@ -76,7 +98,7 @@ def is_inclusion_list_available(state: BeaconState, block: BeaconBlock) -> bool: `MIN_SLOTS_FOR_INCLUSION_LISTS_REQUESTS` """ # Verify that the list is empty if the parent consensus block did not contain a payload - if state.current_signed_execution_payload_header.message != state.latest_execution_payload_header: + if state.signed_execution_payload_header_envelope.message.header != state.latest_execution_payload_header: return true # verify the inclusion list @@ -98,6 +120,8 @@ def validate_ptc_from_block(store: Store, payload_attestation: PayloadAttestatio *Note*: The handler `on_block` is modified to consider the pre `state` of the given consensus beacon block depending not only on the parent block root, but also on the parent blockhash. There is also the addition of the inclusion list availability check. +In addition we delay the checking of blob availability until the processing of the execution payload. + ```python def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: """ @@ -109,10 +133,10 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: # Check if this blocks builds on empty or full parent block parent_block = store.blocks[block.parent_root] - parent_signed_payload_header = parent_block.body.signed_execution_payload_header - parent_payload_hash = paernt_signed_payload_header.message.block_hash - current_signed_payload_header = block.body.signed_execution_payload_header - current_payload_parent_hash = current_signed_payload_header.message.parent_hash + parent_signed_payload_header_envelope = parent_block.body.signed_execution_payload_header_envelope + parent_payload_hash = parent_signed_payload_header_envelope.message.header.block_hash + current_signed_payload_header_envelope = block.body.signed_execution_payload_header_envelope + current_payload_parent_hash = current_signed_payload_header_envelope.message.header.parent_hash # Make a copy of the state to avoid mutability issues if current_payload_parent_hash == parent_payload_hash: assert block.parent_root in store.execution_payload_states @@ -135,10 +159,6 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: ) assert store.finalized_checkpoint.root == finalized_checkpoint_block - # Check if blob data is available - # If not, this block MAY be queued and subsequently considered when blob data becomes available - assert is_data_available(hash_tree_root(block), block.body.blob_kzg_commitments) - # Check if there is a valid inclusion list. # This check is performed only if the block's slot is within the visibility window # If not, this block MAY be queued and subsequently considered when a valid inclusion list becomes available @@ -172,16 +192,20 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: ### `on_execution_payload` ```python -def on_excecution_payload(store: Store, signed_envelope_: SignedExecutionPayloadEnvelope) -> None: +def on_excecution_payload(store: Store, signed_envelope: SignedExecutionPayloadEnvelope) -> None: """ Run ``on_execution_payload`` upon receiving a new execution payload. """ - beacon_block_root = signed_envelope.beacon_block_root + envelope = signed_envelope.message # The corresponding beacon block root needs to be known - assert beacon_block_root in store.block_states + assert envelope.beacon_block_root in store.block_states + + # Check if blob data is available + # If not, this payload MAY be queued and subsequently considered when blob data becomes available + assert is_data_available(envelope.beacon_block_root, envelope.blob_kzg_commitments) # Make a copy of the state to avoid mutability issues - state = copy(store.block_states[beacon_block_root]) + state = copy(store.block_states[envelope.beacon_block_root]) # Process the execution payload process_execution_payload(state, signed_envelope, EXECUTION_ENGINE) From c48d050154e1237ad0691f03b72a10258def6421 Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 29 Aug 2023 10:23:40 -0300 Subject: [PATCH 25/57] ptc message handlers in forkchoice --- specs/_features/epbs/beacon-chain.md | 10 +++++- specs/_features/epbs/fork-choice.md | 49 +++++++++++++++++++++++----- 2 files changed, 49 insertions(+), 10 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 9744798219..7956350ef6 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -122,6 +122,12 @@ For a further introduction please refer to this [ethresear.ch article](https://e | - | - | | `PROPOSER_EQUIVOCATION_PENALTY_FACTOR` | `uint64(2**2)` (= 4) # (New in ePBS)| +### Max operations per block + +| Name | Value | +| - | - | +| `MAX_PAYLOAD_ATTESTATIONS` | `2**1` (= 2) # (New in ePBS) | + ### Incentivization weights | Name | Value | @@ -268,8 +274,10 @@ class BeaconBlockBody(Container): # Execution # Removed execution_payload [Removed in ePBS] # Removed blob_kzg_commitments [Removed in ePBS] - signed_execution_payload_header_envelope: SignedExecutionPayloadHeaderEnvelope # [New in ePBS] bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES] + # PBS + signed_execution_payload_header_envelope: SignedExecutionPayloadHeaderEnvelope # [New in ePBS] + payload_attestations: List[PayloadAttestation, MAX_PAYLOAD_ATTESTATIONS] # [New in ePBS] ``` #### `ExecutionPayload` diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index b665c927a6..df466314da 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -106,12 +106,18 @@ def is_inclusion_list_available(state: BeaconState, block: BeaconBlock) -> bool: return verify_inclusion_list(state, block, inclusion_list, EXECUTION_ENGINE) ``` -### `validate_on_payload_attestation` +### `notify_ptc_messages` ```python -def validate_ptc_from_block(store: Store, payload_attestation: PayloadAttestation) -> None: - # The beacon block root must be known - assert payload_attestation.data.beacon_block_root in store.blocks +def notify_ptc_messages(store: Store, state: BeaconState, payload_attestations: Sequence[PayloadAttestation]) -> None: + """ + Extracts a list of ``PayloadAttestationMessage`` from ``payload_attestations`` and updates the store with them + """ + for payload_attestation in payload_attestations: + indexed_payload_attestation = get_indexed_payload_attestation(state, state.slot - 1, payload_attestation) + for idx in indexed_payload_attestation.attesting_indices: + store.on_payload_attestation_message(PayloadAttestationMessage(validator_index=idx, + data=payload_attestation.data, signature: BLSSignature(), is_from_block=true) ``` ## Updated fork-choice handlers @@ -174,6 +180,9 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: # Add new state for this block to the store store.block_states[block_root] = state + # Notify the store about the payload_attestations in the block + store.notify_ptc_messages(state, block.body.payload_attestations) + # Add proposer score boost if the block is timely time_into_slot = (store.time - store.genesis_time) % SECONDS_PER_SLOT is_before_attesting_interval = time_into_slot < SECONDS_PER_SLOT // INTERVALS_PER_SLOT @@ -214,12 +223,34 @@ def on_excecution_payload(store: Store, signed_envelope: SignedExecutionPayloadE store.execution_payload_states[beacon_block_root] = state ``` -### `on_payload_attestation` +### `on_payload_attestation_message` ```python -def on_payload_attestation(store: Store, ptc_attestation: PayloadAttestation) -> None +def on_payload_attestation_message(store: Store, + ptc_message: PayloadAttestationMessage, is_from_block: bool=False) -> None: """ - Run ``on_payload_attestation`` upon receiving a new ``payload_attestation`` from either within a ``BeaconBlock`` - or directly on the wire. + Run ``on_payload_attestation_message`` upon receiving a new ``ptc_message`` directly on the wire. """ - # + # The beacon block root must be known + data = ptc_message.data + # PTC attestation must be for a known block. If block is unknown, delay consideration until the block is found + state = store.block_states[data.beacon_block_root] + ptc = get_ptc(state, state.slot) + + # Verify the signature and check that its for the current slot if it is coming from the wire + if not is_from_block: + # Check that the attestation is for the current slot + assert state.slot == get_current_slot(store) + # Check that the attester is from the current ptc + assert ptc_message.validator_index in ptc + # Verify the signature + assert is_valid_indexed_payload_attestation(state, + IndexedPayloadAttestation(attesting_indices = [ptc_message.validator_index], data = data, + signature = ptc_message signature)) + # Update the ptc vote for the block + # TODO: Do we want to slash ptc members that equivocate? + # we are updating here the message and so the last vote will be the one that counts. + ptc_index = ptc.index(ptc_message.validator_index) + ptc_vote = store.ptc_vote[data.beacon_block_root] + ptc_vote[ptc_index] = data.present +``` From e81721a48a70e29cde1273379f3642d74741b6fb Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 29 Aug 2023 13:58:18 -0300 Subject: [PATCH 26/57] take ptc members from all committees --- specs/_features/epbs/beacon-chain.md | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 7956350ef6..42d07e0f1b 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -391,6 +391,20 @@ class BeaconState(Container): ## Helper functions +### Math + +#### `bit_floor` + +```python +def bit_floor(n: uint64) -> uint64: + """ + if ``n`` is not zero, returns the largest power of `2` that is not greater than `n`. + """ + if n == 0: + return 0 + return uint64(1) << (n.bit_length() - 1) +``` + ### Predicates #### `is_builder` @@ -430,9 +444,16 @@ def get_ptc(state: BeaconState, slot: Slot) -> Vector[ValidatorIndex, PTC_SIZE]: """ Get the ptc committee for the give ``slot`` """ - beacon_committee = get_beacon_committee(state, slot, 0)[:PTC_SIZE] - validator_indices = [idx for idx in beacon_committee if not is_builder(idx)] - return validator_indices[:PTC_SIZE] + epoch = compute_epoch_at_slot(slot) + committees_per_slot = bit_floor(max(get_committee_count_per_slot(state, epoch), PTC_SIZE)) + members_per_committee = PTC_SIZE/committees_per_slot + + validator_indices = [] + for idx in range(committees_per_slot) + beacon_committee = get_beacon_committee(state, slot, idx) + vals = [idx for idx in beacon_committee if not is_builder(idx)] + validator_indices += vals[:members_per_commitee] + return validator_indices ``` #### `get_payload_attesting_indices` From cd7c0070bf6cd73c7b6c1f90b4800bd91bfc1aff Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 29 Aug 2023 14:25:11 -0300 Subject: [PATCH 27/57] add forkchoice helper to get committee voted --- specs/_features/epbs/fork-choice.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index df466314da..66fe228f0e 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -21,6 +21,12 @@ This is the modification of the fork choice accompanying the ePBS upgrade. +## Constant + +| Name | Value | +| -------------------- | ----------- | +| `PAYLOAD_TIMELY_THRESHOLD` | `PTC_SIZE/2` (=`uint64(256)`) | + ## Helpers ### Modified `Store` @@ -43,6 +49,7 @@ class Store(object): latest_messages: Dict[ValidatorIndex, LatestMessage] = field(default_factory=dict) unrealized_justifications: Dict[Root, Checkpoint] = field(default_factory=dict) execution_payload_states: Dict[Root, BeaconState] = field(default_factory=dict) # [New in ePBS] + ptc_vote: Dict[Root, Vector[bool, PTC_SIZE]] = field(default_factory=dict) # [New in ePBS] ``` ### `verify_inclusion_list` @@ -120,6 +127,19 @@ def notify_ptc_messages(store: Store, state: BeaconState, payload_attestations: data=payload_attestation.data, signature: BLSSignature(), is_from_block=true) ``` +### `is_payload_present` + +```python +def is_payload_present(store: Store, beacon_block_root: Root) -> bool: + """ + return wether the execution payload for the beacon block with root ``beacon_block_root`` was voted as present + by the PTC + """ + # The beacon block root must be known + assert beacon_block_root in store.ptc_vote + return ptc_vote[beacon_block_root].count(True) > PAYLOAD_TIMELY_THRESHOLD +``` + ## Updated fork-choice handlers ### `on_block` @@ -179,6 +199,8 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: store.blocks[block_root] = block # Add new state for this block to the store store.block_states[block_root] = state + # Add a new PTC voting for this block to the store + store.ptc_vote[block_root] = [False]*PTC_SIZE # Notify the store about the payload_attestations in the block store.notify_ptc_messages(state, block.body.payload_attestations) From e317ed825136944b2d9b2a38c43ab6c33a9a8245 Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 29 Aug 2023 14:59:12 -0300 Subject: [PATCH 28/57] add rewards and ptc attestations to the design doc --- specs/_features/epbs/design.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 34c9b5f7f9..fb1c0beb7a 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -36,3 +36,14 @@ So when importing the CL block for slot N, we process the expected withdrawals a - KZG Commitments are now sent on the Execution Payload envelope broadcasted by the EL and the EL block can only be valid if the data is available. - Blobs themselves may be broadcasted by the builder below as soon as it sees the beacon block if he sees it's safe. + +## PTC Rewards +- PTC members are obtained as the first members from each beacon slot committee that are not builders. +- attesters are rewarded as a full attestation when they get the right payload presence: that is, if they vote for full (resp. empty) and the payload is included (resp. not included) then they get their participation bits (target, source and head timely) set. Otherwise they get a penalty as a missed attestation. +- Attestations to the CL block from these members are just ignored. + +## PTC Attestations + +There are two ways to import PTC attestations. CL blocks contain aggregates, called `PayloadAttestation` in the spec. And committee members broadcast unaggregated `PayloadAttestationMessage`s. The latter are only imported over the wire for the current slot, and the former are only imported on blocks for the previous slot. + + From b4afb3abb95162158f9575b2fd8d3b458e18eff2 Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 29 Aug 2023 15:00:05 -0300 Subject: [PATCH 29/57] doctoc --- specs/_features/epbs/beacon-chain.md | 19 ++++++++++++++++++- specs/_features/epbs/design.md | 15 +++++++++++++++ specs/_features/epbs/fork-choice.md | 5 +++++ 3 files changed, 38 insertions(+), 1 deletion(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 42d07e0f1b..9931d6702a 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -17,11 +17,18 @@ - [Gwei values](#gwei-values) - [Time parameters](#time-parameters-1) - [Rewards and penalties](#rewards-and-penalties) + - [Max operations per block](#max-operations-per-block) - [Incentivization weights](#incentivization-weights) - [Execution](#execution) - [Containers](#containers) - [New containers](#new-containers) + - [`PayloadAttestationData`](#payloadattestationdata) + - [`PayloadAttestation`](#payloadattestation) + - [`PayloadAttestationMessage`](#payloadattestationmessage) + - [`IndexedPayloadAttestation`](#indexedpayloadattestation) - [`SignedExecutionPayloadHeader`](#signedexecutionpayloadheader) + - [`ExecutionPayloadHeaderEnvelope`](#executionpayloadheaderenvelope) + - [`SignedExecutionPayloadHeaderEnvelope`](#signedexecutionpayloadheaderenvelope) - [`ExecutionPayloadEnvelope`](#executionpayloadenvelope) - [`SignedExecutionPayloadEnvelope`](#signedexecutionpayloadenvelope) - [`InclusionListSummaryEntry`](#inclusionlistsummaryentry) @@ -34,8 +41,15 @@ - [`ExecutionPayloadHeader`](#executionpayloadheader) - [`BeaconState`](#beaconstate) - [Helper functions](#helper-functions) + - [Math](#math) + - [`bit_floor`](#bit_floor) - [Predicates](#predicates) - [`is_builder`](#is_builder) + - [`is_valid_indexed_payload_attestation`](#is_valid_indexed_payload_attestation) + - [Beacon State accessors](#beacon-state-accessors) + - [`get_ptc`](#get_ptc) + - [`get_payload_attesting_indices`](#get_payload_attesting_indices) + - [`get_indexed_payload_attestation`](#get_indexed_payload_attestation) - [Beacon chain state transition function](#beacon-chain-state-transition-function) - [Epoch processing](#epoch-processing) - [Modified `process_epoch`](#modified-process_epoch) @@ -45,8 +59,11 @@ - [Engine APIs](#engine-apis) - [New `notify_new_inclusion_list`](#new-notify_new_inclusion_list) - [Block processing](#block-processing) + - [Modified `process_operations`](#modified-process_operations) + - [Modified `process_attestation`](#modified-process_attestation) + - [Payload Attestations](#payload-attestations) - [Modified `process_withdrawals`](#modified-process_withdrawals) - - [New `verify_execution_payload_header_signature`](#new-verify_execution_payload_header_signature) + - [New `verify_execution_payload_header_envelope_signature`](#new-verify_execution_payload_header_envelope_signature) - [New `process_execution_payload_header`](#new-process_execution_payload_header) - [New `verify_execution_payload_signature`](#new-verify_execution_payload_signature) - [Modified `process_execution_payload`](#modified-process_execution_payload) diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index fb1c0beb7a..944cd3630d 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -1,3 +1,18 @@ + + +**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* + +- [ePBS design notes](#epbs-design-notes) + - [Inclusion lists](#inclusion-lists) + - [Builders](#builders) + - [Builder Payments](#builder-payments) + - [Withdrawals](#withdrawals) + - [Blobs](#blobs) + - [PTC Rewards](#ptc-rewards) + - [PTC Attestations](#ptc-attestations) + + + # ePBS design notes ## Inclusion lists diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 66fe228f0e..310a1c3598 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -6,13 +6,18 @@ - [Introduction](#introduction) +- [Constant](#constant) - [Helpers](#helpers) + - [Modified `Store`](#modified-store) - [`verify_inclusion_list`](#verify_inclusion_list) - [`is_inclusion_list_available`](#is_inclusion_list_available) + - [`notify_ptc_messages`](#notify_ptc_messages) + - [`is_payload_present`](#is_payload_present) - [Updated fork-choice handlers](#updated-fork-choice-handlers) - [`on_block`](#on_block) - [New fork-choice handlers](#new-fork-choice-handlers) - [`on_execution_payload`](#on_execution_payload) + - [`on_payload_attestation_message`](#on_payload_attestation_message) From e1a904da18836f23bd38111e9eaaf57ce854a71a Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 29 Aug 2023 21:15:04 -0300 Subject: [PATCH 30/57] Design decisions in forkchoice --- specs/_features/epbs/beacon-chain.md | 2 +- specs/_features/epbs/design.md | 56 +++++++++++++++++++++++++++- specs/_features/epbs/fork-choice.md | 1 + 3 files changed, 57 insertions(+), 2 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 9931d6702a..37f74b2341 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -650,7 +650,7 @@ def process_payload_attestation(state: BeaconState, payload_attestation: Payload ## Check that the attestation is for the parent beacon block data = payload_attestation.data assert data.beacon_block_root == state.latest_block_header.parent_root - ## Check that the attestation is for the previous slot + ## Check that the attestation is for the previous slot (TODO: Fix this to be the head block root or simply hurt the ptc members) assert state.slot > 0 assert data.beacon_block_root == state.block_roots[(state.slot - 1) % SLOTS_PER_HISTORICAL_ROOT] diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 944cd3630d..2311abc801 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -61,4 +61,58 @@ So when importing the CL block for slot N, we process the expected withdrawals a There are two ways to import PTC attestations. CL blocks contain aggregates, called `PayloadAttestation` in the spec. And committee members broadcast unaggregated `PayloadAttestationMessage`s. The latter are only imported over the wire for the current slot, and the former are only imported on blocks for the previous slot. - +TODO: These import are current broken in the specs in that they only allow a PTC member of the current slot to vote for a beacon block of the current slot, that is, if their head was a previous beacon block, their vote will be disregarded completely and they will be penalized. + +## Forkchoice changes + +There are significant design decisions to make due to the fact that a slot can have 3 different statuses: + +1. The CL block is not included (therefore no payload can be included). This is a *skipped* slot. +2. The CL block is included and the payload is not revealed. This is an *empty* slot. +3. The CL block and the payload are both included. This is a *full* slot. + +Consider the following fork +```mermaid +graph RL +A[N-1, Full] +B[N, Empty] --> A +C[N, Full] --> A +D[N+1, Full] --> B +``` + +In this fork the proposer of `N+1` is attempting to reorg the payload of `N` that was seen by the majority of the PTC. Suppose that honest validators see that the PTC has voted `N` to be full. Then because of proposer boost, the CL block of `N+1` will have 40% of the committee to start with. Assuming perfect participation, honest validators should see a weight of `100` for `(N, Full)` and a weight of `40` for `N+1` (notice that they attest before seeing the payload). They should choose to vote for `(N, Full)` instead of `N+1`. The question is how do we account for all of this? A few initial comments are in line +- CL attestation do not mention full or empty they simply have a beacon block root. Honest validators will have already set their PTC vote during `N` that `N` was full. +- The only changes to the view of `N` as empty/full could come only when importing `N+1`, a beacon block that contains PTC Messages attesting for the payload of `N`. However, if honest validators have already achieved the threshold for `full`, they will consider the block full. +- This is one design decision: instead of having a hard threshold on the PTC (50% in the current spec) we could have a relative one, say for example a simple majority of the counted votes. This has some minor engineering problems (like keeping track of who voted in forkchoice more than simply if they voted for present or not), but this can easily be changed if there are some liveness concerns. +- The honest builder for `N+1` would not submit a bid here, since honest builders would have seen `N` as full also, they would only build on top of the blockhash included in `N`. +- The honest PTC members for `N+1` will vote for + +So the questions is what changes do we need to make to our current weight accounting so that we have `(N, Full)` and `(N+1, Full)` as viable for head above, but not `(N, Empty)`?. Moreover, we want `(N, Full)` to be the winner in the above situation. Before dwelling in the justification, let me say right away that a proposer for `N+2` would call `get_head` and would get `N.root`. And then he will call `is_payload_present(N.root)` and he will get `True` so he will propose based on `(N, Full)` reorging back the dishonest (malinformed) proposer of `N+1`. The implementation of `is_payload_present` is trivial so the only question is how to do LMD counting so that `N` beats `N+1` in the head computation. + +There are several notions that can be changed when we have *empty* or *full* slots. +- Checkpoints: + - we can consider checkpoints to be of the form `(Epoch, Root, bool)` where the `bool` is to indicate if the Beacon block was full or not. + - Another option is to consider checkpoints to be of the form `(Epoch, Root)` exactly as we do today, but only consider the last *full* block before or equal to the `Epoch` start. +Both have advantages and disadvantages, the second one allows for different contending states to be the first state of an epoch, but it keeps all implementations exactly as they are today. +- Ancestry computations, as in `get_ancestor`. + - We can change the signature of this function to be of the form `get_ancestor(Store, Root, slot) -> (Root, bool)` So that it returns the beacon block root and weather or not it is based on *Full* or *Empty*. + - Otherwise we can simply return the last *Full* block in the line of ancestry. Again there are advantages and disadvantages. In this last case, it would be very hard to know if two given blocks with a given payload status are in the same chain or not. + + +The proposal I am considering at this moment is the following: +- Keep checkpoints as `(Epoch, Root) ` and allow different start of epoch blocks. +- An honest validator, when requesting the state at a given block root, will use its canonical state. That is computed as follows. In the example above, when requesting the state with block root `N`, if a call to `get_head` returned `N+1` then the validator would return the `store.block_states[N.root]` which corresponds to `N, Empty`. If instead returned `N` then it would return the state `store.execution_payload_states[N.root]` which corresponds to `N, Full`. +- Thus, when requesting the *justified state* for example, it will use the state that actually corresponds to its canonical chain and he needs to track only `Epoch, Root` for checkpoints, with minimal code changes. +- For LMD accounting, the proposal is to keep weights exactly as today with one exception: direct votes for `N` are *only* counted in the chains supporting `N, Full` or `N, Empty` according to the PTC vote. So, in the fork above, any honest validator that voted for `N` during slot `N` will be counted in the chain for `N, Full`, but not in the chain of `N+1, Full`. Honest validators during `N+1` will also vote for `N`, and also counting their votes in for `N, Full` and not the attacker's `N+1`. Suppose the chain advances with two more bad blocks as follows +```mermaid +graph RL +A[N-1, Full] +B[N, Empty] --> A +C[N, Full] --> A +D[N+1, Full] --> B +G[N+1, Empty] --> B +E[N+2, Full] --> D +F[N+3, Full] --> G +F ~~~ E +``` +In this case all the attesters for `N+1` will be counted depending on the PTC members that voted for `(N+1, Full)`. Assuming honest PTC members, they would have voted for `N` during `N+1` so any CL attesters for `N+1` would be voting for `N+1, Empty` thus only counting for the head in `(N+3, Full)`. diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 310a1c3598..1c829ae827 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -252,6 +252,7 @@ def on_excecution_payload(store: Store, signed_envelope: SignedExecutionPayloadE ### `on_payload_attestation_message` +TODO: Fix this to allow votes for the head block? (or simply hurt the ptc members) ```python def on_payload_attestation_message(store: Store, ptc_message: PayloadAttestationMessage, is_from_block: bool=False) -> None: From f52bbe4a0857741c1df0d3c85109d338afff05bf Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 29 Aug 2023 21:22:37 -0300 Subject: [PATCH 31/57] checkpoint descriptions --- specs/_features/epbs/design.md | 1 + 1 file changed, 1 insertion(+) diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 2311abc801..4479d4f34b 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -94,6 +94,7 @@ There are several notions that can be changed when we have *empty* or *full* slo - we can consider checkpoints to be of the form `(Epoch, Root, bool)` where the `bool` is to indicate if the Beacon block was full or not. - Another option is to consider checkpoints to be of the form `(Epoch, Root)` exactly as we do today, but only consider the last *full* block before or equal to the `Epoch` start. Both have advantages and disadvantages, the second one allows for different contending states to be the first state of an epoch, but it keeps all implementations exactly as they are today. + - A third approach, which seems the best so far, is to keep `(Epoch, Root)` and let head of the chain determine if it is *Full* or *Empty* as described below. - Ancestry computations, as in `get_ancestor`. - We can change the signature of this function to be of the form `get_ancestor(Store, Root, slot) -> (Root, bool)` So that it returns the beacon block root and weather or not it is based on *Full* or *Empty*. - Otherwise we can simply return the last *Full* block in the line of ancestry. Again there are advantages and disadvantages. In this last case, it would be very hard to know if two given blocks with a given payload status are in the same chain or not. From 1478a19ce40a26372c8a69dc503b332c3e66c873 Mon Sep 17 00:00:00 2001 From: Potuz Date: Wed, 30 Aug 2023 08:28:08 -0300 Subject: [PATCH 32/57] fix PTC rewards for old blocks --- specs/_features/epbs/beacon-chain.md | 14 +++++++------- specs/_features/epbs/design.md | 6 +++--- specs/_features/epbs/fork-choice.md | 10 ++++++---- 3 files changed, 16 insertions(+), 14 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 37f74b2341..e17d310031 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -166,6 +166,7 @@ For a further introduction please refer to this [ethresear.ch article](https://e ```python class PayloadAttestationData(Container): beacon_block_root: Root + slot: Slot payload_present: bool ``` @@ -650,23 +651,22 @@ def process_payload_attestation(state: BeaconState, payload_attestation: Payload ## Check that the attestation is for the parent beacon block data = payload_attestation.data assert data.beacon_block_root == state.latest_block_header.parent_root - ## Check that the attestation is for the previous slot (TODO: Fix this to be the head block root or simply hurt the ptc members) - assert state.slot > 0 - assert data.beacon_block_root == state.block_roots[(state.slot - 1) % SLOTS_PER_HISTORICAL_ROOT] + ## Check that the attestation is for the previous slot + assert data.slot + 1 == state.slot #Verify signature - indexed_payload_attestation = get_indexed_payload_attestation(state, state.slot - 1, payload_attestation) + indexed_payload_attestation = get_indexed_payload_attestation(state, data.slot, payload_attestation) assert is_valid_indexed_payload_attestation(state, indexed_payload_attestation) - ptc = get_ptc(state, state.slot - 1) - if slot % SLOTS_PER_EPOCH == 0: + ptc = get_ptc(state, data.slot) + if state.slot % SLOTS_PER_EPOCH == 0: epoch_participation = state.previous_epoch_participation else: epoch_participation = state.current_epoch_participation # Return early if the attestation is for the wrong payload status latest_payload_timestamp = state.latest_execution_payload_header.timestamp - present_timestamp = compute_timestamp_at_slot(state, state.slot - 1) + present_timestamp = compute_timestamp_at_slot(state, data.slot) payload_was_present = latest_payload_timestamp == present_timestamp if data.payload_present != payload_was_present: return diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 4479d4f34b..626b17f4ad 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -10,6 +10,7 @@ - [Blobs](#blobs) - [PTC Rewards](#ptc-rewards) - [PTC Attestations](#ptc-attestations) + - [Forkchoice changes](#forkchoice-changes) @@ -61,8 +62,6 @@ So when importing the CL block for slot N, we process the expected withdrawals a There are two ways to import PTC attestations. CL blocks contain aggregates, called `PayloadAttestation` in the spec. And committee members broadcast unaggregated `PayloadAttestationMessage`s. The latter are only imported over the wire for the current slot, and the former are only imported on blocks for the previous slot. -TODO: These import are current broken in the specs in that they only allow a PTC member of the current slot to vote for a beacon block of the current slot, that is, if their head was a previous beacon block, their vote will be disregarded completely and they will be penalized. - ## Forkchoice changes There are significant design decisions to make due to the fact that a slot can have 3 different statuses: @@ -85,7 +84,8 @@ In this fork the proposer of `N+1` is attempting to reorg the payload of `N` tha - The only changes to the view of `N` as empty/full could come only when importing `N+1`, a beacon block that contains PTC Messages attesting for the payload of `N`. However, if honest validators have already achieved the threshold for `full`, they will consider the block full. - This is one design decision: instead of having a hard threshold on the PTC (50% in the current spec) we could have a relative one, say for example a simple majority of the counted votes. This has some minor engineering problems (like keeping track of who voted in forkchoice more than simply if they voted for present or not), but this can easily be changed if there are some liveness concerns. - The honest builder for `N+1` would not submit a bid here, since honest builders would have seen `N` as full also, they would only build on top of the blockhash included in `N`. -- The honest PTC members for `N+1` will vote for +- The honest PTC members for `N+1` will vote for `N, Full` they will be rewarded but they will not change the forkchoice view that `N` was already full. +- PTC members voting for a previous blockroot cannot change the forkchoice view of the payload status either way. So the questions is what changes do we need to make to our current weight accounting so that we have `(N, Full)` and `(N+1, Full)` as viable for head above, but not `(N, Empty)`?. Moreover, we want `(N, Full)` to be the winner in the above situation. Before dwelling in the justification, let me say right away that a proposer for `N+2` would call `get_head` and would get `N.root`. And then he will call `is_payload_present(N.root)` and he will get `True` so he will propose based on `(N, Full)` reorging back the dishonest (malinformed) proposer of `N+1`. The implementation of `is_payload_present` is trivial so the only question is how to do LMD counting so that `N` beats `N+1` in the head computation. diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 1c829ae827..502d4973f0 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -252,7 +252,6 @@ def on_excecution_payload(store: Store, signed_envelope: SignedExecutionPayloadE ### `on_payload_attestation_message` -TODO: Fix this to allow votes for the head block? (or simply hurt the ptc members) ```python def on_payload_attestation_message(store: Store, ptc_message: PayloadAttestationMessage, is_from_block: bool=False) -> None: @@ -263,18 +262,21 @@ def on_payload_attestation_message(store: Store, data = ptc_message.data # PTC attestation must be for a known block. If block is unknown, delay consideration until the block is found state = store.block_states[data.beacon_block_root] - ptc = get_ptc(state, state.slot) + ptc = get_ptc(state, data.slot) + # PTC votes can only change the vote for their assigned beacon block, return early otherwise + if data.slot != state.slot: + return # Verify the signature and check that its for the current slot if it is coming from the wire if not is_from_block: # Check that the attestation is for the current slot - assert state.slot == get_current_slot(store) + assert data.slot == get_current_slot(store) # Check that the attester is from the current ptc assert ptc_message.validator_index in ptc # Verify the signature assert is_valid_indexed_payload_attestation(state, IndexedPayloadAttestation(attesting_indices = [ptc_message.validator_index], data = data, - signature = ptc_message signature)) + signature = ptc_message.signature)) # Update the ptc vote for the block # TODO: Do we want to slash ptc members that equivocate? # we are updating here the message and so the last vote will be the one that counts. From c767fbfdfe277f81bbff959ba8f01f9a4a3983c6 Mon Sep 17 00:00:00 2001 From: Potuz Date: Wed, 30 Aug 2023 10:04:06 -0300 Subject: [PATCH 33/57] update forkchoice weights, take 1 --- specs/_features/epbs/fork-choice.md | 106 +++++++++++++++++++++++++++- 1 file changed, 105 insertions(+), 1 deletion(-) diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 502d4973f0..9274304a23 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -144,7 +144,111 @@ def is_payload_present(store: Store, beacon_block_root: Root) -> bool: assert beacon_block_root in store.ptc_vote return ptc_vote[beacon_block_root].count(True) > PAYLOAD_TIMELY_THRESHOLD ``` - + +### Modified `get_ancestor` +**Note:** `get_ancestor` is modified to return whether the chain is based on an *empty* or *full* block. + +```python +def get_ancestor(store: Store, root: Root, slot: Slot) -> tuple[Root, bool]: + """ + returns the beacon block root of the ancestor of the beacon block with ``root`` at``slot`` and it also + returns ``true`` if it based on a full block and ``false`` otherwise. + If the beacon block with ``root`` is already at ``slot`` it returns it's PTC status. + """ + block = store.blocks[root] + if block.slot == slot: + return [root, store.is_payload_present(root)] + parent = store.blocks[block.parent_root] + if parent.slot > slot: + return get_ancestor(store, block.parent_root, slot) + if block.body.signed_execution_payload_header_envelope.message.parent_hash == + parent.body.signed_execution_payload_header_envelope.message.block_hash: + return (block.parent_root, True) + return (block.parent_root, False) +``` + +### Modified `get_checkpoint_block` +**Note:** `get_checkpoint_block` is modified to use the new `get_ancestor` + +```python +def get_checkpoint_block(store: Store, root: Root, epoch: Epoch) -> Root: + """ + Compute the checkpoint block for epoch ``epoch`` in the chain of block ``root`` + """ + epoch_first_slot = compute_start_slot_at_epoch(epoch) + (ancestor_root,_) = get_ancestor(store, root, epoch_first_slot) + return ancestor_root +``` + + +### `is_supporting_vote` + +```python +def is_supporting_vote(store: Store, root: Root, is_payload_present: bool, message_root: Root) -> bool: + """ + returns whether a vote for ``message_root`` supports the chain containing the beacon block ``root`` with the + payload contents indicated by ``is_payload_present``. + """ + (ancestor_root, is_ancestor_full) = get_ancestor(store, message_root, store.blocks[root].slot) + return (root == ancestor_root) and (is_payload_preset == is_ancestor_full) +``` + +### Modified `get_weight` + +**Note:** `get_weight` is modified to only count votes for descending chains that support the status of a pair `Root, bool`, where the `bool` indicates if the block was full or not. + +```python +def get_weight(store: Store, root: Root, is_payload_present: bool) -> Gwei: + state = store.checkpoint_states[store.justified_checkpoint] + unslashed_and_active_indices = [ + i for i in get_active_validator_indices(state, get_current_epoch(state)) + if not state.validators[i].slashed + ] + attestation_score = Gwei(sum( + state.validators[i].effective_balance for i in unslashed_and_active_indices + if (i in store.latest_messages + and i not in store.equivocating_indices + and is_supporting_vote(store, root, is_payload_present, store.latest_messages[i].root)) + )) + if store.proposer_boost_root == Root(): + # Return only attestation score if ``proposer_boost_root`` is not set + return attestation_score + + # Calculate proposer score if ``proposer_boost_root`` is set + proposer_score = Gwei(0) + # Boost is applied if ``root`` is an ancestor of ``proposer_boost_root`` + (ancestor_root, is_ancestor_full) = get_ancestor(store, store.proposer_boost_root, store.blocks[root].slot) + if (ancestor_root == root) and (is_ancestor_full == is_payload_present): + committee_weight = get_total_active_balance(state) // SLOTS_PER_EPOCH + proposer_score = (committee_weight * PROPOSER_SCORE_BOOST) // 100 + return attestation_score + proposer_score +``` + +### Modified `get_head` + +**Note:** `get_head` is modified to use the new `get_weight` function. It returns the Beacon block root of the head block and whether its payload is considered present or not. + +```python +def get_head(store: Store) -> tuple[Root, bool]: + # Get filtered block tree that only includes viable branches + blocks = get_filtered_block_tree(store) + # Execute the LMD-GHOST fork choice + head_root = store.justified_checkpoint.root + head_full = is_payload_present(store, head_root) + while True: + children = [ + (root, present) for root in blocks.keys() + if blocks[root].parent_root == head for present in (True, False) + ] + if len(children) == 0: + return (head_root, head_full) + # Sort by latest attesting balance with ties broken lexicographically + # Ties broken by favoring block with lexicographically higher root + # Ties then broken by favoring full blocks + # TODO: Can (root, full), (root, empty) have the same weight? + head = max(children, key=lambda (root, present): (get_weight(store, root, present), root, present)) +``` + ## Updated fork-choice handlers ### `on_block` From 73ee96b2e64b1865a545665f63ae9010ad0167bc Mon Sep 17 00:00:00 2001 From: Potuz Date: Wed, 30 Aug 2023 10:05:39 -0300 Subject: [PATCH 34/57] doctoc updated --- specs/_features/epbs/fork-choice.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 9274304a23..3bf5231f09 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -13,6 +13,11 @@ - [`is_inclusion_list_available`](#is_inclusion_list_available) - [`notify_ptc_messages`](#notify_ptc_messages) - [`is_payload_present`](#is_payload_present) + - [Modified `get_ancestor`](#modified-get_ancestor) + - [Modified `get_checkpoint_block`](#modified-get_checkpoint_block) + - [`is_supporting_vote`](#is_supporting_vote) + - [Modified `get_weight`](#modified-get_weight) + - [Modified `get_head`](#modified-get_head) - [Updated fork-choice handlers](#updated-fork-choice-handlers) - [`on_block`](#on_block) - [New fork-choice handlers](#new-fork-choice-handlers) From 9c7b739cc83edd24497ba771eec41caab86abdbd Mon Sep 17 00:00:00 2001 From: Potuz Date: Wed, 30 Aug 2023 11:52:36 -0300 Subject: [PATCH 35/57] add head_root --- specs/_features/epbs/fork-choice.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 3bf5231f09..e0b25c321d 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -243,7 +243,7 @@ def get_head(store: Store) -> tuple[Root, bool]: while True: children = [ (root, present) for root in blocks.keys() - if blocks[root].parent_root == head for present in (True, False) + if blocks[root].parent_root == head_root for present in (True, False) ] if len(children) == 0: return (head_root, head_full) From 6046d59977d117b7663932d45b7a334e351635b4 Mon Sep 17 00:00:00 2001 From: Potuz Date: Fri, 1 Sep 2023 13:45:41 -0300 Subject: [PATCH 36/57] add comment on equivocation --- specs/_features/epbs/design.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 626b17f4ad..7ee22d7a4e 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -117,3 +117,13 @@ F[N+3, Full] --> G F ~~~ E ``` In this case all the attesters for `N+1` will be counted depending on the PTC members that voted for `(N+1, Full)`. Assuming honest PTC members, they would have voted for `N` during `N+1` so any CL attesters for `N+1` would be voting for `N+1, Empty` thus only counting for the head in `(N+3, Full)`. + +## Equivocations + +There is no need to do anything about proposer equivocations. Builders should reveal their block anyway. + +- At the time of reveal, the builder already has counted attestations for the current CL blocks, even if there are or not equivocations. Any equivocation available at this time will not have transactions that can unbundle him since he hasn't revealed. +- If the original block to which the builder committed is included, then the builder doesn't lose anything, that was the original intent. So if the original block is the overwhelming winner at the time of reveal, the builder can simply reveal and be safe that if there are any equivocations anyway his block was included. +- If the builder reveals, he knows that he can never be unbundled unless the next committee has a majority of malicious validators: attestations will go for an empty block before a block that is revealed after 8 seconds. +- So since the builder cannot be unbundled, then he either doesn't pay if the block is not included, or pays and its included. +- The splitting grief, that is, the proposer's block has about 50% of the vote at 8 seconds, remains. From 4053e677096a5a18e331ba8f67866417b90a851a Mon Sep 17 00:00:00 2001 From: Potuz Date: Fri, 1 Sep 2023 13:47:33 -0300 Subject: [PATCH 37/57] remove signed execution payload header --- specs/_features/epbs/beacon-chain.md | 7 ------- 1 file changed, 7 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index e17d310031..4cbcca8eeb 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -197,13 +197,6 @@ class IndexedPayloadAttestation(Container): signature: BLSSignature ``` -#### `SignedExecutionPayloadHeader` - -```python -class SignedExecutionPayloadHeader(Container): - message: ExecutionPayloadHeader - signature: BLSSignature -``` #### `ExecutionPayloadHeaderEnvelope` ```python From 7beb2eb7745a753cedbf3729350b0b828438064c Mon Sep 17 00:00:00 2001 From: Potuz Date: Fri, 1 Sep 2023 13:49:19 -0300 Subject: [PATCH 38/57] typos --- specs/_features/epbs/beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 4cbcca8eeb..9363184a45 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -77,7 +77,7 @@ This is the beacon chain specification of the enshrined proposer builder separat *Note:* This specification is built upon [Deneb](../../deneb/beacon-chain.md) and is under active development. -This feature adds new staked consensus participants called *Builders* and new honest validators duties called *payload timeliness attestations*. The slot is divided in **four** intervals as opposed to the current three. Honest validators gather *signed bids* from builders and submit their consensus blocks (a `SignedBlindedBeaconBlock`) at the beginning of the slot. At the start of the second interval, honest validators submit attestations just as they do previous to this feature). At the start of the third interval, aggregators aggregate these attestations (exactly as before this feature) and the honest builder reveals the full payload. At the start of the fourth interval, some honest validators selected to be members of the new **Payload Timeliness Committee** attest to the presence of the builder's payload. +This feature adds new staked consensus participants called *Builders* and new honest validators duties called *payload timeliness attestations*. The slot is divided in **four** intervals as opposed to the current three. Honest validators gather *signed bids* from builders and submit their consensus blocks (a `SigneddBeaconBlock`) at the beginning of the slot. At the start of the second interval, honest validators submit attestations just as they do previous to this feature). At the start of the third interval, aggregators aggregate these attestations (exactly as before this feature) and the honest builder reveals the full payload. At the start of the fourth interval, some honest validators selected to be members of the new **Payload Timeliness Committee** attest to the presence of the builder's payload. At any given slot, the status of the blockchain's head may be either - A *full* block from a previous slot (eg. the current slot's proposer did not submit its block). @@ -453,7 +453,7 @@ def is_valid_indexed_payload_attestation(state: BeaconState, indexed_payload_att ```python def get_ptc(state: BeaconState, slot: Slot) -> Vector[ValidatorIndex, PTC_SIZE]: """ - Get the ptc committee for the give ``slot`` + Get the ptc committee for the given ``slot`` """ epoch = compute_epoch_at_slot(slot) committees_per_slot = bit_floor(max(get_committee_count_per_slot(state, epoch), PTC_SIZE)) From 1d0444c36c132e13d8b21d49cdf42c48f56746b8 Mon Sep 17 00:00:00 2001 From: Potuz Date: Mon, 4 Sep 2023 18:00:33 -0300 Subject: [PATCH 39/57] add max EB changes --- specs/_features/epbs/beacon-chain.md | 609 +++++++++++++++++++++- specs/_features/epbs/design.md | 3 + specs/_features/epbs/fork-choice.md | 2 +- specs/_features/epbs/weak-subjectivity.md | 52 ++ 4 files changed, 655 insertions(+), 11 deletions(-) create mode 100644 specs/_features/epbs/weak-subjectivity.md diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 9363184a45..c57150cb19 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -94,6 +94,11 @@ For a further introduction please refer to this [ethresear.ch article](https://e | - | - | | `BUILDER_WITHDRAWAL_PREFIX` | `Bytes1('0x0b')` # (New in ePBS) | +### Slashing flags +| Name | Value | +| - | - | +| `SLASHED_ATTESTER_FLAG_INDEX`| `0` # (New in ePBS)| +| `SLASHED_PROPOSER_FLAG_INDEX`| `1` # (New in ePBS)| ## Configuration @@ -133,6 +138,12 @@ For a further introduction please refer to this [ethresear.ch article](https://e | - | - | :-: | :-: | | `MIN_SLOTS_FOR_INCLUSION_LISTS_REQUESTS` | `uint64(2)` | slots | 32 seconds # (New in ePBS) | +### State list lenghts +| Name | Value | Unit | Duration | +| - | - | :-: | :-: | +| `MAX_PENDING_BALANCE_DEPOSITS` | `uint64(2**20) = 1 048 576` | `PendingBalanceDeposits` | #(New in ePBS) | +| `MAX_PENDING_PARTIAL_WITHDRAWALS` | `uint64(2**20) = 1 048 576` | `PartialWithdrawals` | # (New in ePBS) | + ### Rewards and penalties | Name | Value | @@ -161,6 +172,32 @@ For a further introduction please refer to this [ethresear.ch article](https://e ### New containers +#### `PendingBalanceDeposit` + +```python +class PendingBalanceDeposit(Container): + index: ValidatorIndex + amount: Gwei +``` + +#### `PartialWithdrawal` + +```python +class PartialWithdrawal(Container) + index: ValidatorIndex + amount: Gwei + withdrawable_epoch: Epoch +``` + +#### `ExecutionLayerWithdrawalRequest` + +```python +class ExecutionLayerWithdrawalRequest(Container) + source_address: ExecutionAddress + validator_pubkey: BLSPubkey + balance: Gwei +``` + #### `PayloadAttestationData` ```python @@ -267,6 +304,23 @@ class InclusionList(Container) ### Modified containers +#### `Validator` +**Note:** The `Validator` class is modified to keep track of the slashed categories. + +```python +class Validator(Container): + pubkey: BLSPubkey + withdrawal_credentials: Bytes32 # Commitment to pubkey for withdrawals + effective_balance: Gwei # Balance at stake + slashed: uint8 # (Modified in ePBS) + # Status epochs + activation_eligibility_epoch: Epoch # When criteria for activation were met + activation_epoch: Epoch + exit_epoch: Epoch + withdrawable_epoch: Epoch # When validator can withdraw funds +``` + + #### `BeaconBlockBody` **Note:** The Beacon Block body is modified to contain a Signed `ExecutionPayloadHeader`. The containers `BeaconBlock` and `SignedBeaconBlock` are modified indirectly. @@ -350,7 +404,7 @@ class ExecutionPayloadHeader(Container): ``` #### `BeaconState` -*Note*: the beacon state is modified to store a signed latest execution payload header. +*Note*: the beacon state is modified to store a signed latest execution payload header, to track the last withdrawals and increased Maximum effective balance fields: `deposit_balance_to_consume`, `exit_balance_to_consume` and `earliest_exit_epoch`. ```python class BeaconState(Container): @@ -398,6 +452,11 @@ class BeaconState(Container): # PBS signed_execution_payload_header_envelope: SignedExecutionPayloadHeaderEnvelop # [New in ePBS] last_withdrawals_root: Root # [New in ePBS] + deposit_balance_to_consume: Gwei # [New in ePBS] + exit_balance_to_consume: Gwei # [New in ePBS] + earliest_exit_epoch: Epoch # [New in ePBS] + pending_balance_deposits: List[PendingBalanceDeposit, MAX_PENDING_BALANCE_DEPOSITS] # [New in ePBS] + pending_partial_withdrawals: List[PartialWithdrawals, MAX_PENDING_PARTIAL_WITHDRAWALS] # [New in ePBS] ``` ## Helper functions @@ -428,6 +487,77 @@ def is_builder(validator: Validator) -> bool: return validator.withdrawal_credentials[0] == BUILDER_WITHDRAWAL_PREFIX ``` +#### `is_eligible_for_activation_queue` + +```python +def is_eligible_for_activation_queue(validator: Validator) -> bool: + """ + Check if ``validator`` is eligible to be placed into the activation queue. + """ + return ( + validator.activation_eligibility_epoch == FAR_FUTURE_EPOCH + and validator.effective_balance >= MIN_ACTIVATION_BALANCE + ) +``` + +#### `is_slashed_proposer` + +```python +def is_slashed_proposer(validator: Validator) -> bool: + """ + return ``true`` if ``validator`` has committed a proposer equivocation + """ + return has_flag(ParticipationFlags(validator.slashed), SLASHED_PROPOSER_FLAG_INDEX) +``` + +#### `is_slashed_attester` + +```python +def is_slashed_attester(validator: Validator) -> bool: + """ + return ``true`` if ``validator`` has committed an attestation slashing offense + """ + return has_flag(ParticipationFlags(validator.slashed), SLASHED_ATTESTSER_FLAG_INDEX) +``` + + +#### Modified `is_slashable_validator` +**Note:** The function `is_slashable_validator` is modified and renamed to `is_attester_slashable_validator`. + +```python +def is_attester_slashable_validator(validator: Validator, epoch: Epoch) -> bool: + """ + Check if ``validator`` is slashable. + """ + return (not is_slashed_attester(validator)) and (validator.activation_epoch <= epoch < validator.withdrawable_epoch) +``` + +#### Modified `is_fully_withdrawable_validator` + +```python +def is_fully_withdrawable_validator(validator: Validator, balance: Gwei, epoch: Epoch) -> bool: + """ + Check if ``validator`` is fully withdrawable. + """ + return ( + (has_eth1_withdrawal_credential(validator) or is_builder(validator)) + and validator.withdrawable_epoch <= epoch + and balance > 0 + ) +``` + +#### `is_partially_withdrawable_validator` + +```python +def is_partially_withdrawable_validator(validator: Validator, balance: Gwei) -> bool: + """ + Check if ``validator`` is partially withdrawable. + """ + if not (has_eth1_withdrawal_credential(validator) or is_builder(validator)): + return False + return get_validator_excess_balance(validator, balance) > 0 +``` + #### `is_valid_indexed_payload_attestation` ```python @@ -448,6 +578,18 @@ def is_valid_indexed_payload_attestation(state: BeaconState, indexed_payload_att ### Beacon State accessors +#### Modified `get_eligible_validator_indices` +**Note:** The function `get_eligible_validator_indices` is modified to use the new flag mechanism for slashings. + +```python +def get_eligible_validator_indices(state: BeaconState) -> Sequence[ValidatorIndex]: + previous_epoch = get_previous_epoch(state) + return [ + ValidatorIndex(index) for index, v in enumerate(state.validators) + if is_active_validator(v, previous_epoch) or (is_slashed_attester(v) and previous_epoch + 1 < v.withdrawable_epoch) + ] +``` + #### `get_ptc` ```python @@ -497,6 +639,196 @@ def get_indexed_payload_attestation(state: BeaconState, ) ``` +#### `get_validator_excess_balance` + +```python +def get_validator_excess_balance(validator: Validator, balance: Gwei) -> Gwei: + if has_eth1_withdrawal_credential(validator) and balance > MIN_ACTIVATION_BALANCE: + return balance - MIN_ACTIVATION_BALANCE + if is_builder(validator) and balance > MAX_EFFECTIVE_BALANCE: + return balance - MAX_EFFECTIVE_BALANCE + return Gwei(0) +``` + +#### Modified `get_validator_churn_limit` + +```python +def get_validator_churn_limit(state: BeaconState) -> Gwei: + """ + Return the validator churn limit for the current epoch. + """ + churn = max(MIN_PER_EPOCH_CHURN_LIMIT * MIN_ACTIVATION_BALANCE, get_total_active_balance(state) // CHURN_LIMIT_QUOTIENT) + return churn - churn % EFFECTIVE_BALANCE_INCREMENT +``` + +#### Modified `get_expected_withdrawals` +**Note:** the function `get_expected_withdrawals` is modified to churn the withdrawals by balance because of the increase in `MAX_EFFECTIVE_BALANCE` + +```python +def get_expected_withdrawals(state: BeaconState) -> Sequence[Withdrawal]: + epoch = get_current_epoch(state) + withdrawal_index = state.next_withdrawal_index + validator_index = state.next_withdrawal_validator_index + withdrawals: List[Withdrawal] = [] + consumed = 0 + for withdrawal in state.pending_partial_withdrawals: + if withdrawal.withdrawable_epoch > epoch or len(withdrawals) == MAX_WITHDRAWALS_PER_PAYLOAD // 2: + break + validator = state.validators[withdrawal.index] + if validator.exit_epoch == FAR_FUTURE_EPOCH and state.balances[withdrawal.index] > MIN_ACTIVATION_BALANCEa: + withdrawble_balance = min(state.balances[withdrawal.index] - MIN_ACTIVATION_BALANCE, withdrawal.amount) + withdrawals.append(Withdrawal( + index=withdrawal_index, + validator_index=withdrawal.index, + address=ExecutionAddress(validator.withdrawal_credentials[12:]), + amount=withdrawable_balance, + )) + withdrawal_index += WithdrawalIndex(1) + consumed += 1 + state.pending_partial_withdrawals = state.pending_partial_withdrawals[consumed:] + + # Sweep for remaining + bound = min(len(state.validators), MAX_VALIDATORS_PER_WITHDRAWALS_SWEEP) + for _ in range(bound): + validator = state.validators[validator_index] + balance = state.balances[validator_index] + if is_fully_withdrawable_validator(validator, balance, epoch): + withdrawals.append(Withdrawal( + index=withdrawal_index, + validator_index=validator_index, + address=ExecutionAddress(validator.withdrawal_credentials[12:]), + amount=balance, + )) + withdrawal_index += WithdrawalIndex(1) + elif is_partially_withdrawable_validator(validator, balance): + withdrawals.append(Withdrawal( + index=withdrawal_index, + validator_index=validator_index, + address=ExecutionAddress(validator.withdrawal_credentials[12:]), + amount=get_validator_excess_balance(validator, balance), + )) + withdrawal_index += WithdrawalIndex(1) + if len(withdrawals) == MAX_WITHDRAWALS_PER_PAYLOAD: + break + validator_index = ValidatorIndex((validator_index + 1) % len(state.validators)) + return withdrawals +``` + +### Beacon state mutators + +#### `compute_exit_epoch_and_update_churn` + +```python +def compute_exit_epoch_and_update_churn(state: BeaconState, exit_balance: Gwei) -> Epoch: + earliest_exit_epoch = compute_activation_exit_epoch(get_current_epoch(state)) + per_epoch_churn = get_validator_churn_limit(state) + # New epoch for exits. + if state.earliest_exit_epoch < earliest_exit_epoch: + state.earliest_exit_epoch = earliest_exit_epoch + state.exit_balance_to_consume = per_epoch_churn + + # Exit fits in the current earliest epoch. + if exit_balance < state.exit_balance_to_consume: + state.exit_balance_to_consume -= exit_balance + else: # Exit doesn't fit in the current earliest epoch. + balance_to_process = exit_balance - state.exit_balance_to_consume + additional_epochs, remainder = divmod(balance_to_process, per_epoch_churn) + state.earliest_exit_epoch += additional_epochs + state.exit_balance_to_consume = per_epoch_churn - remainder + return state.earliest_exit_epoch +``` + +#### Modified `initiate_validator_exit` +**Note:** the function `initiate_validator_exit` is modified to use the new churn mechanism. + +```python +def initiate_validator_exit(state: BeaconState, index: ValidatorIndex) -> None: + """ + Initiate the exit of the validator with index ``index``. + """ + # Return if validator already initiated exit + validator = state.validators[index] + if validator.exit_epoch != FAR_FUTURE_EPOCH: + return + + # Compute exit queue epoch + exit_queue_epoch = compute_exit_epoch_and_update_churn(state, state.balances[index]) + + # Set validator exit epoch and withdrawable epoch + validator.exit_epoch = exit_queue_epoch + validator.withdrawable_epoch = Epoch(validator.exit_epoch + MIN_VALIDATOR_WITHDRAWABILITY_DELAY) +``` + +#### Modified `slash_validator` +**Note:** The function `slash_validator` is modified to use the new flag system. + +```python +def slash_validator(state: BeaconState, + slashed_index: ValidatorIndex, + whistleblower_index: ValidatorIndex=None) -> None: + """ + Slash the validator with index ``slashed_index``. + """ + epoch = get_current_epoch(state) + initiate_validator_exit(state, slashed_index) + validator = state.validators[slashed_index] + validator.slashed = add_flag(validator.slashed, SLASHED_ATTESTER_FLAG_INDEX) + validator.withdrawable_epoch = max(validator.withdrawable_epoch, Epoch(epoch + EPOCHS_PER_SLASHINGS_VECTOR)) + state.slashings[epoch % EPOCHS_PER_SLASHINGS_VECTOR] += validator.effective_balance + decrease_balance(state, slashed_index, validator.effective_balance // MIN_SLASHING_PENALTY_QUOTIENT) + + # Apply proposer and whistleblower rewards + proposer_index = get_beacon_proposer_index(state) + if whistleblower_index is None: + whistleblower_index = proposer_index + whistleblower_reward = Gwei(validator.effective_balance // WHISTLEBLOWER_REWARD_QUOTIENT) + proposer_reward = Gwei(whistleblower_reward // PROPOSER_REWARD_QUOTIENT) + increase_balance(state, proposer_index, proposer_reward) + increase_balance(state, whistleblower_index, Gwei(whistleblower_reward - proposer_reward)) +``` + +## Genesis + +### Modified `initialize_beacon_statre_from_eth1` + +```python +def initialize_beacon_state_from_eth1(eth1_block_hash: Hash32, + eth1_timestamp: uint64, + deposits: Sequence[Deposit]) -> BeaconState: + fork = Fork( + previous_version=GENESIS_FORK_VERSION, + current_version=GENESIS_FORK_VERSION, + epoch=GENESIS_EPOCH, + ) + state = BeaconState( + genesis_time=eth1_timestamp + GENESIS_DELAY, + fork=fork, + eth1_data=Eth1Data(block_hash=eth1_block_hash, deposit_count=uint64(len(deposits))), + latest_block_header=BeaconBlockHeader(body_root=hash_tree_root(BeaconBlockBody())), + randao_mixes=[eth1_block_hash] * EPOCHS_PER_HISTORICAL_VECTOR, # Seed RANDAO with Eth1 entropy + ) + + # Process deposits + leaves = list(map(lambda deposit: deposit.data, deposits)) + for index, deposit in enumerate(deposits): + deposit_data_list = List[DepositData, 2**DEPOSIT_CONTRACT_TREE_DEPTH](*leaves[:index + 1]) + state.eth1_data.deposit_root = hash_tree_root(deposit_data_list) + process_deposit(state, deposit) + + # Process activations + for index, validator in enumerate(state.validators): + balance = state.balances[index] + validator.effective_balance = min(balance - balance % EFFECTIVE_BALANCE_INCREMENT, MAX_EFFECTIVE_BALANCE) + if validator.effective_balance >= MIN_ACTIVATION_BALANCE: + validator.activation_eligibility_epoch = GENESIS_EPOCH + validator.activation_epoch = GENESIS_EPOCH + + # Set genesis validators root for domain separation and chain versioning + state.genesis_validators_root = hash_tree_root(state.validators) + + return state +``` + ## Beacon chain state transition function *Note*: state transition is fundamentally modified in ePBS. The full state transition is broken in two parts, first importing a signed block and then importing an execution payload. @@ -514,11 +846,12 @@ def process_epoch(state: BeaconState) -> None: process_justification_and_finalization(state) process_inactivity_updates(state) process_rewards_and_penalties(state) - process_registry_updates(state) + process_registry_updates(state) # [Modified in ePBS] process_slashings(state) process_eth1_data_reset(state) - process_effective_balance_updates(state) - process_slashings_reset(state) + process_pending_balance_deposits(state) # [New in ePBS] + process_effective_balance_updates(state) # [Modified in ePBS] + process_slashings_reset(state) # [Modified in ePBS] process_randao_mixes_reset(state) process_historical_summaries_update(state) process_participation_flag_updates(state) @@ -526,6 +859,99 @@ def process_epoch(state: BeaconState) -> None: process_builder_updates(state) # [New in ePBS] ``` +#### Helper functions + +##### Modified `process_registry_updates` + +```python +def process_registry_updates(state: BeaconState) -> None: + # Process activation eligibility and ejections + for index, validator in enumerate(state.validators): + if is_eligible_for_activation_queue(validator): + validator.activation_eligibility_epoch = get_current_epoch(state) + 1 + + if ( + is_active_validator(validator, get_current_epoch(state)) + and validator.effective_balance <= EJECTION_BALANCE + ): + initiate_validator_exit(state, ValidatorIndex(index)) + + # Activate all eligible validators + activation_epoch = compute_activation_exit_epoch(get_current_epoch(state)) + for validator in state.validators: + if is_eligible_for_activation(state, validator): + validator.activation_epoch = activation_epoch +``` + +##### `process_pending_balance_deposits` + +```python +def process_pending_balance_deposits(state: BeaconState) -> None: + state.deposit_balance_to_consume += get_validator_churn_limit(state) + next_pending_deposit_index = 0 + for pending_balance_deposit in state.pending_balance_deposits: + if state.deposit_balance_to_consume < pending_balance_deposit.amount: + break + + state.deposit_balance_to_consume -= pending_balance_deposit.amount + increase_balance(state, pending_balance_deposit.index, pending_balance_deposit.amount) + next_pending_deposit_index += 1 + + state.pending_balance_deposits = state.pending_balance_deposits[next_pending_deposit_index:] +``` + +##### Modified `process_effective_balance_updates` + +```python +def process_effective_balance_updates(state: BeaconState) -> None: + # Update effective balances with hysteresis + for index, validator in enumerate(state.validators): + balance = state.balances[index] + HYSTERESIS_INCREMENT = uint64(EFFECTIVE_BALANCE_INCREMENT // HYSTERESIS_QUOTIENT) + DOWNWARD_THRESHOLD = HYSTERESIS_INCREMENT * HYSTERESIS_DOWNWARD_MULTIPLIER + UPWARD_THRESHOLD = HYSTERESIS_INCREMENT * HYSTERESIS_UPWARD_MULTIPLIER + EFFECTIVE_BALANCE_LIMIT = MAX_EFFECTIVE_BALANCE if is_builder(validator) else MIN_ACTIVATION_BALANCE + if ( + balance + DOWNWARD_THRESHOLD < validator.effective_balance + or validator.effective_balance + UPWARD_THRESHOLD < balance + ): + validator.effective_balance = min(balance - balance % EFFECTIVE_BALANCE_INCREMENT, EFFECTIVE_BALANCE_LIMIT) +``` + +##### Modified `process_slashings` +**Note:** The only modification is to use the new flag mechanism + +```python +def process_slashings(state: BeaconState) -> None: + epoch = get_current_epoch(state) + total_balance = get_total_active_balance(state) + adjusted_total_slashing_balance = min(sum(state.slashings) * PROPORTIONAL_SLASHING_MULTIPLIER, total_balance) + for index, validator in enumerate(state.validators): + if is_slashed_attester(validator) and epoch + EPOCHS_PER_SLASHINGS_VECTOR // 2 == validator.withdrawable_epoch: + increment = EFFECTIVE_BALANCE_INCREMENT # Factored out from penalty numerator to avoid uint64 overflow + penalty_numerator = validator.effective_balance // increment * adjusted_total_slashing_balance + penalty = penalty_numerator // total_balance * increment + decrease_balance(state, ValidatorIndex(index), penalty) +``` + +##### Modified `get_unslashed_attesting_indices` +**Note:** The function `get_unslashed_attesting_indices` is modified to return only the attester slashing validators. + +```python +def get_unslashed_participating_indices(state: BeaconState, flag_index: int, epoch: Epoch) -> Set[ValidatorIndex]: + """ + Return the set of validator indices that are both active and unslashed for the given ``flag_index`` and ``epoch``. + """ + assert epoch in (get_previous_epoch(state), get_current_epoch(state)) + if epoch == get_current_epoch(state): + epoch_participation = state.current_epoch_participation + else: + epoch_participation = state.previous_epoch_participation + active_validator_indices = get_active_validator_indices(state, epoch) + participating_indices = [i for i in active_validator_indices if has_flag(epoch_participation[i], flag_index)] + return set(filter(lambda index: not is_slashed_attester(state.validators[index]), participating_indices)) +``` + ### Execution engine #### Request data @@ -563,8 +989,8 @@ def notify_new_inclusion_list(self: ExecutionEngine, ```python def process_block(state: BeaconState, block: BeaconBlock) -> None: - process_block_header(state, block) - process_withdrawals(state) [Modified in ePBS] + process_block_header(state, block) # [Modified in ePBS] + process_withdrawals(state) # [Modified in ePBS] process_execution_payload_header(state, block) # [Modified in ePBS, removed process_execution_payload] process_randao(state, block.body) process_eth1_data(state, block.body) @@ -572,6 +998,33 @@ def process_block(state: BeaconState, block: BeaconBlock) -> None: process_sync_aggregate(state, block.body.sync_aggregate) ``` +#### Modified `process_block_header` +**Note:** the only modification is in the `slashed` verification. + +```python +def process_block_header(state: BeaconState, block: BeaconBlock) -> None: + # Verify that the slots match + assert block.slot == state.slot + # Verify that the block is newer than latest block header + assert block.slot > state.latest_block_header.slot + # Verify that proposer index is the correct index + assert block.proposer_index == get_beacon_proposer_index(state) + # Verify that the parent matches + assert block.parent_root == hash_tree_root(state.latest_block_header) + # Cache current block as the new latest block + state.latest_block_header = BeaconBlockHeader( + slot=block.slot, + proposer_index=block.proposer_index, + parent_root=block.parent_root, + state_root=Bytes32(), # Overwritten in the next process_slot call + body_root=hash_tree_root(block.body), + ) + + # Verify proposer is not slashed + proposer = state.validators[block.proposer_index] + assert proposer.slashed == uint8(0) +``` + #### Modified `process_operations` **Note:** `process_operations` is modified to process PTC attestations @@ -585,16 +1038,71 @@ def process_operations(state: BeaconState, body: BeaconBlockBody) -> None: for operation in operations: fn(state, operation) - for_ops(body.proposer_slashings, process_proposer_slashing) - for_ops(body.attester_slashings, process_attester_slashing) + for_ops(body.proposer_slashings, process_proposer_slashing) # [Modified in ePBS] + for_ops(body.attester_slashings, process_attester_slashing) # [Modified in ePBS] for_ops(body.attestations, process_attestation) for_ops(body.deposits, process_deposit) for_ops(body.voluntary_exits, process_voluntary_exit) for_ops(body.bls_to_execution_changes, process_bls_to_execution_change) for_ops(body.payload_attestations, process_payload_attestation) # [New in ePBS] + for_ops(body.execution_payload_withdraw_request, process_execution_layer_withdraw_request) # [New in ePBS] ``` -#### Modified `process_attestation` +##### Modified Proposer slashings + +```python +def process_proposer_slashing(state: BeaconState, proposer_slashing: ProposerSlashing) -> None: + header_1 = proposer_slashing.signed_header_1.message + header_2 = proposer_slashing.signed_header_2.message + + # Verify header slots match + assert header_1.slot == header_2.slot + # Verify header proposer indices match + assert header_1.proposer_index == header_2.proposer_index + # Verify the headers are different + assert header_1 != header_2 + # Verify the proposer is slashable + proposer = state.validators[header_1.proposer_index] + assert proposer.activation_epoch <= get_current_epoch(state) and not is_slashed_proposer(proposer) + # Verify signatures + for signed_header in (proposer_slashing.signed_header_1, proposer_slashing.signed_header_2): + domain = get_domain(state, DOMAIN_BEACON_PROPOSER, compute_epoch_at_slot(signed_header.message.slot)) + signing_root = compute_signing_root(signed_header.message, domain) + assert bls.Verify(proposer.pubkey, signing_root, signed_header.signature) + + # Apply penalty + penalty = PROPOSER_EQUIVOCATION_PENALTY_FACTOR * EFFECTIVE_BALANCE_INCREMENT + decrease_balance(state, header_1.proposer_index, penalty) + initiate_validator_exit(state, header_1.proposer_index) + proposer.slashed = add_flag(proposer.slashed, SLASHED_PROPOSER_FLAG_INDEX) + + # Apply proposer and whistleblower rewards + proposer_reward = Gwei((penalty // WHISTLEBLOWER_REWARD_QUOTIENT) * PROPOSER_WEIGHT // WEIGHT_DENOMINATOR) + increase_balance(state, get_beacon_proposer_index(state), proposer_reward) +``` + +##### Modified Attester slashings +**Note:** The only modification is the use of `is_attester_slashable_validator` + +```python +def process_attester_slashing(state: BeaconState, attester_slashing: AttesterSlashing) -> None: + attestation_1 = attester_slashing.attestation_1 + attestation_2 = attester_slashing.attestation_2 + assert is_slashable_attestation_data(attestation_1.data, attestation_2.data) + assert is_valid_indexed_attestation(state, attestation_1) + assert is_valid_indexed_attestation(state, attestation_2) + + slashed_any = False + indices = set(attestation_1.attesting_indices).intersection(attestation_2.attesting_indices) + for index in sorted(indices): + if is_attester_slashable_validator(state.validators[index], get_current_epoch(state)): + slash_validator(state, index) + slashed_any = True + assert slashed_any +``` + + +##### Modified `process_attestation` *Note*: The function `process_attestation` is modified to ignore attestations from the ptc @@ -636,6 +1144,51 @@ def process_attestation(state: BeaconState, attestation: Attestation) -> None: increase_balance(state, get_beacon_proposer_index(state), proposer_reward) ``` +##### Modified `get_validator_from_deposit` +**Note:** The function `get_validator_from_deposit` is modified to take only a pubkey and withdrawal credentials and sets the effective balance to zero + +```python +def get_validator_from_deposit(pubkey: BLSPubkey, withdrawal_credentials: Bytes32) -> Validator: + return Validator( + pubkey=pubkey, + withdrawal_credentials=withdrawal_credentials, + activation_eligibility_epoch=FAR_FUTURE_EPOCH, + activation_epoch=FAR_FUTURE_EPOCH, + exit_epoch=FAR_FUTURE_EPOCH, + withdrawable_epoch=FAR_FUTURE_EPOCH, + effective_balance=0, + ) +``` + +##### Modified `apply_deposit` + +```python +def apply_deposit(state: BeaconState, + pubkey: BLSPubkey, + withdrawal_credentials: Bytes32, + amount: uint64, + signature: BLSSignature) -> None: + validator_pubkeys = [v.pubkey for v in state.validators] + if pubkey not in validator_pubkeys: + # Verify the deposit signature (proof of possession) which is not checked by the deposit contract + deposit_message = DepositMessage( + pubkey=pubkey, + withdrawal_credentials=withdrawal_credentials, + amount=amount, + ) + domain = compute_domain(DOMAIN_DEPOSIT) # Fork-agnostic domain since deposits are valid across forks + signing_root = compute_signing_root(deposit_message, domain) + if bls.Verify(pubkey, signing_root, signature): + index = len(state.validators) + state.validators.append(get_validator_from_deposit(pubkey, withdrawal_credentials)) + state.balances.append(0) + state.previous_epoch_participation.append(ParticipationFlags(0b0000_0000)) + state.current_epoch_participation.append(ParticipationFlags(0b0000_0000)) + state.inactivity_scores.append(uint64(0)) + else: + index = ValidatorIndex(validator_pubkeys.index(pubkey)) + state.pending_balance_deposits.append(PendingBalanceDeposit(index=index, amount=amount)) +``` ##### Payload Attestations @@ -677,8 +1230,44 @@ def process_payload_attestation(state: BeaconState, payload_attestation: Payload increase_balance(state, get_beacon_proposer_index(state), proposer_reward) ``` +##### Execution Layer Withdraw Requests + +```python +def process_execution_layer_withdraw_request( + state: BeaconState, + execution_layer_withdraw_request: ExecutionLayerWithdrawRequest + ) -> None: + validator_pubkeys = [v.pubkey for v in state.validators] + validator_index = ValidatorIndex(validator_pubkeys.index(execution_layer_withdraw_request.validator_pubkey)) + validator = state.validators[validator_index] + + # Same conditions as in EIP7002 https://github.com/ethereum/consensus-specs/pull/3349/files#diff-7a6e2ba480d22d8bd035bd88ca91358456caf9d7c2d48a74e1e900fe63d5c4f8R223 + # Verify withdrawal credentials + assert validator.withdrawal_credentials[:1] == ETH1_ADDRESS_WITHDRAWAL_PREFIX + assert validator.withdrawal_credentials[12:] == execution_layer_withdraw_request.source_address + assert is_active_validator(validator, get_current_epoch(state)) + # Verify exit has not been initiated, and slashed + assert validator.exit_epoch == FAR_FUTURE_EPOCH: + # Verify the validator has been active long enough + assert get_current_epoch(state) >= validator.activation_epoch + config.SHARD_COMMITTEE_PERIOD + + pending_balance_to_withdraw = sum(item.amount for item in state.pending_partial_withdrawals if item.index == validator_index) + + available_balance = state.balances[validator_index] - MIN_ACTIVATION_BALANCE - pending_balance_to_withdraw + assert available_balance >= execution_layer_withdraw_request.balance + + exit_queue_epoch = compute_exit_epoch_and_update_churn(state, available_balance) + withdrawable_epoch = Epoch(exit_queue_epoch + MIN_VALIDATOR_WITHDRAWABILITY_DELAY) + + state.pending_partial_withdrawals.append(PartialWithdrawal( + index=validator_index, + amount=available_balance, + withdrawable_epoch=withdrawable_epoch, + )) +``` + #### Modified `process_withdrawals` -**Note:** TODO: This is modified to take only the State as parameter as they are deterministic. Still need to include the MaxEB changes +**Note:** TODO: This is modified to take only the State as parameter as they are deterministic. ```python def process_withdrawals(state: BeaconState) -> None: diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 7ee22d7a4e..743165325d 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -127,3 +127,6 @@ There is no need to do anything about proposer equivocations. Builders should re - If the builder reveals, he knows that he can never be unbundled unless the next committee has a majority of malicious validators: attestations will go for an empty block before a block that is revealed after 8 seconds. - So since the builder cannot be unbundled, then he either doesn't pay if the block is not included, or pays and its included. - The splitting grief, that is, the proposer's block has about 50% of the vote at 8 seconds, remains. + +## Increased Max EB +This PR includes the changes from [this PR](https://github.com/michaelneuder/consensus-specs/pull/3). In particular it includes execution layer triggerable withdrawals. diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index e0b25c321d..ba0d6ad93a 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -207,7 +207,7 @@ def get_weight(store: Store, root: Root, is_payload_present: bool) -> Gwei: state = store.checkpoint_states[store.justified_checkpoint] unslashed_and_active_indices = [ i for i in get_active_validator_indices(state, get_current_epoch(state)) - if not state.validators[i].slashed + if not is_slashed_attester(state.validators[i]) ] attestation_score = Gwei(sum( state.validators[i].effective_balance for i in unslashed_and_active_indices diff --git a/specs/_features/epbs/weak-subjectivity.md b/specs/_features/epbs/weak-subjectivity.md new file mode 100644 index 0000000000..6f79bfbfcd --- /dev/null +++ b/specs/_features/epbs/weak-subjectivity.md @@ -0,0 +1,52 @@ +# ePBS -- Weak Subjectivity Guide + +## Table of contents + + + + + + + +## Weak Subjectivity Period + +### Calculating the Weak Subjectivity Period + +#### Modified `compute_weak_subjectivity_period` +**Note:** The function `compute_weak_subjectivity_period` is modified to use the modified churn in ePBS. + +```python +def compute_weak_subjectivity_period(state: BeaconState) -> uint64: + """ + Returns the weak subjectivity period for the current ``state``. + This computation takes into account the effect of: + - validator set churn (bounded by ``get_validator_churn_limit()`` per epoch), and + - validator balance top-ups (bounded by ``MAX_DEPOSITS * SLOTS_PER_EPOCH`` per epoch). + A detailed calculation can be found at: + https://github.com/runtimeverification/beacon-chain-verification/blob/master/weak-subjectivity/weak-subjectivity-analysis.pdf + """ + ws_period = MIN_VALIDATOR_WITHDRAWABILITY_DELAY + N = len(get_active_validator_indices(state, get_current_epoch(state))) + t = get_total_active_balance(state) // N // ETH_TO_GWEI + T = MAX_EFFECTIVE_BALANCE // ETH_TO_GWEI + delta = get_validator_churn_limit(state) // MIN_ACTIVATION_BALANCE + Delta = MAX_DEPOSITS * SLOTS_PER_EPOCH + D = SAFETY_DECAY + + if T * (200 + 3 * D) < t * (200 + 12 * D): + epochs_for_validator_set_churn = ( + N * (t * (200 + 12 * D) - T * (200 + 3 * D)) // (600 * delta * (2 * t + T)) + ) + epochs_for_balance_top_ups = ( + N * (200 + 3 * D) // (600 * Delta) + ) + ws_period += max(epochs_for_validator_set_churn, epochs_for_balance_top_ups) + else: + ws_period += ( + 3 * N * D * t // (200 * Delta * (T - t)) + ) + + return ws_period +``` + + From d06a874dda838958feea8d654b75a11e38aa52b4 Mon Sep 17 00:00:00 2001 From: Potuz Date: Mon, 4 Sep 2023 18:09:11 -0300 Subject: [PATCH 40/57] add el withdraws in the beacon block body --- specs/_features/epbs/beacon-chain.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index c57150cb19..5c80a08a17 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -155,6 +155,7 @@ For a further introduction please refer to this [ethresear.ch article](https://e | Name | Value | | - | - | | `MAX_PAYLOAD_ATTESTATIONS` | `2**1` (= 2) # (New in ePBS) | +| `MAX_EXECUTION_LAYER_WITHDRAW_REQUESTS` | `2**4` (= 16) # (New in ePBS) | ### Incentivization weights @@ -343,6 +344,8 @@ class BeaconBlockBody(Container): # PBS signed_execution_payload_header_envelope: SignedExecutionPayloadHeaderEnvelope # [New in ePBS] payload_attestations: List[PayloadAttestation, MAX_PAYLOAD_ATTESTATIONS] # [New in ePBS] + execution_payload_withdraw_requests: List[ExecutionLayerWithdrawRequest, MAX_EXECUTION_LAYER_WITHDRAW_REQUESTS] # [New in ePBS] + ``` #### `ExecutionPayload` @@ -1045,7 +1048,7 @@ def process_operations(state: BeaconState, body: BeaconBlockBody) -> None: for_ops(body.voluntary_exits, process_voluntary_exit) for_ops(body.bls_to_execution_changes, process_bls_to_execution_change) for_ops(body.payload_attestations, process_payload_attestation) # [New in ePBS] - for_ops(body.execution_payload_withdraw_request, process_execution_layer_withdraw_request) # [New in ePBS] + for_ops(body.execution_payload_withdraw_requests, process_execution_layer_withdraw_request) # [New in ePBS] ``` ##### Modified Proposer slashings From 470b1f4ba80e78eddb981d38e55029600746df48 Mon Sep 17 00:00:00 2001 From: Potuz Date: Mon, 4 Sep 2023 18:18:16 -0300 Subject: [PATCH 41/57] doctoc --- specs/_features/epbs/beacon-chain.md | 37 +++++++++++++++++++++-- specs/_features/epbs/design.md | 2 ++ specs/_features/epbs/weak-subjectivity.md | 4 +++ 3 files changed, 41 insertions(+), 2 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 5c80a08a17..8dbe69a059 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -9,6 +9,7 @@ - [Introduction](#introduction) - [Constants](#constants) - [Withdrawal prefixes](#withdrawal-prefixes) + - [Slashing flags](#slashing-flags) - [Configuration](#configuration) - [Time parameters](#time-parameters) - [Preset](#preset) @@ -16,17 +17,20 @@ - [Domain types](#domain-types) - [Gwei values](#gwei-values) - [Time parameters](#time-parameters-1) + - [State list lenghts](#state-list-lenghts) - [Rewards and penalties](#rewards-and-penalties) - [Max operations per block](#max-operations-per-block) - [Incentivization weights](#incentivization-weights) - [Execution](#execution) - [Containers](#containers) - [New containers](#new-containers) + - [`PendingBalanceDeposit`](#pendingbalancedeposit) + - [`PartialWithdrawal`](#partialwithdrawal) + - [`ExecutionLayerWithdrawalRequest`](#executionlayerwithdrawalrequest) - [`PayloadAttestationData`](#payloadattestationdata) - [`PayloadAttestation`](#payloadattestation) - [`PayloadAttestationMessage`](#payloadattestationmessage) - [`IndexedPayloadAttestation`](#indexedpayloadattestation) - - [`SignedExecutionPayloadHeader`](#signedexecutionpayloadheader) - [`ExecutionPayloadHeaderEnvelope`](#executionpayloadheaderenvelope) - [`SignedExecutionPayloadHeaderEnvelope`](#signedexecutionpayloadheaderenvelope) - [`ExecutionPayloadEnvelope`](#executionpayloadenvelope) @@ -36,6 +40,7 @@ - [`SignedInclusionListSummary`](#signedinclusionlistsummary) - [`InclusionList`](#inclusionlist) - [Modified containers](#modified-containers) + - [`Validator`](#validator) - [`BeaconBlockBody`](#beaconblockbody) - [`ExecutionPayload`](#executionpayload) - [`ExecutionPayloadHeader`](#executionpayloadheader) @@ -45,23 +50,51 @@ - [`bit_floor`](#bit_floor) - [Predicates](#predicates) - [`is_builder`](#is_builder) + - [`is_eligible_for_activation_queue`](#is_eligible_for_activation_queue) + - [`is_slashed_proposer`](#is_slashed_proposer) + - [`is_slashed_attester`](#is_slashed_attester) + - [Modified `is_slashable_validator`](#modified-is_slashable_validator) + - [Modified `is_fully_withdrawable_validator`](#modified-is_fully_withdrawable_validator) + - [`is_partially_withdrawable_validator`](#is_partially_withdrawable_validator) - [`is_valid_indexed_payload_attestation`](#is_valid_indexed_payload_attestation) - [Beacon State accessors](#beacon-state-accessors) + - [Modified `get_eligible_validator_indices`](#modified-get_eligible_validator_indices) - [`get_ptc`](#get_ptc) - [`get_payload_attesting_indices`](#get_payload_attesting_indices) - [`get_indexed_payload_attestation`](#get_indexed_payload_attestation) + - [`get_validator_excess_balance`](#get_validator_excess_balance) + - [Modified `get_validator_churn_limit`](#modified-get_validator_churn_limit) + - [Modified `get_expected_withdrawals`](#modified-get_expected_withdrawals) + - [Beacon state mutators](#beacon-state-mutators) + - [`compute_exit_epoch_and_update_churn`](#compute_exit_epoch_and_update_churn) + - [Modified `initiate_validator_exit`](#modified-initiate_validator_exit) + - [Modified `slash_validator`](#modified-slash_validator) +- [Genesis](#genesis) + - [Modified `initialize_beacon_statre_from_eth1`](#modified--initialize_beacon_statre_from_eth1) - [Beacon chain state transition function](#beacon-chain-state-transition-function) - [Epoch processing](#epoch-processing) - [Modified `process_epoch`](#modified-process_epoch) + - [Helper functions](#helper-functions-1) + - [Modified `process_registry_updates`](#modified-process_registry_updates) + - [`process_pending_balance_deposits`](#process_pending_balance_deposits) + - [Modified `process_effective_balance_updates`](#modified-process_effective_balance_updates) + - [Modified `process_slashings`](#modified-process_slashings) + - [Modified `get_unslashed_attesting_indices`](#modified-get_unslashed_attesting_indices) - [Execution engine](#execution-engine) - [Request data](#request-data) - [New `NewInclusionListRequest`](#new-newinclusionlistrequest) - [Engine APIs](#engine-apis) - [New `notify_new_inclusion_list`](#new-notify_new_inclusion_list) - [Block processing](#block-processing) + - [Modified `process_block_header`](#modified-process_block_header) - [Modified `process_operations`](#modified-process_operations) - - [Modified `process_attestation`](#modified-process_attestation) + - [Modified Proposer slashings](#modified-proposer-slashings) + - [Modified Attester slashings](#modified-attester-slashings) + - [Modified `process_attestation`](#modified-process_attestation) + - [Modified `get_validator_from_deposit`](#modified-get_validator_from_deposit) + - [Modified `apply_deposit`](#modified-apply_deposit) - [Payload Attestations](#payload-attestations) + - [Execution Layer Withdraw Requests](#execution-layer-withdraw-requests) - [Modified `process_withdrawals`](#modified-process_withdrawals) - [New `verify_execution_payload_header_envelope_signature`](#new-verify_execution_payload_header_envelope_signature) - [New `process_execution_payload_header`](#new-process_execution_payload_header) diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 743165325d..cb885ae438 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -11,6 +11,8 @@ - [PTC Rewards](#ptc-rewards) - [PTC Attestations](#ptc-attestations) - [Forkchoice changes](#forkchoice-changes) + - [Equivocations](#equivocations) + - [Increased Max EB](#increased-max-eb) diff --git a/specs/_features/epbs/weak-subjectivity.md b/specs/_features/epbs/weak-subjectivity.md index 6f79bfbfcd..19bd08068b 100644 --- a/specs/_features/epbs/weak-subjectivity.md +++ b/specs/_features/epbs/weak-subjectivity.md @@ -6,6 +6,10 @@ +- [Weak Subjectivity Period](#weak-subjectivity-period) + - [Calculating the Weak Subjectivity Period](#calculating-the-weak-subjectivity-period) + - [Modified `compute_weak_subjectivity_period`](#modified-compute_weak_subjectivity_period) + ## Weak Subjectivity Period From a6c55576de059a1b2cae69848dee827f6e26e72d Mon Sep 17 00:00:00 2001 From: Potuz Date: Fri, 8 Sep 2023 14:34:36 -0300 Subject: [PATCH 42/57] add slot to IL --- specs/_features/epbs/beacon-chain.md | 1 + 1 file changed, 1 insertion(+) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 8dbe69a059..20b6ec4b4a 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -333,6 +333,7 @@ class SignedInclusionListSummary(Container): ```python class InclusionList(Container) summary: SignedInclusionListSummary + slot: Slot transactions: List[Transaction, MAX_TRANSACTIONS_PER_INCLUSION_LIST] ``` From 86e0e7dfd00bec2133ab4ca1b52754132511b04c Mon Sep 17 00:00:00 2001 From: Potuz Date: Mon, 11 Sep 2023 08:17:16 -0300 Subject: [PATCH 43/57] use head_root in get_head --- specs/_features/epbs/fork-choice.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index ba0d6ad93a..788e759e15 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -251,7 +251,8 @@ def get_head(store: Store) -> tuple[Root, bool]: # Ties broken by favoring block with lexicographically higher root # Ties then broken by favoring full blocks # TODO: Can (root, full), (root, empty) have the same weight? - head = max(children, key=lambda (root, present): (get_weight(store, root, present), root, present)) + head_root = max(children, key=lambda (root, present): (get_weight(store, root, present), root, present)) + head_full = is_payload_present(store, head_root) ``` ## Updated fork-choice handlers From feb30e645fefab65fe3d246dd79cbedb43b54d56 Mon Sep 17 00:00:00 2001 From: Potuz Date: Mon, 11 Sep 2023 08:27:49 -0300 Subject: [PATCH 44/57] design notes --- specs/_features/epbs/design.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index cb885ae438..098bb9a04b 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -59,6 +59,7 @@ So when importing the CL block for slot N, we process the expected withdrawals a - PTC members are obtained as the first members from each beacon slot committee that are not builders. - attesters are rewarded as a full attestation when they get the right payload presence: that is, if they vote for full (resp. empty) and the payload is included (resp. not included) then they get their participation bits (target, source and head timely) set. Otherwise they get a penalty as a missed attestation. - Attestations to the CL block from these members are just ignored. +- The proposer for slot N+1 must include PTC attestations for slot N. There is no rewards (and therefore no incentive) for the proposer to include attestations that voted incorrectly, perhaps we can simply accept 1 PTC attestation per block instead of the current two. ## PTC Attestations @@ -120,6 +121,9 @@ F ~~~ E ``` In this case all the attesters for `N+1` will be counted depending on the PTC members that voted for `(N+1, Full)`. Assuming honest PTC members, they would have voted for `N` during `N+1` so any CL attesters for `N+1` would be voting for `N+1, Empty` thus only counting for the head in `(N+3, Full)`. +### Checkpoint states +There is no current change in `store.checkpoint_states[root]`. In principle the "checkpoint state" should correspond to either the checkpoint block being full or empty. However, payload status does not change any consensus value for the state at the given time, so it does not matter if we continue using `store.block_states` which corresponds to the "empty" case. + ## Equivocations There is no need to do anything about proposer equivocations. Builders should reveal their block anyway. From d7a814e54c369aab8d3f118e45956459886c1c0f Mon Sep 17 00:00:00 2001 From: Potuz Date: Mon, 11 Sep 2023 08:33:20 -0300 Subject: [PATCH 45/57] onboard builders --- specs/_features/epbs/design.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 098bb9a04b..f5eb3a7d5b 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -37,7 +37,7 @@ ePBS introduces forward inclusion lists for proposers to guarantee censor resist There is a new entity `Builder` that is a glorified validator (they are simply validators with a different withdrawal prefix `0x0b`) required to have a higher stake and required to sign when producing execution payloads. - Builders are also validators (otherwise their staked capital depreciates). -- We onboard builders by simply turning validators into builders if they achieve the necessary minimum balance (this way we avoid two forks to onboard builders and keep the same deposit flow, avoid builders to skip the entry churn), we change their withdrawal prefix to be distinguished from normal validators. +- There is nothing to be done to onboard builders as we can simply accept validators with the right `0x0b` withdrawal prefix before the fork. They will be builders automatically. We could however onboard builders by simply turning validators into builders if they achieve the necessary minimum balance and change their withdrawal prefix to be distinguished from normal validators at the fork. - We need to include several changes from the [MaxEB PR](https://github.com/michaelneuder/consensus-specs/pull/3) in order to account with builders having an increased balance that would otherwise depreciate. ## Builder Payments From 85f9f269703a7d6379e7bf3b7bd1c9297f7153e0 Mon Sep 17 00:00:00 2001 From: Potuz Date: Mon, 11 Sep 2023 10:26:46 -0300 Subject: [PATCH 46/57] block-slot is missing --- specs/_features/epbs/design.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index f5eb3a7d5b..356dd57cd1 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -124,6 +124,10 @@ In this case all the attesters for `N+1` will be counted depending on the PTC me ### Checkpoint states There is no current change in `store.checkpoint_states[root]`. In principle the "checkpoint state" should correspond to either the checkpoint block being full or empty. However, payload status does not change any consensus value for the state at the given time, so it does not matter if we continue using `store.block_states` which corresponds to the "empty" case. +### Block slot + +Currently there is no complete implementation of (block, slot) vote: colluding proposers can in principle reveal a late block and base the next block on top of it. TODO: Fix this + ## Equivocations There is no need to do anything about proposer equivocations. Builders should reveal their block anyway. From 178fb1483bb4a796186affc69066fc1907e5fd2e Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 12 Sep 2023 15:03:46 -0300 Subject: [PATCH 47/57] block slot --- specs/_features/epbs/design.md | 49 ++++++++++++++++++++++- specs/_features/epbs/fork-choice.md | 61 +++++++++++++++++++++++------ 2 files changed, 98 insertions(+), 12 deletions(-) diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 356dd57cd1..8b97fb609b 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -11,6 +11,8 @@ - [PTC Rewards](#ptc-rewards) - [PTC Attestations](#ptc-attestations) - [Forkchoice changes](#forkchoice-changes) + - [Checkpoint states](#checkpoint-states) + - [Block slot](#block-slot) - [Equivocations](#equivocations) - [Increased Max EB](#increased-max-eb) @@ -126,7 +128,29 @@ There is no current change in `store.checkpoint_states[root]`. In principle the ### Block slot -Currently there is no complete implementation of (block, slot) vote: colluding proposers can in principle reveal a late block and base the next block on top of it. TODO: Fix this +Honest validators that vote for a parent block when a block is late, are contributing for this parent block support and are explicitly attesting that the current block is not present. This is taken into account in the new computation of `get_head`. Consider the following situation +```mermaid +graph RL +A[N-1, Full] +B[N, Full] --> A +``` +The block `N` has arrived late and the whole committee sees `A` as head and vote for `N-1`. At the start of `N+1` a call to `get_head` will return `N-1` as head and thus if the proposer of `N+1` is honest it will base its block on `N-1`. Suppose however that the proposer bases his block on top of `N`. Then we see +```mermaid +graph RL +A[N-1, Full] +B[N, Full] --> A +C[N+1, Full] --> B +``` +This block was timely so it gets proposer Boost. The real DAG is +```mermaid +graph RL +A[N-1, Full] +B[N, Full] --> A +C[N+1, Full] --> B +D[N-1, Full] --> A +E[N-1, Full] --> D +``` +And honest validators should still see `N-1` as head. The reason being that at the attestation deadline on `N+1` validators have seen block `N+1` appear, this block is valid and has 40% of a committee vote because of proposer boost. However, the branch for `N-1` has the full committee from `N` that has voted for it, and thus honest validators vote for `N-1` as valid. ## Equivocations @@ -138,5 +162,28 @@ There is no need to do anything about proposer equivocations. Builders should re - So since the builder cannot be unbundled, then he either doesn't pay if the block is not included, or pays and its included. - The splitting grief, that is, the proposer's block has about 50% of the vote at 8 seconds, remains. +A little care has to be taken in the case of colluding proposers for `N` and `N+1`. Consider the example of the [previous section](#block-slot). The malicious proposer of `N` sends an early block to the builder and an equivocation after it has seen the payload. No honest validators will have voted for this equivocation. Suppose $\beta$ is the malicious stake. We have $1 - \beta$ for that votes for the early `N` as head and $\beta$ that will vote for the lately revealed block. Assuming $\beta < 0.5$ we have that the PTC committee will declare the equivocation as empty. The malicious proposer of `N+1` proposes based on the equivocating block `N` including some unbundled transactions. Because of the PTC vote, even the $\beta$ attestations for the equivocating block `N` will not count for `N+1` since it builds on *full* instead of empty. The weight of `N+1` is only given by proposer boost. The weight of the early `N` will be $1 - \beta$ thus beating the malicious `N+1` if $\beta < 0.6$ and thus honest validators will vote for the early `N` that included the builders' payload. However, the early block itself may cause a split view, in this case some attesters may have voted for `N-1` as head! in this situation we would have a DAG like this (we are not considering irrelevant branches) +```mermaid +graph RL +A[N-1, Full] +B[N, Full] --> A +H[N, Full] --> B +F[N', Full] --> A +I[N', Empty] --> A +C[N+1, Full] --> F +D[N-1, Full] --> A +E[N-1, Full] --> D +``` + +When recursing from the children of `N-1` the weights for the three children are as follows (when computing after `N+1` has been revealed and before validators for `N+1` attested) +- (N, Full) has gotten some vote $\gamma \leq 1 - \beta$. +- (N', Full) has zero weight. This is an important point. Proposer boost does not apply to it because even though $N+1$ will get proposer boost, it is based on the wrong `PTC` vote, and thus it does not count towards this node's weight. +- (N', Empty) has $\beta$ maximum. +- (N-1, Full) has $1 - \beta - \gamma$. + +Thus, supposing $\gamma < \beta$ we have that $1 - \beta - \gamma > 1 - 2 \beta > \beta$ as long as $\beta < 1/3$. Thus we are protected from these kinds of attacks from attackers up to 33%. + +Note however that if we were to apply proposer boost to `(N', Full)` then we see that there's a split now between the three possible heads. `N'` has proposer boost giving it $0.4$ so if $\gamma = 0.39$ we get that with $1 - \beta - \gamma < 0.4$ whenever $\beta \geq 0.2$. Thus a 20% attacker that can also split the network, would be able to carry this attack with two consecutive blocks. + ## Increased Max EB This PR includes the changes from [this PR](https://github.com/michaelneuder/consensus-specs/pull/3). In particular it includes execution layer triggerable withdrawals. diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 788e759e15..3aa3ad5b8a 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -8,6 +8,8 @@ - [Introduction](#introduction) - [Constant](#constant) - [Helpers](#helpers) + - [Modified `LatestMessage`](#modified-latestmessage) + - [Modified `update_latest_messages`](#modified-update_latest_messages) - [Modified `Store`](#modified-store) - [`verify_inclusion_list`](#verify_inclusion_list) - [`is_inclusion_list_available`](#is_inclusion_list_available) @@ -39,6 +41,29 @@ This is the modification of the fork choice accompanying the ePBS upgrade. ## Helpers +### Modified `LatestMessage` +**Note:** The class is modified to keep track of the slot instead of the epoch + +```python +@dataclass(eq=True, frozen=True) +class LatestMessage(object): + slot: Slot + root: Root +``` + +### Modified `update_latest_messages` +**Note:** the function `update_latest_messages` is updated to use the attestation slot instead of target. Notice that this function is only called on validated attestations and validators cannot attest twice in the same epoch without equivocating. Notice also that target epoch number and slot number are validated on `validate_on_attestation`. + +```python +def update_latest_messages(store: Store, attesting_indices: Sequence[ValidatorIndex], attestation: Attestation) -> None: + slot = attestation.data.slot + beacon_block_root = attestation.data.beacon_block_root + non_equivocating_attesting_indices = [i for i in attesting_indices if i not in store.equivocating_indices] + for i in non_equivocating_attesting_indices: + if i not in store.latest_messages or slot > store.latest_messages[i].slot: + store.latest_messages[i] = LatestMessage(slot=slot, root=beacon_block_root) +``` + ### Modified `Store` **Note:** `Store` is modified to track the intermediate states of "empty" consensus blocks, that is, those consensus blocks for which the corresponding execution payload has not been revealed or has not been included on chain. @@ -189,21 +214,27 @@ def get_checkpoint_block(store: Store, root: Root, epoch: Epoch) -> Root: ### `is_supporting_vote` ```python -def is_supporting_vote(store: Store, root: Root, is_payload_present: bool, message_root: Root) -> bool: +def is_supporting_vote(store: Store, root: Root, slot: Slot, is_payload_present: bool, message: LatestMessage) -> bool: """ returns whether a vote for ``message_root`` supports the chain containing the beacon block ``root`` with the - payload contents indicated by ``is_payload_present``. + payload contents indicated by ``is_payload_present`` as head during slot ``slot``. """ - (ancestor_root, is_ancestor_full) = get_ancestor(store, message_root, store.blocks[root].slot) + if root == message_root: + # an attestation for a given root always counts for that root regardless if full or empty + return slot <= message.slot + message_block = store.blocks[message_root] + if slot > message_block.slot: + return False + (ancestor_root, is_ancestor_full) = get_ancestor(store, message_root, slot) return (root == ancestor_root) and (is_payload_preset == is_ancestor_full) ``` ### Modified `get_weight` -**Note:** `get_weight` is modified to only count votes for descending chains that support the status of a pair `Root, bool`, where the `bool` indicates if the block was full or not. +**Note:** `get_weight` is modified to only count votes for descending chains that support the status of a triple `Root, Slot, bool`, where the `bool` indicates if the block was full or not. ```python -def get_weight(store: Store, root: Root, is_payload_present: bool) -> Gwei: +def get_weight(store: Store, root: Root, slot: Slot, is_payload_present: bool) -> Gwei: state = store.checkpoint_states[store.justified_checkpoint] unslashed_and_active_indices = [ i for i in get_active_validator_indices(state, get_current_epoch(state)) @@ -213,7 +244,7 @@ def get_weight(store: Store, root: Root, is_payload_present: bool) -> Gwei: state.validators[i].effective_balance for i in unslashed_and_active_indices if (i in store.latest_messages and i not in store.equivocating_indices - and is_supporting_vote(store, root, is_payload_present, store.latest_messages[i].root)) + and is_supporting_vote(store, root, slot, is_payload_present, store.latest_messages[i])) )) if store.proposer_boost_root == Root(): # Return only attestation score if ``proposer_boost_root`` is not set @@ -239,19 +270,27 @@ def get_head(store: Store) -> tuple[Root, bool]: blocks = get_filtered_block_tree(store) # Execute the LMD-GHOST fork choice head_root = store.justified_checkpoint.root + head_block = store.blocks[head_root] + head_slot = head_block.slot head_full = is_payload_present(store, head_root) while True: children = [ - (root, present) for root in blocks.keys() - if blocks[root].parent_root == head_root for present in (True, False) + (root, block.slot, present) for (root, block) in blocks.items() + if block.parent_root == head_root for present in (True, False) ] if len(children) == 0: return (head_root, head_full) + # if we have children we consider the current head advanced as a possible head + children += [(head_root, head_slot + 1, head_full)] # Sort by latest attesting balance with ties broken lexicographically - # Ties broken by favoring block with lexicographically higher root - # Ties then broken by favoring full blocks + # Ties broken by favoring full blocks + # Ties broken then by favoring higher slot numbers + # Ties then broken by favoring block with lexicographically higher root # TODO: Can (root, full), (root, empty) have the same weight? - head_root = max(children, key=lambda (root, present): (get_weight(store, root, present), root, present)) + child_root = max(children, key=lambda (root, slot, present): (get_weight(store, root, slot, present), present, slot, root)) + if child_root == head_root: + return (head_root, head_full) + head_root = child_root head_full = is_payload_present(store, head_root) ``` From 8451f12cb81e8587a36f3e42933ad030046a2db5 Mon Sep 17 00:00:00 2001 From: Potuz Date: Thu, 14 Sep 2023 13:04:27 -0300 Subject: [PATCH 48/57] typos --- specs/_features/epbs/beacon-chain.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 20b6ec4b4a..3d6c04a17e 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -487,7 +487,7 @@ class BeaconState(Container): # Deep history valid from Capella onwards historical_summaries: List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT] # PBS - signed_execution_payload_header_envelope: SignedExecutionPayloadHeaderEnvelop # [New in ePBS] + signed_execution_payload_header_envelope: SignedExecutionPayloadHeaderEnvelope # [New in ePBS] last_withdrawals_root: Root # [New in ePBS] deposit_balance_to_consume: Gwei # [New in ePBS] exit_balance_to_consume: Gwei # [New in ePBS] @@ -1391,7 +1391,7 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti assert envelope.beacon_block_root == hash_tree_root(state.latest_block_header) # Verify consistency with the committed header hash = hash_tree_root(payload) - commited_envelope = state.signed_execution_payload_header_envelope.message + committed_envelope = state.signed_execution_payload_header_envelope.message previous_hash = hash_tree_root(committed_envelope.payload) assert hash == previous_hash # Verify consistency with the envelope @@ -1406,7 +1406,7 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti ) ) # Cache the execution payload header - state.latest_execution_payload_header = committed_envelope.payload + state.latest_execution_payload_header = committed_envelope.header # Verify the state root assert envelope.state_root == hash_tree_root(state) ``` From 998ca41f4e3e9c2562536d58ece894c971ee39a4 Mon Sep 17 00:00:00 2001 From: Potuz Date: Mon, 18 Sep 2023 07:55:55 -0300 Subject: [PATCH 49/57] typos Terence found with chatgpt --- specs/_features/epbs/fork-choice.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 3aa3ad5b8a..598329ae6f 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -141,7 +141,7 @@ def is_inclusion_list_available(state: BeaconState, block: BeaconBlock) -> bool: """ # Verify that the list is empty if the parent consensus block did not contain a payload if state.signed_execution_payload_header_envelope.message.header != state.latest_execution_payload_header: - return true + return True # verify the inclusion list inclusion_list = retrieve_inclusion_list(block.slot, block.proposer_index) @@ -159,7 +159,7 @@ def notify_ptc_messages(store: Store, state: BeaconState, payload_attestations: indexed_payload_attestation = get_indexed_payload_attestation(state, state.slot - 1, payload_attestation) for idx in indexed_payload_attestation.attesting_indices: store.on_payload_attestation_message(PayloadAttestationMessage(validator_index=idx, - data=payload_attestation.data, signature: BLSSignature(), is_from_block=true) + data=payload_attestation.data, signature= BLSSignature(), is_from_block=true) ``` ### `is_payload_present` @@ -167,12 +167,12 @@ def notify_ptc_messages(store: Store, state: BeaconState, payload_attestations: ```python def is_payload_present(store: Store, beacon_block_root: Root) -> bool: """ - return wether the execution payload for the beacon block with root ``beacon_block_root`` was voted as present + return whether the execution payload for the beacon block with root ``beacon_block_root`` was voted as present by the PTC """ # The beacon block root must be known assert beacon_block_root in store.ptc_vote - return ptc_vote[beacon_block_root].count(True) > PAYLOAD_TIMELY_THRESHOLD + return store.ptc_vote[beacon_block_root].count(True) > PAYLOAD_TIMELY_THRESHOLD ``` ### Modified `get_ancestor` @@ -396,7 +396,7 @@ def on_excecution_payload(store: Store, signed_envelope: SignedExecutionPayloadE process_execution_payload(state, signed_envelope, EXECUTION_ENGINE) #Add new state for this payload to the store - store.execution_payload_states[beacon_block_root] = state + store.execution_payload_states[envelope.beacon_block_root] = state ``` ### `on_payload_attestation_message` From 0431f0fa8b9b34bb7eb1c5a03ca9db5a42f23202 Mon Sep 17 00:00:00 2001 From: Potuz Date: Wed, 20 Sep 2023 16:55:45 -0300 Subject: [PATCH 50/57] add helper to get payload hash --- specs/_features/epbs/fork-choice.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 598329ae6f..d77a9fab60 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -20,6 +20,7 @@ - [`is_supporting_vote`](#is_supporting_vote) - [Modified `get_weight`](#modified-get_weight) - [Modified `get_head`](#modified-get_head) + - [New `get_block_hash`](#new-get_block_hash) - [Updated fork-choice handlers](#updated-fork-choice-handlers) - [`on_block`](#on_block) - [New fork-choice handlers](#new-fork-choice-handlers) @@ -294,6 +295,20 @@ def get_head(store: Store) -> tuple[Root, bool]: head_full = is_payload_present(store, head_root) ``` +### New `get_block_hash` + +```python +def get_blockhash(store: Store, root: Root) -> Hash32: + """ + returns the blockHash of the latest execution payload in the chain containing the + beacon block with root ``root`` + """ + # The block is known + if is_payload_present(store, root): + return hash(store.execution_payload_states[root].latest_block_header) + return hash(store.block__states[root].latest_block_header) +``` + ## Updated fork-choice handlers ### `on_block` From d4f4b06f3e453e9b74204d0f95fe20a9fe6a31f4 Mon Sep 17 00:00:00 2001 From: Potuz Date: Tue, 10 Oct 2023 20:45:59 -0300 Subject: [PATCH 51/57] minimal churn for transfers --- specs/_features/epbs/beacon-chain.md | 4 ++-- specs/_features/epbs/design.md | 4 ++++ 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 3d6c04a17e..94072e6083 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -1350,13 +1350,13 @@ def verify_execution_payload_header_envelope_signature(state: BeaconState, def process_execution_payload_header(state: BeaconState, block: BeaconBlock) -> None: signed_header_envelope = block.body.signed_execution_payload_header_envelope assert verify_execution_payload_header_envelope_signature(state, signed_header_envelope) - # Check that the builder has funds to cover the bid and transfer the funds + # Check that the builder has funds to cover the bid and schedule the funds for transfer envelope = signed_header_envelope.message builder_index = envelope.builder_index amount = envelope.value assert state.balances[builder_index] >= amount: decrease_balance(state, builder_index, amount) - increase_balance(state, block.proposer_index, amount) + state.pending_balance_deposits.append(PendingBalanceDeposit(index=block.proposer_index, amount=amount)) # Verify the withdrawals_root against the state cached ones assert header.withdrawals_root == state.last_withdrawals_root # Verify consistency of the parent hash with respect to the previous execution payload header diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 8b97fb609b..c60f29f26a 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -15,6 +15,7 @@ - [Block slot](#block-slot) - [Equivocations](#equivocations) - [Increased Max EB](#increased-max-eb) + - [Validator transfers](#validator-transfers) @@ -187,3 +188,6 @@ Note however that if we were to apply proposer boost to `(N', Full)` then we see ## Increased Max EB This PR includes the changes from [this PR](https://github.com/michaelneuder/consensus-specs/pull/3). In particular it includes execution layer triggerable withdrawals. + +## Validator transfers +One of the main problems of the current design is that a builder can transfer arbitrary amounts to proposers by simply paying a large bid. This is dangerous from a forkchoice perspective as it moves weights from one branch to another instantaneously, it may prevent a large penalty in case of slashing, etc. In order to partially mitigate this, we churn the transfer overloading the deposit system of Max EB, that is we append a `PendingBalanceDeposit` object to the beacon state. This churns the increase in the proposer's balance while it discounts immediately the balance of the builder. We may want to revisit this and add also an exit churn and even deal with equivocations on future iterations. From 874e4c714905958ec663e3b3b2b6d6881981eb2f Mon Sep 17 00:00:00 2001 From: Potuz Date: Wed, 11 Oct 2023 12:00:47 -0300 Subject: [PATCH 52/57] add mapping for exclusion --- specs/_features/epbs/beacon-chain.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 94072e6083..867a5e86c7 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -37,6 +37,7 @@ - [`SignedExecutionPayloadEnvelope`](#signedexecutionpayloadenvelope) - [`InclusionListSummaryEntry`](#inclusionlistsummaryentry) - [`InclusionListSummary`](#inclusionlistsummary) + - [`ExclusionListEntry`](#exclusionlistentry) - [`SignedInclusionListSummary`](#signedinclusionlistsummary) - [`InclusionList`](#inclusionlist) - [Modified containers](#modified-containers) @@ -320,6 +321,14 @@ class InclusionListSummary(Container) summary: List[InclusionListSummaryEntry, MAX_TRANSACTIONS_PER_INCLUSION_LIST] ``` +#### `ExclusionListEntry` + +```python +class ExclusionListEntry(Container): + block_index: uint64 + summary_index: uint64 +``` + #### `SignedInclusionListSummary` ```python @@ -408,7 +417,7 @@ class ExecutionPayload(Container): blob_gas_used: uint64 excess_blob_gas: uint64 inclusion_list_summary: SignedInclusionListSummary # [New in ePBS] - inclusion_list_exclusions: List[uint64, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] + inclusion_list_exclusions: List[ExclusionListEntry, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] ``` #### `ExecutionPayloadHeader` From a9d279321a25e9ad939b85f31a2019531029c568 Mon Sep 17 00:00:00 2001 From: Potuz Date: Wed, 11 Oct 2023 12:23:17 -0300 Subject: [PATCH 53/57] Revert "add mapping for exclusion" This reverts commit 874e4c714905958ec663e3b3b2b6d6881981eb2f. --- specs/_features/epbs/beacon-chain.md | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 867a5e86c7..94072e6083 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -37,7 +37,6 @@ - [`SignedExecutionPayloadEnvelope`](#signedexecutionpayloadenvelope) - [`InclusionListSummaryEntry`](#inclusionlistsummaryentry) - [`InclusionListSummary`](#inclusionlistsummary) - - [`ExclusionListEntry`](#exclusionlistentry) - [`SignedInclusionListSummary`](#signedinclusionlistsummary) - [`InclusionList`](#inclusionlist) - [Modified containers](#modified-containers) @@ -321,14 +320,6 @@ class InclusionListSummary(Container) summary: List[InclusionListSummaryEntry, MAX_TRANSACTIONS_PER_INCLUSION_LIST] ``` -#### `ExclusionListEntry` - -```python -class ExclusionListEntry(Container): - block_index: uint64 - summary_index: uint64 -``` - #### `SignedInclusionListSummary` ```python @@ -417,7 +408,7 @@ class ExecutionPayload(Container): blob_gas_used: uint64 excess_blob_gas: uint64 inclusion_list_summary: SignedInclusionListSummary # [New in ePBS] - inclusion_list_exclusions: List[ExclusionListEntry, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] + inclusion_list_exclusions: List[uint64, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] ``` #### `ExecutionPayloadHeader` From a908e5e9085665e17a3b6c2039f701e603153031 Mon Sep 17 00:00:00 2001 From: Potuz Date: Wed, 11 Oct 2023 16:10:51 -0300 Subject: [PATCH 54/57] fix design for exclusion list --- specs/_features/epbs/design.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index c60f29f26a..c1f3ae4025 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -27,9 +27,9 @@ ePBS introduces forward inclusion lists for proposers to guarantee censor resist - Proposer for slot N submits a signed block and in parallel broadcasts pairs of `summaries` and `transactions` to be included at the beginning of slot N+1. `transactions` are just list of transactions that this proposer wants included at the most at the beginning of N+1. `Summaries` are lists consisting on addresses sending those transactions and their gas limits. The summaries are signed, the transactions aren't. An honest proposer is allowed to send many of these pairs that aren't committed to its beacon block so no double proposing slashing is involved. - Validators for slot N will consider the block for validation only if they have seen at least one pair (summary, transactions). They will consider the block invalid if those transactions are not executable at the start of slot N and if they don't have at least 12.5% higher `maxFeePerGas` than the current slot's `maxFeePerGas`. -- The builder for slot N reveals its payload together with a signed summary of the proposer of slot N-1. The payload is considered only valid if the following applies - - Let k >= 0 be the minimum such that tx[0],...,tx[k-1], the first `k` transactions of the payload of slot N, satisfy some entry in the summary and `tx[k]` does not satisfy any entry. - - There exist transactions in the payload for N-1 that satisfy all the remaining entries in the summary. +- The builder for slot N reveals its payload together with a signed summary of the proposer of slot N-1. Along the summary, the builder includes a list of transactions indices (in strictly increasing order) of the previous payload of slot N-1, that satisfy some entry in the signed inclusion list summary. The payload is considered only valid if the following applies + - For each index `i` in the payload's `inclusion_list_exclusions`, check that the ith transaction `tx[i]` of the payload for `N-1` satisfies some transaction `T[i]` of the current inclusion list. Here `T[i]` is the first entry in the payload's inclusion list that is satisfied by `tx[i]`. This `T[i]` is removed from the inclusion list summary. + - The remaining transactions in the inclusion list summary, are all satisfied by the first transactions in the current payload, in increasing order, starting from the first transaction. - The payload is executable, that is, it's valid from the execution layer perspective. **Note:** in the event that the payload for the canonical block in slot N is not revealed, then the summaries and transactions list for slot N-1 remains valid, the honest proposer for slot N+1 is not allowed to submit a new IL and any such message will be ignored. The builder for N+1 still has to satisfy the summary of N-1. If there are k slots in a row that are missing payloads, the next full slot will still need to satisfy the inclusion list for N-1. From 7cb7de7f942b8076990548a67d0f428ef8417e19 Mon Sep 17 00:00:00 2001 From: Potuz Date: Fri, 13 Oct 2023 10:21:23 -0300 Subject: [PATCH 55/57] fix signature verification --- specs/_features/epbs/beacon-chain.md | 36 ++++++++++++++++++++++++---- specs/_features/epbs/fork-choice.md | 13 ++++------ 2 files changed, 36 insertions(+), 13 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 94072e6083..5d90a14fe3 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -26,7 +26,7 @@ - [New containers](#new-containers) - [`PendingBalanceDeposit`](#pendingbalancedeposit) - [`PartialWithdrawal`](#partialwithdrawal) - - [`ExecutionLayerWithdrawalRequest`](#executionlayerwithdrawalrequest) + - [`ExecutionLayerWithdrawRequest`](#executionlayerwithdrawrequest) - [`PayloadAttestationData`](#payloadattestationdata) - [`PayloadAttestation`](#payloadattestation) - [`PayloadAttestationMessage`](#payloadattestationmessage) @@ -99,6 +99,7 @@ - [New `verify_execution_payload_header_envelope_signature`](#new-verify_execution_payload_header_envelope_signature) - [New `process_execution_payload_header`](#new-process_execution_payload_header) - [New `verify_execution_payload_signature`](#new-verify_execution_payload_signature) + - [New `verify_inclusion_list_summary_signature`](#new-verify_inclusion_list_summary_signature) - [Modified `process_execution_payload`](#modified-process_execution_payload) @@ -223,10 +224,10 @@ class PartialWithdrawal(Container) withdrawable_epoch: Epoch ``` -#### `ExecutionLayerWithdrawalRequest` +#### `ExecutionLayerWithdrawRequest` ```python -class ExecutionLayerWithdrawalRequest(Container) +class ExecutionLayerWithdrawRequest(Container) source_address: ExecutionAddress validator_pubkey: BLSPubkey balance: Gwei @@ -293,6 +294,8 @@ class ExecutionPayloadEnvelope(Container): builder_index: ValidatorIndex beacon_block_root: Root blob_kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK] + inclusion_list_proposer_index: ValidatorIndex + inclusion_list_signature: BLSSignature state_root: Root ``` @@ -407,7 +410,7 @@ class ExecutionPayload(Container): withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] blob_gas_used: uint64 excess_blob_gas: uint64 - inclusion_list_summary: SignedInclusionListSummary # [New in ePBS] + inclusion_list_summary: List[InclusionListSummaryEntry, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] inclusion_list_exclusions: List[uint64, MAX_TRANSACTIONS_PER_INCLUSION_LIST] # [New in ePBS] ``` @@ -487,6 +490,7 @@ class BeaconState(Container): # Deep history valid from Capella onwards historical_summaries: List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT] # PBS + latest_execution_payload_proposer: ValidatorIndex # [New in ePBS] signed_execution_payload_header_envelope: SignedExecutionPayloadHeaderEnvelope # [New in ePBS] last_withdrawals_root: Root # [New in ePBS] deposit_balance_to_consume: Gwei # [New in ePBS] @@ -1378,6 +1382,17 @@ def verify_execution_envelope_signature(state: BeaconState, signed_envelope: Sig return bls.Verify(builder.pubkey, signing_root, signed_envelope.signature) ``` +#### New `verify_inclusion_list_summary_signature` + +```python +def verify_inclusion_list_summary_signature(state: BeaconState, signed_summary: SignedInclusionListSummary) -> bool: + # TODO: do we need a new domain? + summary = signed_summary.message + signing_root = compute_signing_root(summary, get_domain(state, DOMAIN_BEACON_PROPOSER)) + proposer = state.validators[message.proposer_index] + return bls.Verify(proposer.pubkey, signing_root, signed_summary.signature) +``` + #### Modified `process_execution_payload` *Note*: `process_execution_payload` is now an independent check in state transition. It is called when importing a signed execution payload proposed by the builder of the current slot. @@ -1387,6 +1402,16 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti assert verify_execution_envelope_signature(state, signed_envelope) envelope = signed_envelope.message payload = envelope.payload + # Verify inclusion list proposer + proposer_index = envelope.inclusion_list_proposer_index + assert proposer_index == state.latest_execution_payload_proposer + # Verify inclusion list summary signature + signed_summary = SignedInclusionListSummary( + message=InclusionListSummary( + proposer_index=proposer_index + summary=payload.inclusion_list_summary) + signature=envelope.inclusion_list_signature) + assert verify_inclusion_list_summary_signature(state, signed_summary) # Verify consistency with the beacon block assert envelope.beacon_block_root == hash_tree_root(state.latest_block_header) # Verify consistency with the committed header @@ -1405,8 +1430,9 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti parent_beacon_block_root=state.latest_block_header.parent_root, ) ) - # Cache the execution payload header + # Cache the execution payload header and proposer state.latest_execution_payload_header = committed_envelope.header + state.latest_execution_payload_proposer = proposer_index # Verify the state root assert envelope.state_root == hash_tree_root(state) ``` diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index d77a9fab60..80eeff1128 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -102,10 +102,7 @@ def verify_inclusion_list(state: BeaconState, block: BeaconBlock, inclusion_list assert block.proposer_index == proposer_index # Check that the signature is correct - # TODO: do we need a new domain? - signing_root = compute_signing_root(signed_summary.message, get_domain(state, DOMAIN_BEACON_PROPOSER)) - proposer = state.validators[proposer_index] - assert bls.Verify(proposer.pubkey, signing_root, signed_summary.signature) + assert verify_inclusion_list_summary_signature(state, signed_summary) # TODO: These checks will also be performed by the EL surely so we can probably remove them from here. # Check the summary and transaction list lengths @@ -217,16 +214,16 @@ def get_checkpoint_block(store: Store, root: Root, epoch: Epoch) -> Root: ```python def is_supporting_vote(store: Store, root: Root, slot: Slot, is_payload_present: bool, message: LatestMessage) -> bool: """ - returns whether a vote for ``message_root`` supports the chain containing the beacon block ``root`` with the + returns whether a vote for ``message.root`` supports the chain containing the beacon block ``root`` with the payload contents indicated by ``is_payload_present`` as head during slot ``slot``. """ - if root == message_root: + if root == message.root: # an attestation for a given root always counts for that root regardless if full or empty return slot <= message.slot - message_block = store.blocks[message_root] + message_block = store.blocks[message.root] if slot > message_block.slot: return False - (ancestor_root, is_ancestor_full) = get_ancestor(store, message_root, slot) + (ancestor_root, is_ancestor_full) = get_ancestor(store, message.root, slot) return (root == ancestor_root) and (is_payload_preset == is_ancestor_full) ``` From 0bc82df7bad60b58b4c783d50df9ec4df5df358d Mon Sep 17 00:00:00 2001 From: Potuz Date: Fri, 13 Oct 2023 11:17:25 -0300 Subject: [PATCH 56/57] fix il proposers --- specs/_features/epbs/beacon-chain.md | 18 +++++++++++++++--- specs/_features/epbs/design.md | 6 ++++++ specs/_features/epbs/fork-choice.md | 4 ++-- 3 files changed, 23 insertions(+), 5 deletions(-) diff --git a/specs/_features/epbs/beacon-chain.md b/specs/_features/epbs/beacon-chain.md index 5d90a14fe3..4855997599 100644 --- a/specs/_features/epbs/beacon-chain.md +++ b/specs/_features/epbs/beacon-chain.md @@ -57,6 +57,7 @@ - [Modified `is_fully_withdrawable_validator`](#modified-is_fully_withdrawable_validator) - [`is_partially_withdrawable_validator`](#is_partially_withdrawable_validator) - [`is_valid_indexed_payload_attestation`](#is_valid_indexed_payload_attestation) + - [`is_parent_block_full`](#is_parent_block_full) - [Beacon State accessors](#beacon-state-accessors) - [Modified `get_eligible_validator_indices`](#modified-get_eligible_validator_indices) - [`get_ptc`](#get_ptc) @@ -490,7 +491,8 @@ class BeaconState(Container): # Deep history valid from Capella onwards historical_summaries: List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT] # PBS - latest_execution_payload_proposer: ValidatorIndex # [New in ePBS] + previous_inclusion_list_proposer: ValidatorIndex # [New in ePBS] + latest_inclusion_list_proposer: ValidatorIndex # [New in ePBS] signed_execution_payload_header_envelope: SignedExecutionPayloadHeaderEnvelope # [New in ePBS] last_withdrawals_root: Root # [New in ePBS] deposit_balance_to_consume: Gwei # [New in ePBS] @@ -617,6 +619,13 @@ def is_valid_indexed_payload_attestation(state: BeaconState, indexed_payload_att return bls.FastAggregateVerify(pubkeys, signing_root, indexed_payload_attestation.signature) ``` +#### `is_parent_block_full` + +```python +def is_parent_block_full(state: BeaconState) -> bool: + return state.signed_execution_payload_header_envelope.message.header == state.latest_execution_payload_header +``` + ### Beacon State accessors #### Modified `get_eligible_validator_indices` @@ -1369,6 +1378,9 @@ def process_execution_payload_header(state: BeaconState, block: BeaconBlock) -> assert header.prev_randao == get_randao_mix(state, get_current_epoch(state)) # Verify timestamp assert header.timestamp == compute_timestamp_at_slot(state, state.slot) + # Cache the inclusion list proposer if the parent block was full + if is_parent_block_full(state): + state.latest_inclusion_list_proposer = block.proposer_index # Cache execution payload header envelope state.signed_execution_payload_header_envelope = signed_header_envelope ``` @@ -1404,7 +1416,7 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti payload = envelope.payload # Verify inclusion list proposer proposer_index = envelope.inclusion_list_proposer_index - assert proposer_index == state.latest_execution_payload_proposer + assert proposer_index == state.previous_inclusion_list_proposer # Verify inclusion list summary signature signed_summary = SignedInclusionListSummary( message=InclusionListSummary( @@ -1432,7 +1444,7 @@ def process_execution_payload(state: BeaconState, signed_envelope: SignedExecuti ) # Cache the execution payload header and proposer state.latest_execution_payload_header = committed_envelope.header - state.latest_execution_payload_proposer = proposer_index + state.previous_inclusion_list_proposer = state.latest_inclusion_list_proposer # Verify the state root assert envelope.state_root == hash_tree_root(state) ``` diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index c1f3ae4025..6eecaeee4d 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -4,6 +4,7 @@ - [ePBS design notes](#epbs-design-notes) - [Inclusion lists](#inclusion-lists) + - [Liveness](#liveness) - [Builders](#builders) - [Builder Payments](#builder-payments) - [Withdrawals](#withdrawals) @@ -34,6 +35,11 @@ ePBS introduces forward inclusion lists for proposers to guarantee censor resist **Note:** in the event that the payload for the canonical block in slot N is not revealed, then the summaries and transactions list for slot N-1 remains valid, the honest proposer for slot N+1 is not allowed to submit a new IL and any such message will be ignored. The builder for N+1 still has to satisfy the summary of N-1. If there are k slots in a row that are missing payloads, the next full slot will still need to satisfy the inclusion list for N-1. +### Liveness + +In the usual case of LMD+Ghost we have a proof of the *plausible liveness* theorem, that is that supermajority links can always be added to produce new finalized checkpoints provided there exist children extending the finalized chain. Here we prove that the next builder can always produce a valid payload, in particular, a payload that can satisfy the pending inclusion list. + +Let N be the last slot which contained a full execution payload, and let $N+1,..., N+k$, $k \geq 1$ be slots in the canonical chain, descending from $N$ that were either skipped or are *empty*, that is, the corresponding execution payload has not been revealed or hasn't been included. The consensus block for $N+k$ has been proposed and it is the canonical head. The builder for $N+k$ has to fulfill the inclusion list proposed by $N$. When importing the block $N$, validators have attested for the ## Builders diff --git a/specs/_features/epbs/fork-choice.md b/specs/_features/epbs/fork-choice.md index 80eeff1128..519613f5a7 100644 --- a/specs/_features/epbs/fork-choice.md +++ b/specs/_features/epbs/fork-choice.md @@ -137,8 +137,8 @@ def is_inclusion_list_available(state: BeaconState, block: BeaconBlock) -> bool: Note: the p2p network does not guarantee sidecar retrieval outside of `MIN_SLOTS_FOR_INCLUSION_LISTS_REQUESTS` """ - # Verify that the list is empty if the parent consensus block did not contain a payload - if state.signed_execution_payload_header_envelope.message.header != state.latest_execution_payload_header: + # Ignore the list if the parent consensus block did not contain a payload + if !is_parent_block_full(state): return True # verify the inclusion list From f31929acd92efea3d3ac92ce7a8c6ea05bb2472c Mon Sep 17 00:00:00 2001 From: Potuz Date: Fri, 13 Oct 2023 11:29:36 -0300 Subject: [PATCH 57/57] add censoring description --- specs/_features/epbs/design.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/specs/_features/epbs/design.md b/specs/_features/epbs/design.md index 6eecaeee4d..4609400011 100644 --- a/specs/_features/epbs/design.md +++ b/specs/_features/epbs/design.md @@ -5,6 +5,7 @@ - [ePBS design notes](#epbs-design-notes) - [Inclusion lists](#inclusion-lists) - [Liveness](#liveness) + - [Censoring](#censoring) - [Builders](#builders) - [Builder Payments](#builder-payments) - [Withdrawals](#withdrawals) @@ -39,7 +40,11 @@ ePBS introduces forward inclusion lists for proposers to guarantee censor resist In the usual case of LMD+Ghost we have a proof of the *plausible liveness* theorem, that is that supermajority links can always be added to produce new finalized checkpoints provided there exist children extending the finalized chain. Here we prove that the next builder can always produce a valid payload, in particular, a payload that can satisfy the pending inclusion list. -Let N be the last slot which contained a full execution payload, and let $N+1,..., N+k$, $k \geq 1$ be slots in the canonical chain, descending from $N$ that were either skipped or are *empty*, that is, the corresponding execution payload has not been revealed or hasn't been included. The consensus block for $N+k$ has been proposed and it is the canonical head. The builder for $N+k$ has to fulfill the inclusion list proposed by $N$. When importing the block $N$, validators have attested for the +Let N be the last slot which contained a full execution payload, and let $N+1,..., N+k$, $k \geq 1$ be slots in the canonical chain, descending from $N$ that were either skipped or are *empty*, that is, the corresponding execution payload has not been revealed or hasn't been included. The consensus block for $N+k$ has been proposed and it is the canonical head. The builder for $N+k$ has to fulfill the inclusion list proposed by $N$. When importing the block $N$, validators have attested for availability of at least one valid inclusion list. That is, those transactions would be executable on top of the head block at the time. Let $P$ be the execution payload included by the builder of $N$, this is the current head payload. Transactions in the attested inclusion list can *only* be invalid in a child of $P$ if there are transactions in $P$ that have the same source address and gas usage that was below the gas limit in the summary. For any such transaction the builder can add such transaction to the exclusion list and not include it in its payload. If there are remaining transactions in its payload from the same address, the nonce will have to be higher nonce than the transaction that was added in the exclusion list. This process can be repeated until there are no more transactions in the summary from that given address that have been invalidated. + +### Censoring + +We prove the following: the builder cannot force a transaction in the inclusion list to revert due to gas limit usage. A malicious builder that attempts to add in the exclusion list some transactions from an address with high gas limit but low usage, so that the remaining transactions in the summary have lower gas limit and the included transaction with higher gas usage reverts. However, this is impossible since any transaction in the exclusion list has to have lower nonce since it was already included in the previous block. That is, any attempt by the builder of changing the order in which to include transactions in the exclusion list, will result in its payload being invalid, and thus the inclusion lists transactions that haven't been invalidated on N will remain valid for the next block. ## Builders