Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposer LMD Score Boosting #2730

Merged
merged 24 commits into from Nov 23, 2021
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
97e6d5c
Merge branch 'fc-dev-validate_target_epoch_scope-patch' into dev
djrtwo Nov 22, 2021
7736c8b
Merge branch 'dev' of github.com:ethereum/eth2.0-specs into dev
djrtwo Nov 22, 2021
2d161b4
Add proposer score boosting & related tests
adiasg Nov 19, 2021
281c1b2
Update validator guide with ATTESTATION_OFFSET_QUOTIENT
adiasg Nov 19, 2021
47fa6d1
Add parameter for score boost value
adiasg Nov 20, 2021
504d82c
Add datatype to new parameters
adiasg Nov 20, 2021
b0fb861
Make PROPOSER_SCORE_BOOST a percentage value
adiasg Nov 20, 2021
3b20e3e
Apply suggestions from code review
adiasg Nov 22, 2021
859bbf4
This reverts commit 4c726cdff39a10c5d096b294fb562cfc99c1f068.
adiasg Nov 22, 2021
88c76ab
Apply Danny's code review
adiasg Nov 22, 2021
cebe6ba
minor formatting cleanups
djrtwo Nov 22, 2021
282d85b
simplify on_tick proposer boost update
djrtwo Nov 22, 2021
ea09df5
toc
djrtwo Nov 22, 2021
1d835c5
Apply Danny's code review & suggestions
adiasg Nov 22, 2021
d85d439
Rename test
adiasg Nov 22, 2021
64b4ca2
add PROPOSER_SCORE_BOOST to configuration yaml files
djrtwo Nov 23, 2021
bdd7b07
Add configuration value checks
hwwhww Nov 23, 2021
a0b5a80
Apply HWW code's review - fix is_before_attesting_interval
adiasg Nov 23, 2021
ecbe919
Apply HWW code's review - properly update test steps
adiasg Nov 23, 2021
2a5c9d8
Set PROPOSER_SCORE_BOOST to 70%
adiasg Nov 23, 2021
6f95637
Merging local branch to remote latest
adiasg Nov 23, 2021
2ba0586
Add `proposer_boost_root` field to test vector "checks" step
hwwhww Nov 23, 2021
40f1c85
Merge pull request #2736 from ethereum/proposer-score-boost-add-check
djrtwo Nov 23, 2021
975931b
pr feedback
djrtwo Nov 23, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 5 additions & 0 deletions configs/mainnet.yaml
Expand Up @@ -70,6 +70,11 @@ MIN_PER_EPOCH_CHURN_LIMIT: 4
CHURN_LIMIT_QUOTIENT: 65536


# Fork choice
# ---------------------------------------------------------------
# 25%
PROPOSER_SCORE_BOOST: 70

# Deposit contract
# ---------------------------------------------------------------
# Ethereum PoW Mainnet
Expand Down
6 changes: 6 additions & 0 deletions configs/minimal.yaml
Expand Up @@ -69,6 +69,12 @@ MIN_PER_EPOCH_CHURN_LIMIT: 4
CHURN_LIMIT_QUOTIENT: 32


# Fork choice
# ---------------------------------------------------------------
# 25%
PROPOSER_SCORE_BOOST: 70


# Deposit contract
# ---------------------------------------------------------------
# Ethereum Goerli testnet
Expand Down
6 changes: 6 additions & 0 deletions specs/merge/fork-choice.md
Expand Up @@ -175,6 +175,12 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None:
# Add new state for this block to the store
store.block_states[hash_tree_root(block)] = state

# Add proposer score boost if the block is timely
time_into_slot = (store.time - store.genesis_time) % SECONDS_PER_SLOT
is_before_attesting_interval = time_into_slot < SECONDS_PER_SLOT // INTERVALS_PER_SLOT
if get_current_slot(store) == block.slot and is_before_attesting_interval:
store.proposer_boost_root = hash_tree_root(block)

# Update justified checkpoint
if state.current_justified_checkpoint.epoch > store.justified_checkpoint.epoch:
if state.current_justified_checkpoint.epoch > store.best_justified_checkpoint.epoch:
Expand Down
1 change: 0 additions & 1 deletion specs/phase0/beacon-chain.md
Expand Up @@ -169,7 +169,6 @@ We define the following Python custom types for type hinting and readability:
| `BLSPubkey` | `Bytes48` | a BLS12-381 public key |
| `BLSSignature` | `Bytes96` | a BLS12-381 signature |


## Constants

The following values are (non-configurable) constants used throughout the specification.
Expand Down
44 changes: 43 additions & 1 deletion specs/phase0/fork-choice.md
Expand Up @@ -7,7 +7,9 @@

- [Introduction](#introduction)
- [Fork choice](#fork-choice)
- [Constant](#constant)
- [Preset](#preset)
- [Configuration](#configuration)
- [Helpers](#helpers)
- [`LatestMessage`](#latestmessage)
- [`Store`](#store)
Expand Down Expand Up @@ -56,12 +58,27 @@ Any of the above handlers that trigger an unhandled exception (e.g. a failed ass
4) **Manual forks**: Manual forks may arbitrarily change the fork choice rule but are expected to be enacted at epoch transitions, with the fork details reflected in `state.fork`.
5) **Implementation**: The implementation found in this specification is constructed for ease of understanding rather than for optimization in computation, space, or any other resource. A number of optimized alternatives can be found [here](https://github.com/protolambda/lmd-ghost).


### Constant

| Name | Value |
| - | - |
| `INTERVALS_PER_SLOT` | `uint64(3)` |

### Preset

| Name | Value | Unit | Duration |
| - | - | :-: | :-: |
| `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` | `2**3` (= 8) | slots | 96 seconds |

### Configuration

| Name | Value |
| - | - |
| `PROPOSER_SCORE_BOOST` | `uint64(70)` |

- The proposer score boost is worth `PROPOSER_SCORE_BOOST` percentage of the committee's weight, i.e., for slot with committee weight `committee_weight` the boost weight is equal to `(committee_weight * PROPOSER_SCORE_BOOST) // 100`.

### Helpers

#### `LatestMessage`
Expand All @@ -83,6 +100,7 @@ class Store(object):
justified_checkpoint: Checkpoint
finalized_checkpoint: Checkpoint
best_justified_checkpoint: Checkpoint
proposer_boost_root: Root
blocks: Dict[Root, BeaconBlock] = field(default_factory=dict)
block_states: Dict[Root, BeaconState] = field(default_factory=dict)
checkpoint_states: Dict[Checkpoint, BeaconState] = field(default_factory=dict)
Expand All @@ -103,12 +121,14 @@ def get_forkchoice_store(anchor_state: BeaconState, anchor_block: BeaconBlock) -
anchor_epoch = get_current_epoch(anchor_state)
justified_checkpoint = Checkpoint(epoch=anchor_epoch, root=anchor_root)
finalized_checkpoint = Checkpoint(epoch=anchor_epoch, root=anchor_root)
proposer_boost_root = Root()
return Store(
time=uint64(anchor_state.genesis_time + SECONDS_PER_SLOT * anchor_state.slot),
genesis_time=anchor_state.genesis_time,
justified_checkpoint=justified_checkpoint,
finalized_checkpoint=finalized_checkpoint,
best_justified_checkpoint=justified_checkpoint,
proposer_boost_root=proposer_boost_root,
blocks={anchor_root: copy(anchor_block)},
block_states={anchor_root: copy(anchor_state)},
checkpoint_states={justified_checkpoint: copy(anchor_state)},
Expand Down Expand Up @@ -156,11 +176,22 @@ def get_ancestor(store: Store, root: Root, slot: Slot) -> Root:
def get_latest_attesting_balance(store: Store, root: Root) -> Gwei:
state = store.checkpoint_states[store.justified_checkpoint]
active_indices = get_active_validator_indices(state, get_current_epoch(state))
return Gwei(sum(
attestation_score = Gwei(sum(
state.validators[i].effective_balance for i in active_indices
if (i in store.latest_messages
and get_ancestor(store, store.latest_messages[i].root, store.blocks[root].slot) == root)
))
proposer_score = Gwei(0)
if store.proposer_boost_root != Root():
block = store.blocks[root]
if get_ancestor(store, root, block.slot) == store.proposer_boost_root:
num_validators = len(get_active_validator_indices(state, get_current_epoch(state)))
avg_balance = get_total_active_balance(state) // num_validators
committee_size = num_validators // SLOTS_PER_EPOCH
committee_weight = committee_size * avg_balance
proposer_score = (committee_weight * PROPOSER_SCORE_BOOST) // 100
return attestation_score + proposer_score

```

#### `filter_block_tree`
Expand Down Expand Up @@ -339,6 +370,11 @@ def on_tick(store: Store, time: uint64) -> None:
store.time = time

current_slot = get_current_slot(store)

# Reset store.proposer_boost_root if this is a new slot
if current_slot > previous_slot:
store.proposer_boost_root = Root()

# Not a new epoch, return
if not (current_slot > previous_slot and compute_slots_since_epoch_start(current_slot) == 0):
return
Expand Down Expand Up @@ -377,6 +413,12 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None:
# Add new state for this block to the store
store.block_states[hash_tree_root(block)] = state

# Add proposer score boost if the block is timely
time_into_slot = (store.time - store.genesis_time) % SECONDS_PER_SLOT
is_before_attesting_interval = time_into_slot < SECONDS_PER_SLOT // INTERVALS_PER_SLOT
if get_current_slot(store) == block.slot and is_before_attesting_interval:
store.proposer_boost_root = hash_tree_root(block)

# Update justified checkpoint
if state.current_justified_checkpoint.epoch > store.justified_checkpoint.epoch:
if state.current_justified_checkpoint.epoch > store.best_justified_checkpoint.epoch:
Expand Down
4 changes: 2 additions & 2 deletions specs/phase0/validator.md
Expand Up @@ -446,7 +446,7 @@ def get_block_signature(state: BeaconState, block: BeaconBlock, privkey: int) ->

A validator is expected to create, sign, and broadcast an attestation during each epoch. The `committee`, assigned `index`, and assigned `slot` for which the validator performs this role during an epoch are defined by `get_committee_assignment(state, epoch, validator_index)`.

A validator should create and broadcast the `attestation` to the associated attestation subnet when either (a) the validator has received a valid block from the expected block proposer for the assigned `slot` or (b) one-third of the `slot` has transpired (`SECONDS_PER_SLOT / 3` seconds after the start of `slot`) -- whichever comes _first_.
A validator should create and broadcast the `attestation` to the associated attestation subnet when either (a) the validator has received a valid block from the expected block proposer for the assigned `slot` or (b) `1 / INTERVALS_PER_SLOT` of the `slot` has transpired (`SECONDS_PER_SLOT / INTERVALS_PER_SLOT` seconds after the start of `slot`) -- whichever comes _first_.
Copy link
Contributor

@hwwhww hwwhww Nov 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpicking: we use SECONDS_PER_SLOT // INTERVALS_PER_SLOT in code and have set SECONDS_PER_SLOT % INTERVALS_PER_SLOT == 0. It could be a bit off when SECONDS_PER_SLOT % INTERVALS_PER_SLOT != 0.

Is SECONDS_PER_SLOT % INTERVALS_PER_SLOT == 0 an invariant that should be checked in test_config_invariants.py?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note that this is division and not integer division.

I don't think this makes sense as a strict config invariant but maybe deserves a note

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EDITED: Sorry, I meant "we use SECONDS_PER_SLOT // INTERVALS_PER_SLOT in code ".
^^^^^ typo fixed

note that this is division and not integer division.

Yes. Clarify: so the value that the validator guide describes may be different from the value from the fork choice rule code when SECONDS_PER_SLOT % INTERVALS_PER_SLOT != 0. INTERVALS_PER_SLOT := 3 is fine for our minimal and mainnet config.

Another option is also using integer division here: "... SECONDS_PER_SLOT // INTERVALS_PER_SLOT seconds after the start of slot..."

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, right. Then it is valuable to have these work well together.


*Note*: Although attestations during `GENESIS_EPOCH` do not count toward FFG finality, these initial attestations do give weight to the fork choice, are rewarded, and should be made.

Expand Down Expand Up @@ -569,7 +569,7 @@ def get_aggregate_signature(attestations: Sequence[Attestation]) -> BLSSignature

#### Broadcast aggregate

If the validator is selected to aggregate (`is_aggregator`), then they broadcast their best aggregate as a `SignedAggregateAndProof` to the global aggregate channel (`beacon_aggregate_and_proof`) two-thirds of the way through the `slot`-that is, `SECONDS_PER_SLOT * 2 / 3` seconds after the start of `slot`.
If the validator is selected to aggregate (`is_aggregator`), then they broadcast their best aggregate as a `SignedAggregateAndProof` to the global aggregate channel (`beacon_aggregate_and_proof`) `2 / INTERVALS_PER_SLOT` of the way through the `slot`-that is, `SECONDS_PER_SLOT * 2 / INTERVALS_PER_SLOT` seconds after the start of `slot`.

Selection proofs are provided in `AggregateAndProof` to prove to the gossip channel that the validator has been selected as an aggregator.

Expand Down
@@ -1,3 +1,4 @@
import random
from eth_utils import encode_hex

from eth2spec.test.context import (
Expand All @@ -19,6 +20,7 @@
add_block,
)
from eth2spec.test.helpers.state import (
next_slots,
next_epoch,
state_transition_and_sign_block,
)
Expand Down Expand Up @@ -103,17 +105,21 @@ def test_split_tie_breaker_no_attestations(spec, state):
}
})

# block at slot 1
# Create block at slot 1
block_1_state = genesis_state.copy()
block_1 = build_empty_block_for_next_slot(spec, block_1_state)
signed_block_1 = state_transition_and_sign_block(spec, block_1_state, block_1)
yield from tick_and_add_block(spec, store, signed_block_1, test_steps)

# additional block at slot 1
# Create additional block at slot 1
block_2_state = genesis_state.copy()
block_2 = build_empty_block_for_next_slot(spec, block_2_state)
block_2.body.graffiti = b'\x42' * 32
signed_block_2 = state_transition_and_sign_block(spec, block_2_state, block_2)

# Tick time past slot 1 so proposer score boost does not apply
spec.on_tick(store, store.genesis_time + (block_2.slot + 1) * spec.config.SECONDS_PER_SLOT)

yield from tick_and_add_block(spec, store, signed_block_1, test_steps)
yield from tick_and_add_block(spec, store, signed_block_2, test_steps)
djrtwo marked this conversation as resolved.
Show resolved Hide resolved

highest_root = max(spec.hash_tree_root(block_1), spec.hash_tree_root(block_2))
Expand Down Expand Up @@ -261,3 +267,67 @@ def test_filtered_block_tree(spec, state):
})

yield 'steps', test_steps


@with_all_phases
@spec_state_test
def test_proposer_boost_correct_head(spec, state):
test_steps = []
genesis_state = state.copy()

# Initialization
store, anchor_block = get_genesis_forkchoice_store_and_block(spec, state)
yield 'anchor_state', state
yield 'anchor_block', anchor_block
anchor_root = get_anchor_root(spec, state)
assert spec.get_head(store) == anchor_root
test_steps.append({
'checks': {
'head': get_formatted_head_output(spec, store),
}
})

# Build block that serves as head ONLY on timely arrival, and ONLY in that slot
state_1 = genesis_state.copy()
next_slots(spec, state_1, 3)
block_1 = build_empty_block_for_next_slot(spec, state_1)
signed_block_1 = state_transition_and_sign_block(spec, state_1, block_1)

# Build block that serves as current head, and remains the head after block_1.slot
state_2 = genesis_state.copy()
next_slots(spec, state_2, 2)
block_2 = build_empty_block_for_next_slot(spec, state_2)
signed_block_2 = state_transition_and_sign_block(spec, state_2.copy(), block_2)
adiasg marked this conversation as resolved.
Show resolved Hide resolved
while spec.hash_tree_root(block_1) >= spec.hash_tree_root(block_2):
block_2.body.graffiti = spec.Bytes32(hex(random.getrandbits(8 * 32))[2:].zfill(64))
signed_block_2 = state_transition_and_sign_block(spec, state_2.copy(), block_2)
assert spec.hash_tree_root(block_1) < spec.hash_tree_root(block_2)

# Tick to block_1 slot time
time = store.genesis_time + block_1.slot * spec.config.SECONDS_PER_SLOT
on_tick_and_append_step(spec, store, time, test_steps)

# Process block_2
yield from add_block(spec, store, signed_block_2, test_steps)
assert store.proposer_boost_root == spec.Root()
assert spec.get_head(store) == spec.hash_tree_root(block_2)

# Process block_1 on timely arrival
# The head should temporarily change to block_1
yield from add_block(spec, store, signed_block_1, test_steps)
assert store.proposer_boost_root == spec.hash_tree_root(block_1)
assert spec.get_head(store) == spec.hash_tree_root(block_1)

# After block_1.slot, the head should revert to block_2
time = store.genesis_time + (block_1.slot + 1) * spec.config.SECONDS_PER_SLOT
on_tick_and_append_step(spec, store, time, test_steps)
assert store.proposer_boost_root == spec.Root()
assert spec.get_head(store) == spec.hash_tree_root(block_2)

test_steps.append({
'checks': {
'head': get_formatted_head_output(spec, store),
}
})

yield 'steps', test_steps
Expand Up @@ -703,3 +703,83 @@ def test_new_finalized_slot_is_justified_checkpoint_ancestor(spec, state):
assert store.justified_checkpoint == another_state.current_justified_checkpoint

yield 'steps', test_steps


@with_all_phases
@spec_state_test
def test_proposer_boost(spec, state):
test_steps = []
genesis_state = state.copy()

# Initialization
store, anchor_block = get_genesis_forkchoice_store_and_block(spec, state)
yield 'anchor_state', state
yield 'anchor_block', anchor_block

# Build block that serves as head ONLY on timely arrival, and ONLY in that slot
state = genesis_state.copy()
next_slots(spec, state, 3)
block = build_empty_block_for_next_slot(spec, state)
signed_block = state_transition_and_sign_block(spec, state, block)

# Process block on timely arrival just before end of boost interval
time = (store.genesis_time + block.slot * spec.config.SECONDS_PER_SLOT +
spec.config.SECONDS_PER_SLOT // spec.INTERVALS_PER_SLOT - 1)
on_tick_and_append_step(spec, store, time, test_steps)
yield from add_block(spec, store, signed_block, test_steps)
assert store.proposer_boost_root == spec.hash_tree_root(block)
assert spec.get_latest_attesting_balance(store, spec.hash_tree_root(block)) > 0

# Ensure that boost is removed after slot is over
time = (store.genesis_time + block.slot * spec.config.SECONDS_PER_SLOT +
spec.config.SECONDS_PER_SLOT)
on_tick_and_append_step(spec, store, time, test_steps)
assert store.proposer_boost_root == spec.Root()
assert spec.get_latest_attesting_balance(store, spec.hash_tree_root(block)) == 0

next_slots(spec, state, 3)
block = build_empty_block_for_next_slot(spec, state)
signed_block = state_transition_and_sign_block(spec, state, block)

# Process block on timely arrival at start of boost interval
time = (store.genesis_time + block.slot * spec.config.SECONDS_PER_SLOT)
on_tick_and_append_step(spec, store, time, test_steps)
yield from add_block(spec, store, signed_block, test_steps)
assert store.proposer_boost_root == spec.hash_tree_root(block)
assert spec.get_latest_attesting_balance(store, spec.hash_tree_root(block)) > 0

# Ensure that boost is removed after slot is over
time = (store.genesis_time + block.slot * spec.config.SECONDS_PER_SLOT +
spec.config.SECONDS_PER_SLOT)
on_tick_and_append_step(spec, store, time, test_steps)
assert store.proposer_boost_root == spec.Root()
assert spec.get_latest_attesting_balance(store, spec.hash_tree_root(block)) == 0

djrtwo marked this conversation as resolved.
Show resolved Hide resolved
yield 'steps', test_steps


@with_all_phases
@spec_state_test
def test_proposer_boost_root_same_slot_untimely_block(spec, state):
test_steps = []
genesis_state = state.copy()

# Initialization
store, anchor_block = get_genesis_forkchoice_store_and_block(spec, state)
yield 'anchor_state', state
yield 'anchor_block', anchor_block

# Build block that serves as head ONLY on timely arrival, and ONLY in that slot
state = genesis_state.copy()
next_slots(spec, state, 3)
block = build_empty_block_for_next_slot(spec, state)
signed_block = state_transition_and_sign_block(spec, state, block)

# Process block on untimely arrival in the same slot
time = (store.genesis_time + block.slot * spec.config.SECONDS_PER_SLOT +
spec.config.SECONDS_PER_SLOT // spec.INTERVALS_PER_SLOT)
on_tick_and_append_step(spec, store, time, test_steps)
yield from add_block(spec, store, signed_block, test_steps)
assert store.proposer_boost_root == spec.Root()

yield 'steps', test_steps
Expand Up @@ -74,3 +74,10 @@ def test_time(spec, state):
@spec_state_test
def test_networking(spec, state):
assert spec.RANDOM_SUBNETS_PER_VALIDATOR <= spec.ATTESTATION_SUBNET_COUNT


@with_all_phases
@spec_state_test
def test_fork_choice(spec, state):
assert spec.INTERVALS_PER_SLOT < spec.config.SECONDS_PER_SLOT
assert spec.config.PROPOSER_SCORE_BOOST <= 100