64 epoch cleanup by @justindrake #2212

djrtwo · 2021-02-23T23:51:02Z

From @JustinDrake :

Pushed my review/cleanup. Largely just polishing :)

Cosmetic fixes

rename leak_score to leak_scores (for consistency with validators, balances, etc.)
rename "regular penalties" to "flag penalties" (for consistency with flag rewards)
rename ValidatorFlag to ParticipationFlags (more precise; also pluralised because as there are 8 flags)
initialise ParticipationFlags with 0b0000_0000 (instead of 0)
introduce the concept of "flag mask" (e.g. rename TIMELY_HEAD_FLAG to TIMELY_HEAD_FLAG_MASK)
disambiguate uses of "new" which sometimes means "modified" and sometimes "added"
rename is_activation_exit_period_boundary to is_activation_exit_epoch (for consistency with compute_activation_exit_epoch)
isolate leak updates in process_leak_updates (extracting logic from process_rewards_and_penalties) and call directly from process_epoch
change type of leak_epoch_counter to Epoch
rename leak_epoch_counter to leak_epochs_counter
rename SYNC_COMMITTEE_PUBKEY_AGGREGATES_SIZE to SYNC_SUBCOMMITTEE_SIZE
add is_leak_active (replacing and deprecating is_in_inactivity_leak)
add LEAK_PENALTY_QUOTIENT (merging with LEAK_SCORE_BIAS; replacing and deprecating INACTIVITY_PENALTY_QUOTIENT)
add EPOCHS_TO_LEAK_PENALTIES (replacing and deprecating MIN_EPOCHS_TO_INACTIVITY_PENALTY)
remove unnecessary for index in get_eligible_validator_indices(state) in process_leak_updates and get_flag_rewards

Substantive fixes

run process_effective_balance_updates only at activation-exit epochs
properly round up in compute_activation_exit_epoch (notice special case when epoch % EPOCHS_PER_ACTIVATION_EXIT_PERIOD == 0)
remove and is_matching_target in process_attestation when append(TIMELY_HEAD_FLAG_MASK)

@JustinDrake

port @JustinDrake's general cleanups from #2212

hwwhww

My first scan. 👀

remove and is_matching_target in process_attestation when append(TIMELY_HEAD_FLAG_MASK)

I can't find it in this PR. Was it left during the previous cleanups?

hwwhww · 2021-03-02T18:41:32Z

configs/mainnet/phase0.yaml

-# 2**26 (= 67,108,864)
-INACTIVITY_PENALTY_QUOTIENT: 67108864
+# 2**28 (= 268,435,456)
+LEAK_PENALTY_QUOTIENT: 268435456


INACTIVITY_PENALTY_QUOTIENT was renamed here but it was not modified in phase0/beacon-chain.md.

The phase 0 value should be 2**26 (= 67,108,864).

Double check: is it necessary to rename the variable of phase0?

get_inactivity_penalty_deltas will be rewritten anyway. So we only have to rename it in HF1.

Besides that the client implementations have to change their phase 0 code, the additional documents may also need to sync the differences.

I don't think we should rename any variables from phase 0. It's way to easy to bork test vectors and cause issues in clients, imo

hwwhww · 2021-03-02T18:53:21Z

specs/lightclient/beacon-chain.md


 ### Time parameters

 | Name | Value | Unit | Duration |
 | - | - | :-: | :-: |
-| `EPOCHS_PER_SYNC_COMMITTEE_PERIOD` | `Epoch(2**8)` (= 256) | epochs | ~27 hours |
-| `EPOCHS_PER_ACTIVATION_EXIT_PERIOD` | `Epoch(2**6)` (= 64) | epochs | ~6.8 hours |
+| `EPOCHS_TO_LEAK_PENALTIES` | `uint64(2**2)` (= 4) | epochs | 25.6 minutes |


leaving in here for now until we decide about renaming in general

hwwhww · 2021-03-02T18:56:51Z

configs/mainnet/phase0.yaml

@@ -99,7 +99,7 @@ MIN_VALIDATOR_WITHDRAWABILITY_DELAY: 256
 # 2**8 (= 256) epochs ~27 hours
 SHARD_COMMITTEE_PERIOD: 256
 # 2**2 (= 4) epochs 25.6 minutes
-MIN_EPOCHS_TO_INACTIVITY_PENALTY: 4
+EPOCHS_TO_LEAK_PENALTIES: 4


Again, want to make sure if it's a necessary change in phase 0.

specs/lightclient/beacon-chain.md

hwwhww · 2021-03-02T19:04:51Z

specs/lightclient/beacon-chain.md

+#### `is_leak_active`
+
+```python
+def is_leak_active(state: BeaconState) -> bool:
+    return get_finality_delay(state) > EPOCHS_TO_LEAK_PENALTIES
+```


It's a 100% duplicate of the phase 0 version.

leaving in until we decide on cross-cutting changes on phases

specs/lightclient/beacon-chain.md

hwwhww · 2021-03-02T19:38:18Z

specs/lightclient/beacon-chain.md

-    process_rewards(state)
-    process_penalties(state)


I can see why this change makes sense, but IMHO it's good to have multiple purpose-focused functions. It would be more flexible when we have to modify it in the future forks.

yeah, I would tend to agree given the surgery we've had to do across forks thus far

hwwhww · 2021-03-02T19:44:26Z

specs/lightclient/beacon-chain.md

@@ -602,21 +555,19 @@ def process_sync_committee(state: BeaconState, body: BeaconBlockBody) -> None:

 ### Modified helpers

-#### New `compute_activation_exit_epoch`
+#### Modified `compute_activation_exit_epoch`


TODO: need to add some unit tests for it.

djrtwo · 2021-03-05T17:54:24Z

I'm against changing constants names from phase 0 (and especially values e.g. INACTIVITY_PENALTY_QUOTIENT).

The general policy has been "never change constants at forks, only add new ones and update calling code". This helps keep configuration clean and avoid hard to track down changes and issues in testing (that might also surface in clients)

djrtwo

minor

djrtwo · 2021-03-01T19:40:23Z

specs/lightclient/beacon-chain.md

-def add_validator_flags(flags: ValidatorFlag, add: ValidatorFlag) -> ValidatorFlag:
-    return flags | add
+def add_flag(flags: ParticipationFlags, flag_index: int) -> ParticipationFlags:
+    flag = ParticipationFlags(2**flag_index)


Suggested change

flag = ParticipationFlags(2**flag_index)

flag = ParticipationFlags(2 ** flag_index)

tests/core/pyspec/eth2spec/test/helpers/rewards.py

djrtwo · 2021-03-05T18:00:02Z

configs/mainnet/phase0.yaml

-# 2**26 (= 67,108,864)
-INACTIVITY_PENALTY_QUOTIENT: 67108864
+# 2**28 (= 268,435,456)
+LEAK_PENALTY_QUOTIENT: 268435456


I don't think we should rename any variables from phase 0. It's way to easy to bork test vectors and cause issues in clients, imo

djrtwo · 2021-03-05T18:00:17Z

configs/mainnet/phase0.yaml

@@ -99,7 +99,7 @@ MIN_VALIDATOR_WITHDRAWABILITY_DELAY: 256
 # 2**8 (= 256) epochs ~27 hours
 SHARD_COMMITTEE_PERIOD: 256
 # 2**2 (= 4) epochs 25.6 minutes
-MIN_EPOCHS_TO_INACTIVITY_PENALTY: 4
+EPOCHS_TO_LEAK_PENALTIES: 4


djrtwo · 2021-03-05T18:01:37Z

specs/lightclient/beacon-chain.md


 ### Time parameters

 | Name | Value | Unit | Duration |
 | - | - | :-: | :-: |
-| `EPOCHS_PER_SYNC_COMMITTEE_PERIOD` | `Epoch(2**8)` (= 256) | epochs | ~27 hours |
-| `EPOCHS_PER_ACTIVATION_EXIT_PERIOD` | `Epoch(2**6)` (= 64) | epochs | ~6.8 hours |
+| `EPOCHS_TO_LEAK_PENALTIES` | `uint64(2**2)` (= 4) | epochs | 25.6 minutes |


leaving in here for now until we decide about renaming in general

specs/lightclient/beacon-chain.md

djrtwo · 2021-03-05T18:08:06Z

specs/lightclient/beacon-chain.md

    penalties = [Gwei(0)] * len(state.validators)
-
-    # Only give penalties on period boundaries


The problem with removing this blocking condition is that test vectors can very strangely be generated. That is, the vectors for get_leak_penalties specifically if run not on an epoch boundary will return non-zero values. I'd prefer keeping the blocking condition in to avoid misuse in alternative contexts even though the calling function in the spec had the conditional

djrtwo · 2021-03-05T18:14:54Z

remove and is_matching_target in process_attestation when append(TIMELY_HEAD_FLAG_MASK)

this is purposeful

it matches phase 0 functionality, see get_matching_head_attestations which calls get_matching_target_attestations as its base
this functionality was put into phase 0 to ensure that there was not a perverse scenario where you can essentially net 0 reward/penalty and not contribute to finality (e.g. get correct head but insert a bad target). It is instead natural for correctness to stem from source into target up to head (voting on a cohesive chain) rather than each being independent

djrtwo · 2021-03-11T22:16:53Z

This optimization will not go into Altair. More R&D is desired before making such spec optimizations.

See call notes here ethereum/eth2.0-pm#208

Justin and others added 4 commits February 23, 2021 10:53

Justin review

f0cbab1

Add ToC and fix typo

9cc57ee

change ParticipationFlags from uint8 to Bitvector[8]

7093338

merge

ad331c6

djrtwo mentioned this pull request Feb 23, 2021

Penalties+activs+exits only at 64-epoch boundary #2192

Closed

djrtwo and others added 5 commits February 23, 2021 16:55

toc

1401d0b

Merge branch 'vbuterin-patch-10' into 64-epochs-justin

fc96f26

test_finality_rule_* tests not passing

d001be9

minor fix for tests

f3fe410

spelling

b2e29e7

djrtwo added a commit that referenced this pull request Mar 2, 2021

port @JustinDrake's general cleanups from #2212

a7fcc1d

djrtwo mentioned this pull request Mar 2, 2021

port @justindrake's general cleanups from #2212 #2216

Merged

djrtwo added a commit that referenced this pull request Mar 2, 2021

port @JustinDrake's general cleanups from #2212

f86b56b

djrtwo added a commit that referenced this pull request Mar 2, 2021

port @JustinDrake's general cleanups from #2212

628308d

djrtwo added a commit that referenced this pull request Mar 2, 2021

port @JustinDrake's general cleanups from #2212

893ab17

djrtwo added a commit that referenced this pull request Mar 2, 2021

port @JustinDrake's general cleanups from #2212

deace87

djrtwo added a commit that referenced this pull request Mar 2, 2021

Merge pull request #2216 from ethereum/justin-hf1-cleanups

09648cf

port @JustinDrake's general cleanups from #2212

Merge branch 'vbuterin-patch-10' into 64-epochs-justin

414c0b2

djrtwo force-pushed the 64-epochs-justin branch from 9bce21e to 414c0b2 Compare March 2, 2021 17:57

hwwhww reviewed Mar 2, 2021

View reviewed changes

JustinDrake added the Altair aka HF1 label Mar 3, 2021

djrtwo commented Mar 5, 2021

View reviewed changes

pr feedback

4d79e9c

djrtwo mentioned this pull request Mar 9, 2021

independent leak score feature #2222

Merged

djrtwo added the DO NOT MERGE label Mar 10, 2021

mratsim mentioned this pull request Mar 11, 2021

Eth2 Call 59 ethereum/eth2.0-pm#208

Closed

djrtwo closed this Mar 11, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

64 epoch cleanup by @justindrake #2212

64 epoch cleanup by @justindrake #2212

djrtwo commented Feb 23, 2021 •

edited

Loading

hwwhww left a comment

hwwhww Mar 2, 2021

djrtwo Mar 5, 2021

hwwhww Mar 2, 2021

djrtwo Mar 5, 2021

hwwhww Mar 2, 2021

djrtwo Mar 5, 2021

hwwhww Mar 2, 2021 •

edited

Loading

djrtwo Mar 5, 2021

hwwhww Mar 2, 2021 •

edited

Loading

djrtwo Mar 5, 2021 •

edited

Loading

hwwhww Mar 2, 2021

djrtwo commented Mar 5, 2021

djrtwo left a comment

djrtwo Mar 1, 2021

djrtwo Mar 5, 2021

djrtwo Mar 5, 2021

djrtwo Mar 5, 2021

djrtwo Mar 5, 2021

djrtwo commented Mar 5, 2021

djrtwo commented Mar 11, 2021

	flag = ParticipationFlags(2**flag_index)
	flag = ParticipationFlags(2 ** flag_index)

		penalties = [Gwei(0)] * len(state.validators)

		# Only give penalties on period boundaries

64 epoch cleanup by @justindrake #2212

64 epoch cleanup by @justindrake #2212

Conversation

djrtwo commented Feb 23, 2021 • edited Loading

Cosmetic fixes

Substantive fixes

hwwhww left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hwwhww Mar 2, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hwwhww Mar 2, 2021 • edited Loading

Choose a reason for hiding this comment

djrtwo Mar 5, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

djrtwo commented Mar 5, 2021

djrtwo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

djrtwo commented Mar 5, 2021

djrtwo commented Mar 11, 2021

djrtwo commented Feb 23, 2021 •

edited

Loading

hwwhww Mar 2, 2021 •

edited

Loading

hwwhww Mar 2, 2021 •

edited

Loading

djrtwo Mar 5, 2021 •

edited

Loading