Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Status code / slot reorganization #374

Merged
merged 17 commits into from
Jan 6, 2019
Merged

Status code / slot reorganization #374

merged 17 commits into from
Jan 6, 2019

Conversation

vbuterin
Copy link
Contributor

@vbuterin vbuterin commented Dec 28, 2018

Supersedes #350 and #354

  • Added activation_slot, exit_slot, penalized_slot, withdrawal_slot, use these to determine if a validator is active
  • Universal min activation/exit delay of 256 slots
  • Min exit time of ~1 day, but penalization delays this to ~18 days
  • Penalty calculation period of [time penalized - 18 days, time penalized + 18 days]; made the total penalties array fixed size and wraparound to make calculation more fine-grained
  • Processes withdrawals in all epochs, not just dynasty-changing epochs
  • Change get_shuffling function to take slot as argument

Not yet done:

  • Removed shard_committees from the state
  • Removed persistent committees from the state

* Added `activation_slot`, `exit_slot`, `penalized_slot`, `withdrawal_slot`, use these to determine if a validator is active
* Universal min activation/exit delay of 256 slots
* Min exit time of ~1 day, but penalization delays this to ~18 days
* Penalty calculation period of `[time penalized - 18 days, time penalized + 18 days]`; made the total penalties array fixed size and wraparound to make calculation more fine-grained
* Processes withdrawals in all epochs, not just dynasty-changing epochs
* Change `get_shuffling` function to take slot as argument

Not yet done:

* Removed `shard_committees` from the state
* Removed persistent committees from the state
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Show resolved Hide resolved
Copy link
Collaborator

@JustinDrake JustinDrake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few possible bugs and some nitpicks.

specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
vbuterin and others added 4 commits December 30, 2018 20:42
Cleanups

* (typo) Remove `get_new_validator_registry_delta_chain_tip` from table of contents
* (typo) Update "Routines for updating validator status" in table of contents
* Update `FAR_FUTURE_SLOT` from `2**63` to `2**64 - 1`
* Put more constants in "Initial values", homogenise
* Cleanup note formatting
* Remove `ZERO_BALANCE_VALIDATOR_TTL` logic (to be possibly reintroduced in phase 2).
* Cleanup `min_empty_validator_index`
* Rename `deposit` to `amount` in `process_deposit` and `DepositData`.
* (typo) Remove new line under `process_penalties_and_exits`
* (typo) "Status codes" => "Status flags" in the table of contents
* (typo) `(state.slot - EPOCH_LENGTH) % LATEST_RANDAO_MIXES_LENGTH` => Use `SEED_LOOKAHEAD` instead.
* Put `state.validator_registry_latest_change_slot = state.slot` in `update_validator_registry`.
* Use `GENESIS_SLOT` for `last_poc_change_slot=0` and `second_last_poc_change_slot=0`.

Bugfixes

* (typo) `validator_exit` => `exit.validator_index`
* Separate initial deposits and initial activations to avoid double activations
* Replace `proposer.status != EXITED_WITH_PENALTY` with `validator.penalized_slot > state.slot` in two different places.
* Replace `status == EXITED_WITH_PENALTY` with `validator.penalized_slot <= state.slot` (and validator active) in two different places.
@JustinDrake
Copy link
Collaborator

Comments regarding process_penalties_and_exits:

  • validators is used but not defined
  • all_indices is used but not defined
  • MIN_VALIDATOR_WITHDRAWAL_TIME is used but not defined
  • total_penalties * 3 => Explain the hardcoded number 3 (maybe create a new constant)
  • "# TODO: calculate and apply penalties for slashed validators" => Is that done above (see state.validator_balances[index] -= penalty)?
  • Did we agree on no withdrawal logic in phase 0? That is, can the 20 lines of code starting from def eligible(index): be deleted? (That would explain a lot of the above!)

@djrtwo djrtwo mentioned this pull request Jan 1, 2019
17 tasks
specs/core/0_beacon-chain.md Show resolved Hide resolved
Activate the validator with the given ``index``.
Note that this function mutates ``state``.
"""
def activate_validator(state: BeaconState, index: int, genesis: bool) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could take in param activation_slot instead of genesis to remove the forever genesis conditional sitting around.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea that the fact that activation is delayed by 256 slots is part of the activation function, and not whatever is calling it; it feels like the right place to have that logic. activation_slot would make things less intuitive in that regard imo; it would also break symmetry with exit_validator and other similar functions that don't have an activation slot.

Copy link
Contributor Author

@vbuterin vbuterin Jan 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Did we agree on no withdrawal logic in phase 0? That is, can the 20 lines of code starting from def eligible(index): be deleted? (That would explain a lot of the above!)

Did we? I thought we agreed that we finish at validators being in the WITHDRAWABLE state, and "no withdrawals" meant we don't try to move beyond that point (as doing that would require for example specifying a withdrawal shard, and in general is more dependent on state execution).

If we don't go up to WITHDRAWABLE in phase 0, then in phase 1 we would either need to have special-case code for a one-time sweep that converts all already-eligible accounts to the WITHDRAWABLE state, or have a large one-time backlog of validators that go through the exit queue, both of which seem uglier than just implementing it now.

  • total_penalties * 3 => Explain the hardcoded number 3 (maybe create a new constant)

It ensures that penalties are 100% in the specific case where enough validators misbehave to cause a safety failure. I'm ok with making it a constant or not making it a constant.

specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Show resolved Hide resolved
specs/core/0_beacon-chain.md Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
@djrtwo
Copy link
Contributor

djrtwo commented Jan 1, 2019

total_penalties * 3 => Explain the hardcoded number 3 (maybe create a new constant)

This 3 is related to the justification/finalization threshold (1/3). We might define that in the spec in general so that it can be set depending on the instantiation of the protocol (for instance #357). This number here can then be a function of the safety threshold constant(s)

@hwwhww hwwhww mentioned this pull request Jan 2, 2019
29 tasks
Copy link
Contributor

@terencechain terencechain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I implemented these changes and found a few questions =)

specs/core/0_beacon-chain.md Show resolved Hide resolved
specs/core/0_beacon-chain.md Show resolved Hide resolved
specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
@djrtwo
Copy link
Contributor

djrtwo commented Jan 3, 2019

Did we? I thought we agreed that we finish at validators being in the WITHDRAWABLE state, and "no withdrawals" meant we don't try to move beyond that point (as doing that would require for example specifying a withdrawal shard, and in general is more dependent on state execution).

I'm pro having them end in the withdrawable state. I think it's cleaner to in terms of processing penalties.

Copy link
Contributor

@djrtwo djrtwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

specs/core/0_beacon-chain.md Outdated Show resolved Hide resolved
state.latest_penalized_exit_balances[(state.slot // EPOCH_LENGTH) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index)

whistleblower_index = get_beacon_proposer_index(state, state.slot)
whistleblower_reward = get_effective_balance(state, index) // WHISTLEBLOWER_REWARD_QUOTIENT
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_effective_balance(state, index) is called twice in this function, could be refactored:

    exit_validator(state, index)
    validator = state.validator_registry[index]

    last_penalized_epoch = (state.slot // EPOCH_LENGTH) % LATEST_PENALIZED_EXIT_LENGTH
    effective_balance = get_effective_balance(state, index)

    state.latest_penalized_exit_balances[last_penalized_epoch] += effective_balance
    whistleblower_index = get_beacon_proposer_index(state, state.slot)
    whistleblower_reward = effective_balance // WHISTLEBLOWER_REWARD_QUOTIENT

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_effective_balance is not a significant computational cost and it's not much longer than a variable name, so not sure that's actually a net improvement, but I'm fine wither way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

helper var not needed here in the spec imo

@djrtwo
Copy link
Contributor

djrtwo commented Jan 5, 2019

fixed merge conflict. Ready to merge @vbuterin

@terencechain
Copy link
Contributor

I've also implemented these changes in code, if y'all wanna see what the diff looks like in code, here it is 😄 https://github.com/prysmaticlabs/prysm/pull/1233/files

@vbuterin vbuterin merged commit c63803a into master Jan 6, 2019
hwwhww added a commit to hwwhww/py-evm that referenced this pull request Jan 7, 2019
@djrtwo djrtwo deleted the vbuterin-patch-19 branch January 7, 2019 16:53
hwwhww added a commit to hwwhww/py-evm that referenced this pull request Jan 9, 2019
hwwhww added a commit to hwwhww/py-evm that referenced this pull request Jan 10, 2019
hwwhww added a commit to ethereum/py-evm that referenced this pull request Jan 10, 2019
@hwwhww hwwhww mentioned this pull request Apr 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants