Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Retrospective invalidation of exec. payloads for opt. sync #2837

Closed
wants to merge 101 commits into from

Conversation

paulhauner
Copy link
Member

@paulhauner paulhauner commented Nov 30, 2021

Issue Addressed

NA

Proposed Changes

Adds the functionality to allow blocks to be validated/invalidated after their import as per the optimistic sync spec. This means:

  • Updating ProtoArray to allow flipping the execution_status of ancestors/descendants based on payload validity updates.
  • Creating separation between execution_layer and the beacon_chain by creating a PayloadStatus struct.
  • Refactoring how the execution_layer selects a PayloadStatus from the multiple statuses returned from multiple EEs.
  • Adding testing framework for optimistic imports.
  • Add ExecutionBlockHash(Hash256) new-type struct to avoid confusion between beacon block roots and execution payload hashes.
  • Add merge to FORKS in the Makefile to ensure we test the beacon chain with merge settings.
    • Fix some tests here that were failing due to a missing execution layer.

TODO

  • Balance tests

@paulhauner paulhauner removed the blocked label Feb 8, 2022
@paulhauner paulhauner changed the title Retrospective invalidation of executon payloads for optimistic sync Retrospective invalidation of exec. payloads for opt. sync Feb 8, 2022
@paulhauner
Copy link
Member Author

Thanks @realbigsean and @michaelsproul! That was a great review, thanks for the improvements and bug fixes 😬

I have addressed all your comments and am waiting for CI to pass ☺️

I can't see this [FORKS] change, maybe you've got it lurking locally?

I must have lost this one between my desktop and laptop, thanks! Added here 2ae6d78.

@paulhauner paulhauner added ready-for-review The code is ready for review and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Feb 26, 2022
Copy link
Member

@michaelsproul michaelsproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! I'm satisfied with the latest updates, and the test coverage

Just a few docs that I think we should update, then we can merge

consensus/proto_array/src/proto_array.rs Outdated Show resolved Hide resolved
Comment on lines +134 to +136
0_i64
.checked_sub(node.weight as i64)
.ok_or(Error::InvalidExecutionDeltaOverflow(node_index))?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

consensus/proto_array/src/proto_array.rs Outdated Show resolved Hide resolved
paulhauner and others added 2 commits February 28, 2022 10:53
Co-authored-by: Michael Sproul <micsproul@gmail.com>
@paulhauner
Copy link
Member Author

Thanks for the approval @michaelsproul 🙏

I'll wait for approval/comments from @realbigsean before merging ☺️

Start adding execution_status tests

Complete test 01

Add tests 2, 3

Appease clippy
@paulhauner
Copy link
Member Author

I added some non-substantive tests for the proto_array crate in 1ab1c43.

I was testing to check balance propagation with and without proposer boost. I didn't find any bugs.

Copy link
Member

@realbigsean realbigsean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok changes look good to me!

@paulhauner
Copy link
Member Author

bors r+

🎉

@paulhauner paulhauner added ready-for-merge This PR is ready to merge. and removed ready-for-review The code is ready for review labels Feb 28, 2022
bors bot pushed a commit that referenced this pull request Feb 28, 2022
## Issue Addressed

NA

## Proposed Changes

Adds the functionality to allow blocks to be validated/invalidated after their import as per the [optimistic sync spec](https://github.com/ethereum/consensus-specs/blob/dev/sync/optimistic.md#how-to-optimistically-import-blocks). This means:

- Updating `ProtoArray` to allow flipping the `execution_status` of ancestors/descendants based on payload validity updates.
- Creating separation between `execution_layer` and the `beacon_chain` by creating a `PayloadStatus` struct.
- Refactoring how the `execution_layer` selects a `PayloadStatus` from the multiple statuses returned from multiple EEs.
- Adding testing framework for optimistic imports.
- Add `ExecutionBlockHash(Hash256)` new-type struct to avoid confusion between *beacon block roots* and *execution payload hashes*.
- Add `merge` to [`FORKS`](https://github.com/sigp/lighthouse/blob/c3a793fd73a3b11b130b82032904d39c952869e4/Makefile#L17) in the `Makefile` to ensure we test the beacon chain with merge settings.
    - Fix some tests here that were failing due to a missing execution layer.

## TODO

- [ ] Balance tests

Co-authored-by: Mark Mackey <mark@sigmaprime.io>
@bors bors bot changed the title Retrospective invalidation of exec. payloads for opt. sync [Merged by Bors] - Retrospective invalidation of exec. payloads for opt. sync Mar 1, 2022
@bors bors bot closed this Mar 1, 2022
bors bot pushed a commit that referenced this pull request Mar 8, 2022
## Issue Addressed

There has been an [`engine_exchangetransitionconfigurationv1`](https://github.com/ethereum/execution-apis/blob/main/src/engine/specification.md#engine_exchangetransitionconfigurationv1) method added to the execution API specs.

The `engine_exchangetransitionconfigurationv1` will be polled every 60s as per this PR: ethereum/execution-apis#189. If that PR is merged as-is, then we will be matching the spec. If that PR *is not* merged, we are still fully compatible with the spec, but just doing more than we are required.

## Additional Info

- [x] ~~Blocked on #2837~~
- [x] Add method to EE integration tests
bors bot pushed a commit that referenced this pull request Mar 9, 2022
## Issue Addressed

Resolves #2936

## Proposed Changes

Adds functionality for calling [`validator/prepare_beacon_proposer`](https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/Validator/prepareBeaconProposer) in advance.

There is a `BeaconChain::prepare_beacon_proposer` method which, which called, computes the proposer for the next slot. If that proposer has been registered via the `validator/prepare_beacon_proposer` API method, then the `beacon_chain.execution_layer` will be provided the `PayloadAttributes` for us in all future forkchoiceUpdated calls. An artificial forkchoiceUpdated call will be created 4s before each slot, when the head updates and when a validator updates their information.

Additionally, I added strict ordering for calls from the `BeaconChain` to the `ExecutionLayer`. I'm not certain the `ExecutionLayer` will always maintain this ordering, but it's a good start to have consistency from the `BeaconChain`. There are some deadlock opportunities introduced, they are documented in the code.

## Additional Info

- ~~Blocked on #2837~~

Co-authored-by: realbigsean <seananderson33@GMAIL.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bellatrix Required to support the Bellatrix Upgrade ready-for-merge This PR is ready to merge.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants