Skip to content

Sync Re-Write #663

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 34 commits into from
Dec 9, 2019
Merged

Sync Re-Write #663

merged 34 commits into from
Dec 9, 2019

Conversation

@AgeManning
Copy link
Member

@AgeManning AgeManning commented Dec 4, 2019 •
edited
Loading

Description

This is another re-write of Lighthouse's syncing logic. This employs a new long-range syncing algorithm which has a variety of features and improvements on its predecessor:

  • Groups peers to similar chains and load-balances the downloading of batches from pool'd peers.
  • Groups peers based on identical finalised chains then on identical head chains
  • Removes polling and searching to progress state and now a single action progresses the global syncing state machine to completion

Note that this syncing version is not entirely complete. Error handling and malicious actors are not accounted for yet.

Along with the syncing improvement, this PR comes with minor bug fixes:

  • Corrects maxpeers lock
  • Changes ports for gossipsub tests
  • Cleans up logging

This addresses #509

@AgeManning AgeManning added this to the Public Testnet milestone Dec 4, 2019
paulhauner and others added 15 commits December 5, 2019 13:36
@paulhauner
Push attestation processed log down to dbg
d8a959e
@AgeManning
Correct math error, downgraded logs
6cee8b9
@AgeManning
Add RPC error handling and improved syncing code
d0dd2a0
@AgeManning
Add libp2p stream error handling and dropping of invalid peers
bbe4d46
@AgeManning
Lower logs
4fdc362
@AgeManning
Add discovery tweak
29470be
@AgeManning
Correct libp2p service locking
f97c73c
@AgeManning
Merge in latest master
4491155
@AgeManning
Handles peer disconnects for sync
7a8a92a
@AgeManning
Add logs downgrade discovery log
8617e3f
@paulhauner
Less fork choice (#679)
12880b3 
* Try merge in change to reduce fork choice calls

* Remove fork choice from process block

* Minor log fix

* Check successes > 0

* Fix failing beacon chain tests

* Fix re-org warnings

* Fix mistake in prev commit
@AgeManning
Range sync refactor
215ce75 
- Introduces `ChainCollection`
- Correct Disconnect node handling
- Removes duplicate code
@AgeManning
Various bug fixes
682cce4
@AgeManning
Remove unnecessary logs
b10bc6d
@AgeManning
Maintain syncing state in the transition from finalied to head
2266622
@AgeManning AgeManning merged commit 5853326 into master Dec 9, 2019
@AgeManning AgeManning mentioned this pull request Dec 9, 2019
Closed
@AgeManning AgeManning deleted the sync-load-balancing branch January 3, 2020 06:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@paulhauner paulhauner paulhauner approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

Public Testnet

Development

Successfully merging this pull request may close these issues.

3 participants

@AgeManning @paulhauner