header: hardening syncing logic #334

Wondertan · 2022-01-10T10:50:29Z

Previously, we were syncing only once on Start and were catching up the chain by Pubsub topic. This PR changes that we can sync multiple times during runtime and if we missed some new Headers from PubSub topic, that will trigger sync again to catch-up. Also, this PR starts to verify new non-adjacent headers from PubSub and caches them to later be applied adjacently to avoid unnecessary request of already rcvd headers

Also fixes a variety of minor things.

Depends on #327
Closes: #242
Closes: #326

service/header/store.go

service/header/sync.go

CHANGELOG-PENDING.md

service/header/p2p_exchange.go

service/header/p2p_subscriber.go

service/header/store.go

service/header/sync.go

service/header/sync_test.go

liamsi

Generally looks very good. Left some comments and agree with @adlerjohn and most of @renaynay's comments.

service/header/sync.go

service/header/verify.go

Wondertan · 2022-01-20T15:14:47Z

I was changing network interfaces, hibernating, suspending for long, and still, my Light Node was syncing without needing to restart.

service/header/sync_ranges.go

Wondertan · 2022-01-21T20:49:04Z

Changes since your last reviews

Handling of unbonding/trusting period for headers
- Look at trustedHead
More refactorings and optimizations
Better logging
Better code comments

Wondertan · 2022-01-21T21:24:24Z

Tests are failing due to known #340.

renaynay

need to pass over it again tomorrow.

service/header/store.go

node/services/service.go

service/header/core_exchange.go

service/header/header_verify.go

service/header/local_exchange.go

service/header/sync_ranges.go

.gitignore

service/header/header_verify.go

service/header/interface.go

service/header/store.go

service/header/sync.go

Wondertan · 2022-01-27T13:08:51Z

There are two broken tests:

port already in use
- not sure what to do with it
syncer related one:
- this is type of flakeyness that will be fixed with younger bro PR, as it allows to wait for stuff instead of sleeps

Bidon15

Okay, let's go 👍

service/header/interface.go

renaynay

Almost ready.

service/header/interface.go

service/header/sync.go

Co-authored-by: rene <41963722+renaynay@users.noreply.github.com>

Wondertan · 2022-01-27T15:41:50Z

@renaynay, responded

renaynay · 2022-01-27T17:23:52Z

@Wondertan sir responded

Wondertan · 2022-01-27T18:32:29Z

@renaynay, time to approve the behemoth beast

Wondertan · 2022-01-28T10:43:33Z

One of the suggestions broke the linter and I had to fix it, therefore I need reapprovals @Bidon15

Co-authored-by: rene <41963722+renaynay@users.noreply.github.com>

Wondertan · 2022-01-28T11:08:18Z

Broken test is a flakey one, but will be fixed by a younger bro

renaynay

LEMS (let's end my suffering)

Wondertan · 2022-01-28T13:28:05Z

Lol, no drama pls

liamsi

Looks behemoth to me :D

But seriously, Thanks for tackling this, thanks fort the thorough reviews and for addressing all feedback.

I really hope we can split PRs like this into smaller PRs in the future.

Wondertan force-pushed the hlib/sync-disconnects branch 4 times, most recently from aa88d6a to 2541ac2 Compare January 12, 2022 10:59

Wondertan changed the title ~~header: slightly rework syncing logic~~ header: hardening syncing logic Jan 12, 2022

Wondertan added a commit that referenced this pull request Jan 13, 2022

changelog: add entry for #334

973807f

Wondertan marked this pull request as ready for review January 13, 2022 17:52

Wondertan requested review from liamsi and renaynay as code owners January 13, 2022 17:52

adlerjohn reviewed Jan 13, 2022

View reviewed changes

service/header/store.go Outdated Show resolved Hide resolved

service/header/sync.go Outdated Show resolved Hide resolved

service/header/sync.go Outdated Show resolved Hide resolved

service/header/sync.go Outdated Show resolved Hide resolved

renaynay reviewed Jan 18, 2022

View reviewed changes

renaynay added the area:header Extended header label Jan 20, 2022

liamsi reviewed Jan 20, 2022

View reviewed changes

service/header/sync.go Outdated Show resolved Hide resolved

service/header/sync.go Outdated Show resolved Hide resolved

service/header/sync.go Show resolved Hide resolved

service/header/sync.go Show resolved Hide resolved

service/header/verify.go Outdated Show resolved Hide resolved

Wondertan added a commit that referenced this pull request Jan 20, 2022

changelog: add entry for #334

5fd8732

Wondertan force-pushed the hlib/sync-disconnects branch 3 times, most recently from f816e7c to 36caec9 Compare January 20, 2022 14:26

renaynay reviewed Jan 21, 2022

View reviewed changes

service/header/sync_ranges.go Outdated Show resolved Hide resolved

service/header/sync_ranges.go Outdated Show resolved Hide resolved

service/header/sync_ranges.go Outdated Show resolved Hide resolved

Wondertan force-pushed the hlib/sync-disconnects branch from 18775e6 to c614f5b Compare January 21, 2022 20:42

Wondertan requested review from renaynay and liamsi January 21, 2022 21:15

renaynay reviewed Jan 23, 2022

View reviewed changes

service/header/store.go Outdated Show resolved Hide resolved

Wondertan added a commit that referenced this pull request Jan 24, 2022

changelog: add entry for #334

6b49d31

Wondertan force-pushed the hlib/sync-disconnects branch from 8347ad0 to 318ecff Compare January 24, 2022 13:42

liamsi reviewed Jan 24, 2022

View reviewed changes

renaynay reviewed Jan 24, 2022

View reviewed changes

Wondertan added a commit that referenced this pull request Jan 25, 2022

changelog: add entry for #334

37ef84b

Wondertan added a commit that referenced this pull request Jan 27, 2022

changelog: add entry for #334

062f894

Bidon15 self-requested a review January 27, 2022 13:38

Bidon15 previously approved these changes Jan 27, 2022

View reviewed changes

renaynay reviewed Jan 27, 2022

View reviewed changes

service/header/interface.go Show resolved Hide resolved

renaynay reviewed Jan 27, 2022

View reviewed changes

service/header/interface.go Outdated Show resolved Hide resolved

service/header/interface.go Outdated Show resolved Hide resolved

service/header/sync.go Outdated Show resolved Hide resolved

service/header/sync.go Outdated Show resolved Hide resolved

Update service/header/sync.go

7eca78f

Co-authored-by: rene <41963722+renaynay@users.noreply.github.com>

Wondertan dismissed Bidon15’s stale review via 7eca78f January 27, 2022 15:38

Wondertan and others added 2 commits January 27, 2022 17:39

Update service/header/sync.go

37b604d

Co-authored-by: rene <41963722+renaynay@users.noreply.github.com>

Update service/header/interface.go

d2dd0a2

Co-authored-by: rene <41963722+renaynay@users.noreply.github.com>

Bidon15 previously approved these changes Jan 27, 2022

View reviewed changes

Wondertan dismissed Bidon15’s stale review via 094553a January 28, 2022 10:42

Wondertan force-pushed the hlib/sync-disconnects branch from f861652 to 094553a Compare January 28, 2022 10:42

Wondertan force-pushed the hlib/sync-disconnects branch from 094553a to 5eddee9 Compare January 28, 2022 10:44

Update service/header/interface.go

96280ba

Co-authored-by: rene <41963722+renaynay@users.noreply.github.com>

Wondertan force-pushed the hlib/sync-disconnects branch from 5eddee9 to 96280ba Compare January 28, 2022 10:45

renaynay approved these changes Jan 28, 2022

View reviewed changes

adlerjohn approved these changes Jan 28, 2022

View reviewed changes

liamsi approved these changes Jan 29, 2022

View reviewed changes

Wondertan merged commit 40c4c62 into main Jan 29, 2022

Wondertan deleted the hlib/sync-disconnects branch January 29, 2022 05:53

Wondertan mentioned this pull request Feb 2, 2022

service/header: Extract topic validator from sync.go into its own file #318

Closed

Wondertan mentioned this pull request Feb 16, 2022

service/header: Syncer should provide an API to access its state #450

Closed

renaynay mentioned this pull request Apr 26, 2022

header/store: unboding period handling to mitigate long-range attack #205

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

header: hardening syncing logic #334

header: hardening syncing logic #334

Wondertan commented Jan 10, 2022 •

edited

Loading

liamsi left a comment

Wondertan commented Jan 20, 2022

Wondertan commented Jan 21, 2022 •

edited

Loading

Wondertan commented Jan 21, 2022

renaynay left a comment

Wondertan commented Jan 27, 2022

Bidon15 left a comment

renaynay left a comment

Wondertan commented Jan 27, 2022

renaynay commented Jan 27, 2022

Wondertan commented Jan 27, 2022

Wondertan commented Jan 28, 2022

Wondertan commented Jan 28, 2022 •

edited

Loading

renaynay left a comment

Wondertan commented Jan 28, 2022

liamsi left a comment

header: hardening syncing logic #334

header: hardening syncing logic #334

Conversation

Wondertan commented Jan 10, 2022 • edited Loading

liamsi left a comment

Choose a reason for hiding this comment

Wondertan commented Jan 20, 2022

Wondertan commented Jan 21, 2022 • edited Loading

Wondertan commented Jan 21, 2022

renaynay left a comment

Choose a reason for hiding this comment

Wondertan commented Jan 27, 2022

Bidon15 left a comment

Choose a reason for hiding this comment

renaynay left a comment

Choose a reason for hiding this comment

Wondertan commented Jan 27, 2022

renaynay commented Jan 27, 2022

Wondertan commented Jan 27, 2022

Wondertan commented Jan 28, 2022

Wondertan commented Jan 28, 2022 • edited Loading

renaynay left a comment

Choose a reason for hiding this comment

Wondertan commented Jan 28, 2022

liamsi left a comment

Choose a reason for hiding this comment

Wondertan commented Jan 10, 2022 •

edited

Loading

Wondertan commented Jan 21, 2022 •

edited

Loading

Wondertan commented Jan 28, 2022 •

edited

Loading