[Config] Change default barriers of mainnet to 0 #6225

smartprogrammer93 · 2023-10-25T16:25:50Z

Changes

change default of barriers to 0 for mainnet without affecting previously synced nodes

Types of changes

What types of changes does your code introduce?

Bugfix (a non-breaking change that fixes an issue)
New feature (a non-breaking change that adds functionality)
Breaking change (a change that causes existing functionality not to work as expected)
Optimization
Refactoring
Documentation update
Build-related changes
Other: Description

Testing

Requires testing

Yes
No

If yes, did you write tests?

Yes
No

Notes on testing

@shashankshampi we need to run a test for this one where we sync a node with default configurations from master then update the node to this branch and check if it syncs the old data or not. if not, then all is good.

Documentation

Requires documentation update

Yes
No

Requires explanation in Release Notes

Yes
No

default to barriers on mainnet configuration changed to 0 to support the network. If you wish to use the old default barrier you need to set it manually.

MarekM25 · 2023-10-25T17:08:30Z

src/Nethermind/Nethermind.Synchronization/FastBlocks/BodiesSyncFeed.cs

+            || WithinOldBarrierDefault;
+
+        // This property was introduced when we switched defaults of barriers from 11052984 to 0 to not disturb existing node operators
+        private bool WithinOldBarrierDefault => _blockTree.LowestInsertedBodyNumber <= DepositContractBarrier


are you sure it will work correctly in every case?

We are covering 99 percent of cases. the only case that is not covered is if someone is syncing, and suddenly stopped the client at exactly around the old default, then unfortunately the sync will stop.

Do other network have barrier too?

Yeah, they do. The assumption is we only want to do that for mainnet

By default we only had barriers on mainnet

MarekM25 · 2023-10-25T17:09:27Z

src/Nethermind/Nethermind.Synchronization.Test/FastSync/ReceiptsSyncFeedTests.cs

@@ -231,6 +231,20 @@ public async Task When_configured_to_skip_receipts_then_finishes_immediately()
            _measuredProgressQueue.HasEnded.Should().BeTrue();
        }

+        [Test]
+        public async Task When_finished_sync_with_old_default_barrier_then_finishes_imedietely()


tests for ReceiptsSyncFeed, but no tests for bodies
no tests for every possible case

There are no tests for bodies in the first place. which was surprising tbh

Added more cases for receipts

deffrian · 2023-10-25T19:27:05Z

src/Nethermind/Nethermind.Runner/configs/mainnet.cfg

@@ -16,8 +16,6 @@
    "PivotHash": "0x24290b7c113efe59095ff1d3afc15efec735655e6d4065d24704759fa5c9a308",
    "PivotTotalDifficulty": "58750003716598352816469",
    "FastBlocks": true,
-    "AncientBodiesBarrier": 11052984,
-    "AncientReceiptsBarrier": 11052984,


Do we want to remove these values completely? If user decide to enable them he can set value close to the head and CL will not be able to sync deposit contract. Maybe we can introduce something like Sync.EnableBarrier?

I dont think the user will set the value from his mind. If the user wants to enable them, we can have something in the docs to guide?

Fair, but I think it's better option to have default value for this. User may decide to set it to current head

If that is desired, we can just make another config file, like mainet-barriers, for example

I'd better go with mainnet-light or something but not sure.

asdacap · 2023-10-25T23:39:44Z

You'll need to change SyncProgressResolver also, there is a duplicated logic there. See #6184 for an attempt to consolidate the logic.

smartprogrammer93 · 2023-11-04T14:29:37Z

Following tests were perfomed

Sync this branch fully on mainnet (all bodies and receipts synced)
Sync Master with barriers and then switch to this branch
Dont see any issues.

The current implementation covers the following case:

Node syncing from scretch = syncing all bodies and receipts
Node already synced with barriers = no change. It will not sync the rest even with the config change.
Node syncing from scratch restarted at any point, including at the previous barrier default = will continue sync no problem.
Node syncing from scratch with barriers configured = will stop at the barrier no problem
I believe this covers all cases.

…tarted.

kamilchodola · 2023-11-07T09:50:53Z

@smartprogrammer93 Let's make sure it is well tested since may break some setups.

@shashankshampi Important topic - please prepare a lot of scenarios including nodes synced on all version (before and after autopivot). Please also put a summary of testing here under PR before it would be merged - let's don't rush this one - it would be included in next version so we have a lot of time to make sure we are well prepared.

shashankshampi · 2023-11-14T05:21:37Z

The test is done and looks good to go.

shashankshampi

Tested this branch with couple of scenarios and looks stable to process ahead

MarekM25

It looks much better now. Added two comments to consider, but PR looks good

MarekM25 · 2023-11-23T19:04:33Z

src/Nethermind/Nethermind.Synchronization/FastBlocks/ReceiptsSyncFeed.cs

+                else if (_specProvider.ChainId == BlockchainIds.Mainnet)
+                {
+                    // Assume the receipts barrier was the previous defualt (deposit contract barrier) only for mainnet
+                    _barrierWhenStarted = DepositContractBarrier;


Edge case (not sure if worth handling): if the mainnet user with config barriers different than default deposits will upgrade Nethermind during receipts download it will stop receipts sync.

I would ignore case when someone upgrades during sync

MarekM25 · 2023-11-23T19:06:13Z

src/Nethermind/Nethermind.Synchronization/FastBlocks/BodiesSyncFeed.cs

@@ -72,8 +92,27 @@ public override void InitializeFeed()
                _barrier = _syncConfig.AncientBodiesBarrierCalc;
                if (_logger.IsInfo) _logger.Info($"Changed pivot in bodies sync. Now using pivot {_pivotNumber} and barrier {_barrier}");
                ResetSyncStatusList();
+                if (_blockTree.LowestInsertedBodyNumber is null || _blockTree.LowestInsertedBodyNumber >= _syncConfig.PivotNumberParsed)


Think about refactoring BodiesSyncFeed and ReceiptSyncFeed to reuse the same (similar) code

MarekM25

Looks good to me. Make sure that it was retested after all changes.

smartprogrammer93 · 2023-12-07T15:32:05Z

Retested and got green light from kamil. Merging

smartprogrammer93 requested review from LukaszRozmej, asdacap, MarekM25 and shashankshampi and removed request for LukaszRozmej October 25, 2023 16:25

MarekM25 requested changes Oct 25, 2023

View reviewed changes

deffrian reviewed Oct 25, 2023

View reviewed changes

smartprogrammer93 marked this pull request as draft October 27, 2023 11:47

smartprogrammer93 force-pushed the feature/change-mainnet-default-barriers-to-0 branch from 455cbd1 to a86d436 Compare November 3, 2023 13:10

smartprogrammer93 changed the base branch from master to refactor/encapsulate-sync-state November 3, 2023 13:11

smartprogrammer93 marked this pull request as ready for review November 3, 2023 17:04

smartprogrammer93 requested review from MarekM25, deffrian and kamilchodola November 4, 2023 14:34

asdacap force-pushed the refactor/encapsulate-sync-state branch from 189f2b0 to 44ab04c Compare November 6, 2023 03:42

Base automatically changed from refactor/encapsulate-sync-state to master November 6, 2023 10:36

smartprogrammer93 added 11 commits November 6, 2023 22:26

init

2665609

tests

545e0a7

format

a79085d

fix config tests

b0b89cc

cover more cases

dfd0196

Add same logic to SyncProgressResolver with tests

43b4a8d

wip metadatadb usage to detect with what configuration was the sync s…

ed1029a

…tarted.

add logic to ReceiptsSyncFeed

3ad31de

fix ReceiptsSyncFeed

8f8a782

apply fix

74cc0ea

remove useless using

ca21641

Fix bodies not finishing

e25b7d2

smartprogrammer93 force-pushed the feature/change-mainnet-default-barriers-to-0 branch from 7bf3b7d to e25b7d2 Compare November 6, 2023 19:29

asdacap approved these changes Nov 7, 2023

View reviewed changes

shashankshampi approved these changes Nov 14, 2023

View reviewed changes

MarekM25 approved these changes Nov 23, 2023

View reviewed changes

smartprogrammer93 added 2 commits November 29, 2023 00:12

Merge branch 'master' into feature/change-mainnet-default-barriers-to-0

245241a

refactor and minor fixes

93f380c

smartprogrammer93 requested a review from MarekM25 November 28, 2023 22:32

MarekM25 approved these changes Dec 4, 2023

View reviewed changes

smartprogrammer93 added the DO NOT MERGE label Dec 6, 2023

smartprogrammer93 added 3 commits December 6, 2023 15:17

Merge branch 'master' into feature/change-mainnet-default-barriers-to-0

810de42

Merge branch 'master' into feature/change-mainnet-default-barriers-to-0

63a8d16

fix test issue

91d261b

smartprogrammer93 removed the DO NOT MERGE label Dec 7, 2023

smartprogrammer93 merged commit 5a1eeec into master Dec 7, 2023
65 checks passed

smartprogrammer93 deleted the feature/change-mainnet-default-barriers-to-0 branch December 7, 2023 15:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Config] Change default barriers of mainnet to 0 #6225

[Config] Change default barriers of mainnet to 0 #6225

smartprogrammer93 commented Oct 25, 2023 •

edited

MarekM25 Oct 25, 2023

smartprogrammer93 Oct 25, 2023 •

edited

asdacap Oct 25, 2023

smartprogrammer93 Oct 26, 2023

LukaszRozmej Oct 26, 2023

MarekM25 Oct 25, 2023

smartprogrammer93 Oct 25, 2023

smartprogrammer93 Oct 25, 2023

deffrian Oct 25, 2023

smartprogrammer93 Oct 25, 2023

deffrian Oct 25, 2023

smartprogrammer93 Oct 25, 2023

kamilchodola Oct 30, 2023

asdacap commented Oct 25, 2023

smartprogrammer93 commented Nov 4, 2023

kamilchodola commented Nov 7, 2023

shashankshampi commented Nov 14, 2023

shashankshampi left a comment

MarekM25 left a comment

MarekM25 Nov 23, 2023

LukaszRozmej Nov 24, 2023

MarekM25 Nov 23, 2023

MarekM25 left a comment

smartprogrammer93 commented Dec 7, 2023

[Config] Change default barriers of mainnet to 0 #6225

[Config] Change default barriers of mainnet to 0 #6225

Conversation

smartprogrammer93 commented Oct 25, 2023 • edited

Changes

Types of changes

What types of changes does your code introduce?

Testing

Requires testing

If yes, did you write tests?

Notes on testing

Documentation

Requires documentation update

Requires explanation in Release Notes

Choose a reason for hiding this comment

smartprogrammer93 Oct 25, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

asdacap commented Oct 25, 2023

smartprogrammer93 commented Nov 4, 2023

kamilchodola commented Nov 7, 2023

shashankshampi commented Nov 14, 2023

shashankshampi left a comment

Choose a reason for hiding this comment

MarekM25 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MarekM25 left a comment

Choose a reason for hiding this comment

smartprogrammer93 commented Dec 7, 2023

smartprogrammer93 commented Oct 25, 2023 •

edited

smartprogrammer93 Oct 25, 2023 •

edited