-
Notifications
You must be signed in to change notification settings - Fork 425
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update Fast Sync configuration in Nethermind repository #3973
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
marcindsobczak
added a commit
that referenced
this pull request
May 13, 2022
* Updating Fast Sync config files (#3948) Co-authored-by: matilote <matilote@users.noreply.github.com> * Blockinfo metadata (#3918) * Add `FinalizationStatus` to `BlockInfo` * Add tests for block info decoder backwards compatibility * Change `FinalizationStatus` to `Metadata` and drop finalization check in decoder * Add flags to roundtrip test * fix setting finalization to false * remove non default `BlockInfoDecoder` initializatoin * Remove special rlp initialization for AuRa * skip locals init (#3921) * [Account Abstraction] Make default subscription give only RequestId (#3932) * make default to only give requestId * fixed tests * Fix build instructions * skip DeleteInvalidBlocks in ReadOnly mode (#3950) * Measure processing time in accurate way (#3952) * Updating Fast Sync config files (#3951) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3954) Co-authored-by: matilote <matilote@users.noreply.github.com> * update int256 submodule (#3955) * Updating Fast Sync config files (#3957) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3959) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3963) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3964) Co-authored-by: matilote <matilote@users.noreply.github.com> * is seal mandatory? (#3969) * Updating Fast Sync config files (#3966) Co-authored-by: matilote <matilote@users.noreply.github.com> * Refactor receipt recovery (#3956) * Fix recovering receipts only when needed, skip on sync * fix tests * fix subscriptions * simplify LogFinder * simplifications * Revert "simplifications" This reverts commit 461f81d. * Revert "simplify LogFinder" This reverts commit 134167a. * Revert "fix subscriptions" This reverts commit 3aa3674. * rise event only if useful * add ReceiptFinder to LogsSubscription in order to support recovery * simplify tests * update subscription tests * fix null pointer exception when asking by rpc for not existing receipts * fix test * unify and simplify Get() in PersistentReceiptStorage * cosmetics * fix build * force => forceRecoverSender * no more mixing abstraction layers in ReceiptsInserted * add flag Co-authored-by: lukasz.rozmej <lukasz.rozmej@gmail.com> * Updating Fast Sync config files (#3973) Co-authored-by: matilote <matilote@users.noreply.github.com> * Add priority peer option (for AA needs) (#3895) * basic implementation * add priority peers limit * fix tests * fix? * fix counter * fix * set priority = true only if number of max priority peers is positive * fix description * simplification * fix tests * add description in ISyncPeerPool * cleaning * add logs for debugging * fix SyncPeerPool constructor * add tests * increase number of priority peers instead of resetting it * use counter for PriorityPeerCount * fix test * add tests for incrementing/decrementing * override instead of new * simplify * Updating Fast Sync config files (#3976) Co-authored-by: matilote <matilote@users.noreply.github.com> * Fix/sync peer timer (#3977) * fix SyncPeerPool timer Initialize * fix StateSyncFeed timeout check * Updating Fast Sync config files (#3980) Co-authored-by: matilote <matilote@users.noreply.github.com> * Update enode list for Gnosis Chain (#3981) * Updating Fast Sync config files (#3984) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3986) Co-authored-by: matilote <matilote@users.noreply.github.com> * Fix receipts download (#3987) * fix receipts download * ReceiptsRecoveryResult instead of bool * add test * cosmetic * Updating Fast Sync config files (#3991) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3995) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating README.md with badges (#3993) * Updating README.md with badges * making minor corrections * Updating Fast Sync config files (#3997) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3998) Co-authored-by: matilote <matilote@users.noreply.github.com> * Snap Sync (#3840) * SnapSync networking - draft * Recreate State Tree from Account ranges * Recreate Storage from ranges * Rebase fix * Use StateSyncFeed for healing * Introduce TreeSync * Healing tests * Namespace refactor * Rebase fix * Draft SnapStorage (FlatDB) * Fix MultiSyncModeSelector tests * Account Range deserialization modified * Fix after rebase * Fix proof handling with the Geth structure * AccountRangeMessageSerializer serialize fixed * Test fix * For testing * testing * Follow the HEAD * Switch from Ranges to Healing (nodes) * Get Staorage Slots - fixed * Tests fixed * simple test fix * Additional tracing for concurrency bug testing * Draft concurrency fix * Apply proper paths to boundary tree * Tweaks for tests * Test draft ranges and switch to healing * Test fix * Test * test * Pivot Header * test * Retries * Rebase fix * Retry requests * test * test * test * Test * Prune dirty cache * Use separate TrieStore * Ignore expected root hash for storages * fix * Simple stitching - DoChildExist * Get Byte Codes - draft * Test * GetByteCodes retries * Test * Active requests counters * test * FIsnish GetRange phase * Test * Persist Ranges phase end * Stitching fixed and code refactor * Progress tracker fix * Test * test * test * test * Draft Refresh Account Storage Root * tests * Refresh Storage Root hash via GetTrieNodes * rebase fix * Tests fixed and MultiSyncModeSelector modified * fix * test * test * Report account refresh finished * test * progress bar * test * Fix * Handle bad peers * metrics * Fixes to bad peers handling * Remove snap capability when SnapSync is finished (#3982) * add capability and attach to event only if snap sync not finished yet * Bring back Lukasz fox to the sync selector * Fix tests * Finishing beta verstion - Clean up * Logging update * Logging * Remove allocations in SnapProviderHelper.FillBoundaryTree * Minor changes * low hanging fruit Co-authored-by: Marcin Sobczak <77129288+marcindsobczak@users.noreply.github.com> Co-authored-by: lukasz.rozmej <lukasz.rozmej@gmail.com> * Updating Fast Sync config files (#4012) Co-authored-by: matilote <matilote@users.noreply.github.com> * Add more snap sync metrics (#4005) * SnapSync networking - draft * Recreate State Tree from Account ranges * Recreate Storage from ranges * Rebase fix * Use StateSyncFeed for healing * Introduce TreeSync * Healing tests * Namespace refactor * Rebase fix * Draft SnapStorage (FlatDB) * Fix MultiSyncModeSelector tests * Account Range deserialization modified * Fix after rebase * Fix proof handling with the Geth structure * AccountRangeMessageSerializer serialize fixed * Test fix * For testing * testing * Follow the HEAD * Switch from Ranges to Healing (nodes) * Get Staorage Slots - fixed * Tests fixed * simple test fix * Additional tracing for concurrency bug testing * Draft concurrency fix * Apply proper paths to boundary tree * Tweaks for tests * Test draft ranges and switch to healing * Test fix * Test * test * Pivot Header * test * Retries * Rebase fix * Retry requests * test * test * test * Test * Prune dirty cache * Use separate TrieStore * Ignore expected root hash for storages * fix * Simple stitching - DoChildExist * Get Byte Codes - draft * Test * GetByteCodes retries * Test * Active requests counters * test * FIsnish GetRange phase * Test * Persist Ranges phase end * Stitching fixed and code refactor * Progress tracker fix * Test * test * test * test * Draft Refresh Account Storage Root * tests * Refresh Storage Root hash via GetTrieNodes * rebase fix * Tests fixed and MultiSyncModeSelector modified * fix * test * test * Report account refresh finished * test * progress bar * test * Fix * Handle bad peers * metrics * Fixes to bad peers handling * Remove snap capability when SnapSync is finished (#3982) * add capability and attach to event only if snap sync not finished yet * Bring back Lukasz fox to the sync selector * Fix tests * Finishing beta verstion - Clean up * Logging update * Logging * more metrics for SnapProtocol messages * add snap sync progress metric * fix after merge * one more post merge fix Co-authored-by: DCeleda <dceleda@hotmail.com> * remove pruned configs (again) (#4013) Removed in #3826 Resurected in #3781 * Updating Fast Sync config files (#4014) Co-authored-by: matilote <matilote@users.noreply.github.com> * Update arm64 library dependency to not depend on pmull instruction which is optional in Arm v8 implementations (#4015) * Updating Fast Sync config files (#4017) Co-authored-by: matilote <matilote@users.noreply.github.com> * post merge fixes * one more Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: matilote <matilote@users.noreply.github.com> Co-authored-by: Jorge Mederos <46798594+jmederosalvarado@users.noreply.github.com> Co-authored-by: Szymon Kulec <scooletz@gmail.com> Co-authored-by: Kristof Gazso <kristof.gazso@gmail.com> Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> Co-authored-by: Tanishq Jasoria <jasoriatanishq@gmail.com> Co-authored-by: varasev <33550681+varasev@users.noreply.github.com> Co-authored-by: Mateusz Jędrzejewski <33068017+matilote@users.noreply.github.com> Co-authored-by: Daniel Celeda <dceleda@hotmail.com>
LukaszRozmej
added a commit
that referenced
this pull request
May 26, 2022
* fix tests * more logs * small logs fixes * fix config * fix right best known pointer * more tests * fix? * fix test * add comments * test * return TD 0 on missing parent during TD backfill * change beacon header syncing conditions * experiment * experiment * experiments * fix config * Merge peer allocation strategy tests (#3902) * Add pre-merge test in MergePeerAllocationStrategyTests * Add allocate by speed post merge test Co-authored-by: hrugved <whrugved@gmail.com> * Authentication (#3894) * Jwt auth implementation using microsoft library * Todos removed * Old library removed * Authentication logging * Filename added * Additional log info Co-authored-by: Nikita Mescheryakov <root@nmescheryakov.com> * change logs * experiments * fix syncing? * refactoring TD usages * refactoring BestPeerStrategy * Implemented MergeBetterPeerStrategy * added it in initialization * fix validator initialization * fix tests * fix * fix one of the test * load beacon pivot from db * add block tree tests * wip forward syncing * clean up of old syncing code * logic for dangling chain * fix * fix * fix * experiment * experiment * updated pivots * more logging - experiments * experiment * experiment * logging * fix engine module tests * experiments * one more experiment * experiment * experiment * experiment * updated genesis for goerli_shadowfork.json * update TTD * code clean up and fix tests * updated goerli shadowfork configs * adjust config * Invalid parent hash test (#3927) Co-authored-by: Nikita Mescheryakov <root@nmescheryakov.com> * fix TerminalBlockHash in config * fix test logic * add new beacon pivot test * fix? * updated kiln configs * fix * fix config * updated goerli_shadowfork.cfg * Merge sync block tree tests (#3931) * Add test to BlockTreeTests.cs * Add sync test scenarios to BlockTreeTests.cs Co-authored-by: hrugved <whrugved@gmail.com> * experiment * revert experiments * mainnet_shadowfork * experiments with refreshing peers * more logging * update syncing logic * setup syncing tests * add logging on block insertion from cache * add sync tests * FastSyncLag * Fixed incorrect TD in some tests * add todos * fix merge conflicts * fix tests * fix build * PayloadPreparationService additional logs * fix config * Fix beacon pivot destination number report numbers * add logging to beacon headers * experiments * fix * fix pivot set condition * logging fixes * fix config * experiment * one more experiment * more logs * more logs * add comment * comment BeaconPivot after restart * commented pivot reloading * goerli_shadowfork.cfg pivots * fix again * fix missing TotalDifficulty in ProduceBranchV1 for tests * fixed invalid parent hash * fixed hive test? * reduced NLog.config * reverting logs && added goerli_shadowfork-4 * Plugin loader test fixed * updated TTD * fix StateSyncFeed * Latest block after reorg test (#3961) Co-authored-by: Nikita Mescheryakov <root@nmescheryakov.com> * workin on ChainLevelHelper * fix processing with state > 0 * one more fix? * add TD * add test for MarkChainAsProcessed * nlog for hive tests * more logs * experiments * experiments * experiments * experiments * updated NLog * Add mergeSynchronizer tests (#3975) Co-authored-by: hrugved <whrugved@gmail.com> * fix synchronizer test setup * change in tests * experiments * reduce logging, more experiments * Fix DropUselessPeers? * NLog * Add first MergeBlockDownloaderTests * experiments * NLog.config * fix NLog.config * experiments * fix * fix? * archive fix? * NLog.config * fix? * kiln terminalBlockNumber * cleanup NewPayloadV1Handler * change one logs * revert NLog.config * Experiments with forward syncing (#3992) * experiments * add logs * small cleanup * Add Eth2MergeFullWithoutTTD and fix failing tests (#3989) * Merge sync - merge better peer strategy tests (#3990) * Add MergeBetterPeerStrategyTests * add more MergeBetterPeerStrategyTests * Add more TotalDifficultyBasedBetterPeerStrategyTests * fix MarkChainAsProcessed_does_not_change_main_chain test * small changes * fix? * removed beacon full sync * NLog changes * fix? * log change & NLog revert * updated goerli_shadowfork.cfg * temp fix for Engine API initialization * updated configs * update fcu * marked failing BlockTreeTests as ignored * update MarkChainAsProcessed * updated mainnet_shadowfork configs * Update kiln.cfg (#4007) Update EthStats.Name to "Nethermind Kiln" * Fixes in forward syncing: finding terminal block, forkchoiceUpdatedHandler, hive configs and processing (#4004) * fixes? * refactored BlockchainProcessor * fixes * tests * fixing tests * fix test init * fix tests * changed log * NLog changes, hive config changes * cosmetic * Revert NLog.config * more logs * Nlog * logs + cleanup * cleanup things * logs fixes * fix for nimbus? * fixes? * fixes * tests * fix test init * fix tests * NLog changes, hive config changes * Blockinfo metadata (#3918) * Add `FinalizationStatus` to `BlockInfo` * Add tests for block info decoder backwards compatibility * Change `FinalizationStatus` to `Metadata` and drop finalization check in decoder * Add flags to roundtrip test * fix setting finalization to false * remove non default `BlockInfoDecoder` initializatoin * Remove special rlp initialization for AuRa * Use Metadata to reinitialize pointers on restart * fix tests * revert merge downloader changes * cleanup unused code * working on fcU changes * changes in error codes * changes in forkchoiceUpdated * added test for safeZeroHash * skip one test * Forkchoice updated changes (#4020) * fix MarkChainAsProcessed_does_not_change_main_chain test * update fcu * marked failing BlockTreeTests as ignored * update MarkChainAsProcessed * working on fcU changes * changes in error codes * changes in forkchoiceUpdated * added test for safeZeroHash * skip one test Co-authored-by: hrugved <whrugved@gmail.com> * add hive test debug logging * fix state provider (#4023) * fixes in restarts and one hive tests inconsistent forkchoiceState * ropsten config * fixes? * NLog.config temp changes * Merge master and resolve conflicts (#4024) * Updating Fast Sync config files (#3948) Co-authored-by: matilote <matilote@users.noreply.github.com> * Blockinfo metadata (#3918) * Add `FinalizationStatus` to `BlockInfo` * Add tests for block info decoder backwards compatibility * Change `FinalizationStatus` to `Metadata` and drop finalization check in decoder * Add flags to roundtrip test * fix setting finalization to false * remove non default `BlockInfoDecoder` initializatoin * Remove special rlp initialization for AuRa * skip locals init (#3921) * [Account Abstraction] Make default subscription give only RequestId (#3932) * make default to only give requestId * fixed tests * Fix build instructions * skip DeleteInvalidBlocks in ReadOnly mode (#3950) * Measure processing time in accurate way (#3952) * Updating Fast Sync config files (#3951) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3954) Co-authored-by: matilote <matilote@users.noreply.github.com> * update int256 submodule (#3955) * Updating Fast Sync config files (#3957) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3959) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3963) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3964) Co-authored-by: matilote <matilote@users.noreply.github.com> * is seal mandatory? (#3969) * Updating Fast Sync config files (#3966) Co-authored-by: matilote <matilote@users.noreply.github.com> * Refactor receipt recovery (#3956) * Fix recovering receipts only when needed, skip on sync * fix tests * fix subscriptions * simplify LogFinder * simplifications * Revert "simplifications" This reverts commit 461f81d. * Revert "simplify LogFinder" This reverts commit 134167a. * Revert "fix subscriptions" This reverts commit 3aa3674. * rise event only if useful * add ReceiptFinder to LogsSubscription in order to support recovery * simplify tests * update subscription tests * fix null pointer exception when asking by rpc for not existing receipts * fix test * unify and simplify Get() in PersistentReceiptStorage * cosmetics * fix build * force => forceRecoverSender * no more mixing abstraction layers in ReceiptsInserted * add flag Co-authored-by: lukasz.rozmej <lukasz.rozmej@gmail.com> * Updating Fast Sync config files (#3973) Co-authored-by: matilote <matilote@users.noreply.github.com> * Add priority peer option (for AA needs) (#3895) * basic implementation * add priority peers limit * fix tests * fix? * fix counter * fix * set priority = true only if number of max priority peers is positive * fix description * simplification * fix tests * add description in ISyncPeerPool * cleaning * add logs for debugging * fix SyncPeerPool constructor * add tests * increase number of priority peers instead of resetting it * use counter for PriorityPeerCount * fix test * add tests for incrementing/decrementing * override instead of new * simplify * Updating Fast Sync config files (#3976) Co-authored-by: matilote <matilote@users.noreply.github.com> * Fix/sync peer timer (#3977) * fix SyncPeerPool timer Initialize * fix StateSyncFeed timeout check * Updating Fast Sync config files (#3980) Co-authored-by: matilote <matilote@users.noreply.github.com> * Update enode list for Gnosis Chain (#3981) * Updating Fast Sync config files (#3984) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3986) Co-authored-by: matilote <matilote@users.noreply.github.com> * Fix receipts download (#3987) * fix receipts download * ReceiptsRecoveryResult instead of bool * add test * cosmetic * Updating Fast Sync config files (#3991) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3995) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating README.md with badges (#3993) * Updating README.md with badges * making minor corrections * Updating Fast Sync config files (#3997) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3998) Co-authored-by: matilote <matilote@users.noreply.github.com> * Snap Sync (#3840) * SnapSync networking - draft * Recreate State Tree from Account ranges * Recreate Storage from ranges * Rebase fix * Use StateSyncFeed for healing * Introduce TreeSync * Healing tests * Namespace refactor * Rebase fix * Draft SnapStorage (FlatDB) * Fix MultiSyncModeSelector tests * Account Range deserialization modified * Fix after rebase * Fix proof handling with the Geth structure * AccountRangeMessageSerializer serialize fixed * Test fix * For testing * testing * Follow the HEAD * Switch from Ranges to Healing (nodes) * Get Staorage Slots - fixed * Tests fixed * simple test fix * Additional tracing for concurrency bug testing * Draft concurrency fix * Apply proper paths to boundary tree * Tweaks for tests * Test draft ranges and switch to healing * Test fix * Test * test * Pivot Header * test * Retries * Rebase fix * Retry requests * test * test * test * Test * Prune dirty cache * Use separate TrieStore * Ignore expected root hash for storages * fix * Simple stitching - DoChildExist * Get Byte Codes - draft * Test * GetByteCodes retries * Test * Active requests counters * test * FIsnish GetRange phase * Test * Persist Ranges phase end * Stitching fixed and code refactor * Progress tracker fix * Test * test * test * test * Draft Refresh Account Storage Root * tests * Refresh Storage Root hash via GetTrieNodes * rebase fix * Tests fixed and MultiSyncModeSelector modified * fix * test * test * Report account refresh finished * test * progress bar * test * Fix * Handle bad peers * metrics * Fixes to bad peers handling * Remove snap capability when SnapSync is finished (#3982) * add capability and attach to event only if snap sync not finished yet * Bring back Lukasz fox to the sync selector * Fix tests * Finishing beta verstion - Clean up * Logging update * Logging * Remove allocations in SnapProviderHelper.FillBoundaryTree * Minor changes * low hanging fruit Co-authored-by: Marcin Sobczak <77129288+marcindsobczak@users.noreply.github.com> Co-authored-by: lukasz.rozmej <lukasz.rozmej@gmail.com> * Updating Fast Sync config files (#4012) Co-authored-by: matilote <matilote@users.noreply.github.com> * Add more snap sync metrics (#4005) * SnapSync networking - draft * Recreate State Tree from Account ranges * Recreate Storage from ranges * Rebase fix * Use StateSyncFeed for healing * Introduce TreeSync * Healing tests * Namespace refactor * Rebase fix * Draft SnapStorage (FlatDB) * Fix MultiSyncModeSelector tests * Account Range deserialization modified * Fix after rebase * Fix proof handling with the Geth structure * AccountRangeMessageSerializer serialize fixed * Test fix * For testing * testing * Follow the HEAD * Switch from Ranges to Healing (nodes) * Get Staorage Slots - fixed * Tests fixed * simple test fix * Additional tracing for concurrency bug testing * Draft concurrency fix * Apply proper paths to boundary tree * Tweaks for tests * Test draft ranges and switch to healing * Test fix * Test * test * Pivot Header * test * Retries * Rebase fix * Retry requests * test * test * test * Test * Prune dirty cache * Use separate TrieStore * Ignore expected root hash for storages * fix * Simple stitching - DoChildExist * Get Byte Codes - draft * Test * GetByteCodes retries * Test * Active requests counters * test * FIsnish GetRange phase * Test * Persist Ranges phase end * Stitching fixed and code refactor * Progress tracker fix * Test * test * test * test * Draft Refresh Account Storage Root * tests * Refresh Storage Root hash via GetTrieNodes * rebase fix * Tests fixed and MultiSyncModeSelector modified * fix * test * test * Report account refresh finished * test * progress bar * test * Fix * Handle bad peers * metrics * Fixes to bad peers handling * Remove snap capability when SnapSync is finished (#3982) * add capability and attach to event only if snap sync not finished yet * Bring back Lukasz fox to the sync selector * Fix tests * Finishing beta verstion - Clean up * Logging update * Logging * more metrics for SnapProtocol messages * add snap sync progress metric * fix after merge * one more post merge fix Co-authored-by: DCeleda <dceleda@hotmail.com> * remove pruned configs (again) (#4013) Removed in #3826 Resurected in #3781 * Updating Fast Sync config files (#4014) Co-authored-by: matilote <matilote@users.noreply.github.com> * Update arm64 library dependency to not depend on pmull instruction which is optional in Arm v8 implementations (#4015) * Updating Fast Sync config files (#4017) Co-authored-by: matilote <matilote@users.noreply.github.com> * post merge fixes * one more Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: matilote <matilote@users.noreply.github.com> Co-authored-by: Jorge Mederos <46798594+jmederosalvarado@users.noreply.github.com> Co-authored-by: Szymon Kulec <scooletz@gmail.com> Co-authored-by: Kristof Gazso <kristof.gazso@gmail.com> Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> Co-authored-by: Tanishq Jasoria <jasoriatanishq@gmail.com> Co-authored-by: varasev <33550681+varasev@users.noreply.github.com> Co-authored-by: Mateusz Jędrzejewski <33068017+matilote@users.noreply.github.com> Co-authored-by: Daniel Celeda <dceleda@hotmail.com> * post merge fix * fix build? * fix benchmarks * fix test * experiment * reverted logging changes * PoSSwitcher add logs * cosmetic * PoSSwitcher refactoring * NLog * fix? * experiments * experiments * experiments * fix? * experiment * revert logging changes * cosmetic * unignore passing tests * Ropsten TheMerge config (#4029) * Configs * cosmetic * cosmetic v2 * fix tests * fix tests * add beacon pointer test (#4035) * Kiln fix gossip policy (#4028) * fix gossip policy according to spec changes * refactor * fix test * refactor Co-authored-by: Marcin Sobczak <marcindsobczak@gmail.com> * Kiln finalized tag (#3871) * Add finalized tag and ForkChoiceUpdate method to block tree * Add unit test for block tree forkChoiceUpdate of finalized hash * Add finalized update test to EngineModuleTests.V1.cs * Update finalized block update test * Add safe tag and metadataDb to BlockTree.cs * Remove BlockConfirmationManager from ForkchoiceUpdatedV1Handler.cs * Fix failing EngineModuleTests * Add missing metadataDb fields to tests * add json rpc changes * fix BlockFinderExtensions.cs * update BlockFinderExtensions.cs * update BlockTreeTests.cs ForkChoiceUpdated function * Old blocks from canonical chain (#4047) * fix gossip policy according to spec changes * working on tests * refactor * fix test * refactor * changing the way of handling metadata * adjusted invalid status to the new spec * Ignoring old payloads in NewPayloadV1Handler * changed ConfiguredTerminalBlockNumber * FinalTotalDifficulty * fixed tests * preTTD pivot config * change workflow? * added test * update fcu TryGetBranch * removed beacon pivot exists check * remove ignored fcu * refactor fcu TryGetBranch * reverted workflow * fix * fix test Co-authored-by: Marcin Sobczak <marcindsobczak@gmail.com> Co-authored-by: hrugved <whrugved@gmail.com> * update IPoSSwitcher.cs fcu (#4046) * removed feerecipient configs * fix? * remove incorrect destinatnion * small changes * Kiln restarts + fixes (#4060) * ugly hack for restarts * more logs * fix? * fix build * fix? * fix? * fix * fix tests * small changes * Reverted NLog.config * one more condition * fix NLog.config * multisyncmodeselector logs * reverted NLog Co-authored-by: Marcin Sobczak <marcindsobczak@gmail.com> * fix launchSettings.json remove hive duplicate * fix workflow * Revert Ethash mining (#4062) * Revert Ethash mining * Don't revert useful things * small refactorings Co-authored-by: avalonche <avalonche@protonmail.com> Co-authored-by: hrugved <whrugved@gmail.com> Co-authored-by: Nikita Mescheryakov <deffrian@protonmail.com> Co-authored-by: Nikita Mescheryakov <root@nmescheryakov.com> Co-authored-by: lukasz.rozmej <lukasz.rozmej@gmail.com> Co-authored-by: Hrugved <45076119+Hrugved@users.noreply.github.com> Co-authored-by: SeaMonkey82 <seamonkey@seamonkey.tech> Co-authored-by: Jorge Mederos <46798594+jmederosalvarado@users.noreply.github.com> Co-authored-by: Marcin Sobczak <77129288+marcindsobczak@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: matilote <matilote@users.noreply.github.com> Co-authored-by: Szymon Kulec <scooletz@gmail.com> Co-authored-by: Kristof Gazso <kristof.gazso@gmail.com> Co-authored-by: Tanishq Jasoria <jasoriatanishq@gmail.com> Co-authored-by: varasev <33550681+varasev@users.noreply.github.com> Co-authored-by: Mateusz Jędrzejewski <33068017+matilote@users.noreply.github.com> Co-authored-by: Daniel Celeda <dceleda@hotmail.com> Co-authored-by: Marcin Sobczak <marcindsobczak@gmail.com>
LukaszRozmej
added a commit
that referenced
this pull request
Jun 14, 2022
* experiment * experiment * experiment * updated genesis for goerli_shadowfork.json * update TTD * code clean up and fix tests * updated goerli shadowfork configs * adjust config * Invalid parent hash test (#3927) Co-authored-by: Nikita Mescheryakov <root@nmescheryakov.com> * fix TerminalBlockHash in config * fix test logic * add new beacon pivot test * fix? * updated kiln configs * fix * fix config * updated goerli_shadowfork.cfg * Merge sync block tree tests (#3931) * Add test to BlockTreeTests.cs * Add sync test scenarios to BlockTreeTests.cs Co-authored-by: hrugved <whrugved@gmail.com> * experiment * revert experiments * mainnet_shadowfork * experiments with refreshing peers * more logging * update syncing logic * setup syncing tests * add logging on block insertion from cache * add sync tests * FastSyncLag * Fixed incorrect TD in some tests * add todos * fix merge conflicts * fix tests * Keep reward calcultor post merge if using AuRa * Add AuRaBlockProducerEnvFactory * Use BlockProducerEnvFactory in AuRa * fix build * PayloadPreparationService additional logs * fix config * Fix beacon pivot destination number report numbers * add logging to beacon headers * Add MergePlugin for GnosisChain * experiments * fix * fix pivot set condition * logging fixes * fix config * experiment * one more experiment * more logs * more logs * Revert "Use BlockProducerEnvFactory in AuRa" This reverts commit b892b69. * Revert "Add AuRaBlockProducerEnvFactory" This reverts commit 1e81ff3. * Override tx pool in merge plugin for aura * Rename GCMergePlugin to GnosisMergePlugin * Process empty blocks post merge * add comment * comment BeaconPivot after restart * commented pivot reloading * goerli_shadowfork.cfg pivots * fix again * fix missing TotalDifficulty in ProduceBranchV1 for tests * fixed invalid parent hash * fixed hive test? * reduced NLog.config * reverting logs && added goerli_shadowfork-4 * Plugin loader test fixed * updated TTD * Add per chain switch to merge plugin * Add tests for gnosis merge * Style * fix StateSyncFeed * Latest block after reorg test (#3961) Co-authored-by: Nikita Mescheryakov <root@nmescheryakov.com> * fix test with AuRa test blockchain * Skip some merge tests in aura merge specific test * add some comments in skipped tests * workin on ChainLevelHelper * fix processing with state > 0 * one more fix? * add TD * add test for MarkChainAsProcessed * nlog for hive tests * more logs * experiments * experiments * experiments * experiments * updated NLog * Add mergeSynchronizer tests (#3975) Co-authored-by: hrugved <whrugved@gmail.com> * fix synchronizer test setup * change in tests * experiments * reduce logging, more experiments * Fix DropUselessPeers? * NLog * Add first MergeBlockDownloaderTests * experiments * NLog.config * fix NLog.config * experiments * fix * fix? * archive fix? * NLog.config * fix? * kiln terminalBlockNumber * cleanup NewPayloadV1Handler * change one logs * revert NLog.config * Experiments with forward syncing (#3992) * experiments * add logs * small cleanup * Add Eth2MergeFullWithoutTTD and fix failing tests (#3989) * Merge sync - merge better peer strategy tests (#3990) * Add MergeBetterPeerStrategyTests * add more MergeBetterPeerStrategyTests * Add more TotalDifficultyBasedBetterPeerStrategyTests * fix MarkChainAsProcessed_does_not_change_main_chain test * small changes * fix? * removed beacon full sync * NLog changes * fix? * log change & NLog revert * updated goerli_shadowfork.cfg * temp fix for Engine API initialization * updated configs * update fcu * marked failing BlockTreeTests as ignored * update MarkChainAsProcessed * updated mainnet_shadowfork configs * Remove cyclic deps * Update kiln.cfg (#4007) Update EthStats.Name to "Nethermind Kiln" * Fix MergeHeaderValidator * Fixes in forward syncing: finding terminal block, forkchoiceUpdatedHandler, hive configs and processing (#4004) * fixes? * refactored BlockchainProcessor * fixes * tests * fixing tests * fix test init * fix tests * changed log * NLog changes, hive config changes * cosmetic * Revert NLog.config * more logs * Nlog * logs + cleanup * cleanup things * logs fixes * fix for nimbus? * Add log in MergePlugin * fixes? * fixes * tests * fix test init * fix tests * NLog changes, hive config changes * Blockinfo metadata (#3918) * Add `FinalizationStatus` to `BlockInfo` * Add tests for block info decoder backwards compatibility * Change `FinalizationStatus` to `Metadata` and drop finalization check in decoder * Add flags to roundtrip test * fix setting finalization to false * remove non default `BlockInfoDecoder` initializatoin * Remove special rlp initialization for AuRa * Use Metadata to reinitialize pointers on restart * fix tests * revert merge downloader changes * cleanup unused code * working on fcU changes * changes in error codes * changes in forkchoiceUpdated * added test for safeZeroHash * skip one test * Forkchoice updated changes (#4020) * fix MarkChainAsProcessed_does_not_change_main_chain test * update fcu * marked failing BlockTreeTests as ignored * update MarkChainAsProcessed * working on fcU changes * changes in error codes * changes in forkchoiceUpdated * added test for safeZeroHash * skip one test Co-authored-by: hrugved <whrugved@gmail.com> * Load different variants for MergePlugin * add hive test debug logging * fix state provider (#4023) * fixes in restarts and one hive tests inconsistent forkchoiceState * ropsten config * Fix build * fixes? * NLog.config temp changes * Merge master and resolve conflicts (#4024) * Updating Fast Sync config files (#3948) Co-authored-by: matilote <matilote@users.noreply.github.com> * Blockinfo metadata (#3918) * Add `FinalizationStatus` to `BlockInfo` * Add tests for block info decoder backwards compatibility * Change `FinalizationStatus` to `Metadata` and drop finalization check in decoder * Add flags to roundtrip test * fix setting finalization to false * remove non default `BlockInfoDecoder` initializatoin * Remove special rlp initialization for AuRa * skip locals init (#3921) * [Account Abstraction] Make default subscription give only RequestId (#3932) * make default to only give requestId * fixed tests * Fix build instructions * skip DeleteInvalidBlocks in ReadOnly mode (#3950) * Measure processing time in accurate way (#3952) * Updating Fast Sync config files (#3951) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3954) Co-authored-by: matilote <matilote@users.noreply.github.com> * update int256 submodule (#3955) * Updating Fast Sync config files (#3957) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3959) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3963) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3964) Co-authored-by: matilote <matilote@users.noreply.github.com> * is seal mandatory? (#3969) * Updating Fast Sync config files (#3966) Co-authored-by: matilote <matilote@users.noreply.github.com> * Refactor receipt recovery (#3956) * Fix recovering receipts only when needed, skip on sync * fix tests * fix subscriptions * simplify LogFinder * simplifications * Revert "simplifications" This reverts commit 461f81d. * Revert "simplify LogFinder" This reverts commit 134167a. * Revert "fix subscriptions" This reverts commit 3aa3674. * rise event only if useful * add ReceiptFinder to LogsSubscription in order to support recovery * simplify tests * update subscription tests * fix null pointer exception when asking by rpc for not existing receipts * fix test * unify and simplify Get() in PersistentReceiptStorage * cosmetics * fix build * force => forceRecoverSender * no more mixing abstraction layers in ReceiptsInserted * add flag Co-authored-by: lukasz.rozmej <lukasz.rozmej@gmail.com> * Updating Fast Sync config files (#3973) Co-authored-by: matilote <matilote@users.noreply.github.com> * Add priority peer option (for AA needs) (#3895) * basic implementation * add priority peers limit * fix tests * fix? * fix counter * fix * set priority = true only if number of max priority peers is positive * fix description * simplification * fix tests * add description in ISyncPeerPool * cleaning * add logs for debugging * fix SyncPeerPool constructor * add tests * increase number of priority peers instead of resetting it * use counter for PriorityPeerCount * fix test * add tests for incrementing/decrementing * override instead of new * simplify * Updating Fast Sync config files (#3976) Co-authored-by: matilote <matilote@users.noreply.github.com> * Fix/sync peer timer (#3977) * fix SyncPeerPool timer Initialize * fix StateSyncFeed timeout check * Updating Fast Sync config files (#3980) Co-authored-by: matilote <matilote@users.noreply.github.com> * Update enode list for Gnosis Chain (#3981) * Updating Fast Sync config files (#3984) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3986) Co-authored-by: matilote <matilote@users.noreply.github.com> * Fix receipts download (#3987) * fix receipts download * ReceiptsRecoveryResult instead of bool * add test * cosmetic * Updating Fast Sync config files (#3991) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3995) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating README.md with badges (#3993) * Updating README.md with badges * making minor corrections * Updating Fast Sync config files (#3997) Co-authored-by: matilote <matilote@users.noreply.github.com> * Updating Fast Sync config files (#3998) Co-authored-by: matilote <matilote@users.noreply.github.com> * Snap Sync (#3840) * SnapSync networking - draft * Recreate State Tree from Account ranges * Recreate Storage from ranges * Rebase fix * Use StateSyncFeed for healing * Introduce TreeSync * Healing tests * Namespace refactor * Rebase fix * Draft SnapStorage (FlatDB) * Fix MultiSyncModeSelector tests * Account Range deserialization modified * Fix after rebase * Fix proof handling with the Geth structure * AccountRangeMessageSerializer serialize fixed * Test fix * For testing * testing * Follow the HEAD * Switch from Ranges to Healing (nodes) * Get Staorage Slots - fixed * Tests fixed * simple test fix * Additional tracing for concurrency bug testing * Draft concurrency fix * Apply proper paths to boundary tree * Tweaks for tests * Test draft ranges and switch to healing * Test fix * Test * test * Pivot Header * test * Retries * Rebase fix * Retry requests * test * test * test * Test * Prune dirty cache * Use separate TrieStore * Ignore expected root hash for storages * fix * Simple stitching - DoChildExist * Get Byte Codes - draft * Test * GetByteCodes retries * Test * Active requests counters * test * FIsnish GetRange phase * Test * Persist Ranges phase end * Stitching fixed and code refactor * Progress tracker fix * Test * test * test * test * Draft Refresh Account Storage Root * tests * Refresh Storage Root hash via GetTrieNodes * rebase fix * Tests fixed and MultiSyncModeSelector modified * fix * test * test * Report account refresh finished * test * progress bar * test * Fix * Handle bad peers * metrics * Fixes to bad peers handling * Remove snap capability when SnapSync is finished (#3982) * add capability and attach to event only if snap sync not finished yet * Bring back Lukasz fox to the sync selector * Fix tests * Finishing beta verstion - Clean up * Logging update * Logging * Remove allocations in SnapProviderHelper.FillBoundaryTree * Minor changes * low hanging fruit Co-authored-by: Marcin Sobczak <77129288+marcindsobczak@users.noreply.github.com> Co-authored-by: lukasz.rozmej <lukasz.rozmej@gmail.com> * Updating Fast Sync config files (#4012) Co-authored-by: matilote <matilote@users.noreply.github.com> * Add more snap sync metrics (#4005) * SnapSync networking - draft * Recreate State Tree from Account ranges * Recreate Storage from ranges * Rebase fix * Use StateSyncFeed for healing * Introduce TreeSync * Healing tests * Namespace refactor * Rebase fix * Draft SnapStorage (FlatDB) * Fix MultiSyncModeSelector tests * Account Range deserialization modified * Fix after rebase * Fix proof handling with the Geth structure * AccountRangeMessageSerializer serialize fixed * Test fix * For testing * testing * Follow the HEAD * Switch from Ranges to Healing (nodes) * Get Staorage Slots - fixed * Tests fixed * simple test fix * Additional tracing for concurrency bug testing * Draft concurrency fix * Apply proper paths to boundary tree * Tweaks for tests * Test draft ranges and switch to healing * Test fix * Test * test * Pivot Header * test * Retries * Rebase fix * Retry requests * test * test * test * Test * Prune dirty cache * Use separate TrieStore * Ignore expected root hash for storages * fix * Simple stitching - DoChildExist * Get Byte Codes - draft * Test * GetByteCodes retries * Test * Active requests counters * test * FIsnish GetRange phase * Test * Persist Ranges phase end * Stitching fixed and code refactor * Progress tracker fix * Test * test * test * test * Draft Refresh Account Storage Root * tests * Refresh Storage Root hash via GetTrieNodes * rebase fix * Tests fixed and MultiSyncModeSelector modified * fix * test * test * Report account refresh finished * test * progress bar * test * Fix * Handle bad peers * metrics * Fixes to bad peers handling * Remove snap capability when SnapSync is finished (#3982) * add capability and attach to event only if snap sync not finished yet * Bring back Lukasz fox to the sync selector * Fix tests * Finishing beta verstion - Clean up * Logging update * Logging * more metrics for SnapProtocol messages * add snap sync progress metric * fix after merge * one more post merge fix Co-authored-by: DCeleda <dceleda@hotmail.com> * remove pruned configs (again) (#4013) Removed in #3826 Resurected in #3781 * Updating Fast Sync config files (#4014) Co-authored-by: matilote <matilote@users.noreply.github.com> * Update arm64 library dependency to not depend on pmull instruction which is optional in Arm v8 implementations (#4015) * Updating Fast Sync config files (#4017) Co-authored-by: matilote <matilote@users.noreply.github.com> * post merge fixes * one more Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: matilote <matilote@users.noreply.github.com> Co-authored-by: Jorge Mederos <46798594+jmederosalvarado@users.noreply.github.com> Co-authored-by: Szymon Kulec <scooletz@gmail.com> Co-authored-by: Kristof Gazso <kristof.gazso@gmail.com> Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> Co-authored-by: Tanishq Jasoria <jasoriatanishq@gmail.com> Co-authored-by: varasev <33550681+varasev@users.noreply.github.com> Co-authored-by: Mateusz Jędrzejewski <33068017+matilote@users.noreply.github.com> Co-authored-by: Daniel Celeda <dceleda@hotmail.com> * post merge fix * fix build? * fix benchmarks * fix test * experiment * reverted logging changes * PoSSwitcher add logs * cosmetic * PoSSwitcher refactoring * NLog * fix? * experiments * experiments * experiments * fix? * experiment * revert logging changes * cosmetic * unignore passing tests * Ropsten TheMerge config (#4029) * Configs * cosmetic * cosmetic v2 * fix tests * fix tests * add beacon pointer test (#4035) * Kiln fix gossip policy (#4028) * fix gossip policy according to spec changes * refactor * fix test * refactor Co-authored-by: Marcin Sobczak <marcindsobczak@gmail.com> * Kiln finalized tag (#3871) * Add finalized tag and ForkChoiceUpdate method to block tree * Add unit test for block tree forkChoiceUpdate of finalized hash * Add finalized update test to EngineModuleTests.V1.cs * Update finalized block update test * Add safe tag and metadataDb to BlockTree.cs * Remove BlockConfirmationManager from ForkchoiceUpdatedV1Handler.cs * Fix failing EngineModuleTests * Add missing metadataDb fields to tests * add json rpc changes * fix BlockFinderExtensions.cs * update BlockFinderExtensions.cs * update BlockTreeTests.cs ForkChoiceUpdated function * Old blocks from canonical chain (#4047) * fix gossip policy according to spec changes * working on tests * refactor * fix test * refactor * changing the way of handling metadata * adjusted invalid status to the new spec * Ignoring old payloads in NewPayloadV1Handler * changed ConfiguredTerminalBlockNumber * FinalTotalDifficulty * fixed tests * preTTD pivot config * change workflow? * added test * update fcu TryGetBranch * removed beacon pivot exists check * remove ignored fcu * refactor fcu TryGetBranch * reverted workflow * fix * fix test Co-authored-by: Marcin Sobczak <marcindsobczak@gmail.com> Co-authored-by: hrugved <whrugved@gmail.com> * update IPoSSwitcher.cs fcu (#4046) * removed feerecipient configs * fix? * Update submodules * Fix merge issue * Introduce IInitializationPlugin that can contain steps # Conflicts: # src/Nethermind/Nethermind.Consensus.AuRa/AuRaPlugin.cs * Change private fields to protected in InitializeBlockchainAuRa * Add custom init steps for aura merge plugin * Decide if custom init steps should be run based on config * Keep reward calculator source in aura custom init steps * Merge into * Fix merge errors * Fix merge (as in git merge) stuff :) * `AuRaBlockProcessor.BaseProcessBlock` -> `AuRaBlockProcessor.PostMergeProcessBlock` * fix merge issue in PostMergeBlockProducer * remove debug logs * add removed plugins to nethermind.runner * Use async await in GnosisMergePlugin.Init * refactor InitializeBlockchainAuRa.CreateBlockProcessor * fix more merge bugs :( * Rename Gnosis to AuRa :fingers_crosse: * Submodules back to master version * Document IInitializationPlugin * Refactor create tx source * Fix plugin order config * Fix merge plugin enabling mechanism * fix empty BlockProduction * fix merge plugin disabling from aura merge plugin * Properly document IInitializationPlugin * Add note stating that AuRaMergePlugin should always be before MergePlugin * Fix build errors * Disable MergeTest in AuRa which has problems with expected block hash * Add AuRaMerge tests to ci * Final refactorings * min change test Co-authored-by: MarekM25 <marekm2504@gmail.com> Co-authored-by: avalonche <avalonche@protonmail.com> Co-authored-by: Nikita Mescheryakov <root@nmescheryakov.com> Co-authored-by: hrugved <whrugved@gmail.com> Co-authored-by: lukasz.rozmej <lukasz.rozmej@gmail.com> Co-authored-by: Nikita Mescheryakov <deffrian@protonmail.com> Co-authored-by: Hrugved <45076119+Hrugved@users.noreply.github.com> Co-authored-by: SeaMonkey82 <seamonkey@seamonkey.tech> Co-authored-by: Marcin Sobczak <77129288+marcindsobczak@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: matilote <matilote@users.noreply.github.com> Co-authored-by: Szymon Kulec <scooletz@gmail.com> Co-authored-by: Kristof Gazso <kristof.gazso@gmail.com> Co-authored-by: Tanishq Jasoria <jasoriatanishq@gmail.com> Co-authored-by: varasev <33550681+varasev@users.noreply.github.com> Co-authored-by: Mateusz Jędrzejewski <33068017+matilote@users.noreply.github.com> Co-authored-by: Daniel Celeda <dceleda@hotmail.com> Co-authored-by: Marcin Sobczak <marcindsobczak@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Config files included:
Properties refreshed:
PivotNumber
PivotHash
PivotTotalDifficulty
Auto-generated by [create-pull-request][1]
[1]: https://github.com/peter-evans/create-pull-request