New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of blockchain synchronisation process #2384

Closed
karek314 opened this Issue Sep 10, 2018 · 2 comments

Comments

Projects
3 participants
@karek314
Copy link

karek314 commented Sep 10, 2018

I was looking at 1.0.0 synchronisation process and I can propose few improvements to the process.

  1. While syncing, usually block verification process halts for 2-6 seconds when new batch of blocks is acquired from network and even more if there is timeout issue 6-15s. Considering that there is over 7m blocks on mainnet right now, it's causing significant delay with reaching latest height. Why not introduce async networking task to accumulate blocks way ahead of verification process? Amount enough to prevent verification process from halting due refetching blocks from network.

  2. For some reason synchronisation process halts when incorrect block occurred on latest height (?)

[inf] 2018-09-08 22:28:23 | Verify->verifyBlock succeeded for block 7417561181081319467 at height 4931452.
[inf] 2018-09-08 22:28:23 | Block 7417561181081319467 loaded from: 89.40.125.40:8001 - height: 4931452
[inf] 2018-09-08 22:28:23 | Verify->verifyBlock succeeded for block 13371346198298708762 at height 4931453.
[inf] 2018-09-08 22:28:23 | Releasing enqueued broadcasts
[inf] 2018-09-08 22:28:23 | Queue empty
[inf] 2018-09-08 22:28:23 | Block 13371346198298708762 loaded from: 89.40.125.40:8001 - height: 4931453
[inf] 2018-09-08 22:28:23 | Verify->verifyBlock succeeded for block 6374672892190167342 at height 4931454.
[inf] 2018-09-08 22:28:24 | Block 6374672892190167342 loaded from: 89.40.125.40:8001 - height: 4931454
[inf] 2018-09-08 22:28:24 | Verify->verifyBlock succeeded for block 1406692377958485661 at height 4931455.
[inf] 2018-09-08 22:28:24 | Block 1406692377958485661 loaded from: 89.40.125.40:8001 - height: 4931455
[inf] 2018-09-08 22:28:24 | Looking for common block with: 47.93.19.241:8001
[inf] 2018-09-08 22:28:24 | Found common block: 1406692377958485661 with: 47.93.19.241:8001
[inf] 2018-09-08 22:28:24 | Loading blocks from: 47.93.19.241:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 46.28.204.68:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 18.188.59.171:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 13.78.12.96:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 13.230.14.157:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 106.14.189.130:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 13.124.192.90:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 47.92.167.12:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 209.222.98.250:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 47.105.74.40:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 13.228.53.209:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 13.250.103.141:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 77.234.128.230:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 13.125.252.0:8001
[inf] 2018-09-08 22:28:24 | Removed peer - 111.239.180.234:8001
[inf] 2018-09-08 22:28:26 | Removed peer - 47.93.19.241:8001
[ERR] 2018-09-08 22:28:26 | Error loading blocks: TimeoutError: Event response for 'rpc-request' timed out
[ERR] 2018-09-08 22:28:26 | Failed to load blocks from: 47.93.19.241:8001
[WRN] 2018-09-08 22:28:26 | Peer REMOVE error: code: 4210, message: Peer is not listed, description: undefined on peer 47.93.19.241:8001
[inf] 2018-09-08 22:28:26 | Broadhash consensus now 0 %
[inf] 2018-09-08 22:28:26 | Finished sync
[ERR] 2018-09-08 22:28:26 | Failed to process / verify unconfirmed transaction: 6721925551788835826 - Account does not have enough LSK: 15610359283786884938L balance: 6145.59374277
[inf] 2018-09-08 22:28:28 | Releasing enqueued broadcasts
[inf] 2018-09-08 22:28:28 | Queue empty
[WRN] 2018-09-08 22:28:30 | Discarded block that does not match with current chain: 11378173401586699330 height: 6991275 round: 69221 slot: 7233651 generator: 9c99976107b5d98e5669452392d912edf33c968e5832b52f2eedcd044b5cc2f2
[WRN] 2018-09-08 22:28:30 | Discarded block that does not match with current chain: 11378173401586699330 height: 6991275 round: 69221 slot: 7233651 generator: 9c99976107b5d98e5669452392d912edf33c968e5832b52f2eedcd044b5cc2f2
[WRN] 2018-09-08 22:28:30 | Discarded block that does not match with current chain: 11378173401586699330 height: 6991275 round: 69221 slot: 7233651 generator: 9c99976107b5d98e5669452392d912edf33c968e5832b52f2eedcd044b5cc2f2
[WRN] 2018-09-08 22:28:30 | Discarded block that does not match with current chain: 11378173401586699330 height: 6991275 round: 69221 slot: 7233651 generator: 9c99976107b5d98e5669452392d912edf33c968e5832b52f2eedcd044b5cc2f2
[WRN] 2018-09-08 22:28:30 | Discarded block that does not match with current chain: 11378173401586699330 height: 6991275 round: 69221 slot: 7233651 generator: 9c99976107b5d98e5669452392d912edf33c968e5832b52f2eedcd044b5cc2f2
[WRN] 2018-09-08 22:28:30 | Discarded block that does not match with current chain: 11378173401586699330 height: 6991275 round: 69221 slot: 7233651 generator: 9c99976107b5d98e5669452392d912edf33c968e5832b52f2eedcd044b5cc2f2
[WRN] 2018-09-08 22:28:30 | Discarded block that does not match with current chain: 11378173401586699330 height: 6991275 round: 69221 slot: 7233651 generator: 9c99976107b5d98e5669452392d912edf33c968e5832b52f2eedcd044b5cc2f2
[inf] 2018-09-08 22:28:34 | Releasing enqueued broadcasts
[inf] 2018-09-08 22:28:34 | Queue empty
[inf] 2018-09-08 22:28:36 | Starting sync
[inf] 2018-09-08 22:28:36 | Broadhash consensus now 0 %
[inf] 2018-09-08 22:28:36 | Looking for common block with: 51.38.41.33:8001
[inf] 2018-09-08 22:28:36 | Found common block: 1406692377958485661 with: 51.38.41.33:8001
[inf] 2018-09-08 22:28:36 | Loading blocks from: 51.38.41.33:8001
[inf] 2018-09-08 22:28:36 | Verify->verifyBlock succeeded for block 4427058323217238750 at height 4931456.
[inf] 2018-09-08 22:28:37 | Block 4427058323217238750 loaded from: 51.38.41.33:8001 - height: 4931456
[inf] 2018-09-08 22:28:37 | Verify->verifyBlock succeeded for block 14102728579845815760 at height 4931457.
[inf] 2018-09-08 22:28:37 | Block 14102728579845815760 loaded from: 51.38.41.33:8001 - height: 4931457
[inf] 2018-09-08 22:28:37 | Verify->verifyBlock succeeded for block 3278138967888859972 at height 4931458.
[inf] 2018-09-08 22:28:37 | Block 3278138967888859972 loaded from: 51.38.41.33:8001 - height: 4931458
[inf] 2018-09-08 22:28:37 | Verify->verifyBlock succeeded for block 4039235188847302655 at height 4931459.
[inf] 2018-09-08 22:28:38 | Block 4039235188847302655 loaded from: 51.38.41.33:8001 - height: 4931459
[inf] 2018-09-08 22:28:38 | Verify->verifyBlock succeeded for block 13787311908105607391 at height 4931460.
[inf] 2018-09-08 22:28:38 | Block 13787311908105607391 loaded from: 51.38.41.33:8001 - height: 4931460
[inf] 2018-09-08 22:28:38 | Verify->verifyBlock succeeded for block 6698863283491527380 at height 4931461.
[inf] 2018-09-08 22:28:38 | Block 6698863283491527380 loaded from: 51.38.41.33:8001 - height: 4931461
[inf] 2018-09-08 22:28:38 | Verify->verifyBlock succeeded for block 437120241675817136 at height 4931462.
[inf] 2018-09-08 22:28:38 | Block 437120241675817136 loaded from: 51.38.41.33:8001 - height: 4931462
[inf] 2018-09-08 22:28:38 | Verify->verifyBlock succeeded for block 12890385530354862481 at height 493
  1. Still entire process is using 25% or less of CPU. Moved to #2564
    Why not make these functions below executed asynchronously on background threads, await for completion and then proceed with result? Or even better, spawn threads with batched sets of blocks.
    for (const i in block.transactions) {

    Verify.prototype.verifyBlock = function(block) {

@diego-G diego-G added this to Product Backlog in Lisk Pipelines Sep 11, 2018

@MaciejBaj MaciejBaj moved this from Product Backlog to Sprint Candidate in Lisk Pipelines Oct 9, 2018

@diego-G diego-G assigned diego-G and unassigned diego-G Oct 15, 2018

@MaciejBaj MaciejBaj moved this from Sprint Candidate to Sprint Backlog in Lisk Pipelines Oct 16, 2018

@MaciejBaj MaciejBaj added this to To do in Version 1.4.0 via automation Oct 16, 2018

@MaciejBaj MaciejBaj added this to the Version 1.4.0 milestone Oct 25, 2018

@diego-G diego-G added bug *hard labels Nov 8, 2018

MaciejBaj added a commit that referenced this issue Nov 22, 2018

Merge pull request #2555 from LiskHQ/2384-improve_performance_synchro…
…nisation_process

Clean up loadBlocksFromNetwork function from loader module - Closes #2292 & #2384
@MaciejBaj

This comment has been minimized.

Copy link
Member

MaciejBaj commented Nov 22, 2018

Closed by #2555.

@MaciejBaj MaciejBaj closed this Nov 22, 2018

Version 1.4.0 automation moved this from To do to Done Nov 22, 2018

@MaciejBaj

This comment has been minimized.

Copy link
Member

MaciejBaj commented Nov 22, 2018

The verification will be continued in #2564.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment