Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lodestar gossip queues to wrap processRpcMessage() #3554

Merged
merged 3 commits into from
Dec 28, 2021
Merged

Lodestar gossip queues to wrap processRpcMessage() #3554

merged 3 commits into from
Dec 28, 2021

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Dec 27, 2021 •
edited

Motivation

  • When we subscribe to all subnets (or the node have so many connected validators), we waste time uncompressing messages, getting message id and drop it in the end

Description

  • Lodestar gossip queues to wrap processRpcMessage() function instead of validate() function

Closes #3553

Test Result with subscribeAllSubnets flag
This shows significant improvement with subscribeAllSubnets flag and 2 contabo nodes (contabo-19/master vs contabo-20/this-branch)

  • Master cannot maintain a good number of peer

Screen Shot 2021-12-27 at 16 57 09

  • This branch can

Screen Shot 2021-12-27 at 16 57 35

  • Master can process up to 1024 attestation messages

Screen Shot 2021-12-27 at 16 58 58

  • This branch can process 4x number of messages

Screen Shot 2021-12-27 at 16 59 36

  • Master: gossip block is delayed a lot and node uses UnknownBlock sync frequently

Screen Shot 2021-12-27 at 17 01 07

  • This branch: gossip block is delayed mostly <= 10s (40s 1 time), UnknownBlock sync is not used

Screen Shot 2021-12-27 at 17 02 00

@codeclimate
Copy link

codeclimate bot commented Dec 27, 2021 •
edited

Code Climate has analyzed commit a01b025 and detected 1 issue on this pull request.

Here's the issue category breakdown:

Category Count
Complexity 1

View more on Code Climate.

@codecov
Copy link

codecov bot commented Dec 27, 2021 •
edited

Codecov Report

Merging #3554 (a01b025) into master (38dbc47) will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3554      +/-   ##
==========================================
- Coverage   37.57%   37.56%   -0.02%     
==========================================
  Files         311      311              
  Lines        8266     8272       +6     
  Branches     1276     1278       +2     
==========================================
+ Hits         3106     3107       +1     
- Misses       5012     5017       +5     
  Partials      148      148

@github-actions
Copy link
Contributor

github-actions bot commented Dec 27, 2021 •
edited

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 281fb4d Previous: 38dbc47 Ratio
BeaconState.hashTreeRoot - No change 656.00 ns/op 651.00 ns/op 1.01
BeaconState.hashTreeRoot - 1 full validator 147.68 us/op 149.78 us/op 0.99
BeaconState.hashTreeRoot - 32 full validator 2.3115 ms/op 2.2729 ms/op 1.02
BeaconState.hashTreeRoot - 512 full validator 31.043 ms/op 30.469 ms/op 1.02
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 146.06 us/op 150.46 us/op 0.97
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.4843 ms/op 2.4904 ms/op 1.00
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 32.863 ms/op 33.031 ms/op 0.99
BeaconState.hashTreeRoot - 1 balances 96.253 us/op 108.06 us/op 0.89
BeaconState.hashTreeRoot - 32 balances 895.79 us/op 924.24 us/op 0.97
BeaconState.hashTreeRoot - 512 balances 8.3520 ms/op 8.9987 ms/op 0.93
BeaconState.hashTreeRoot - 250000 balances 163.12 ms/op 159.46 ms/op 1.02
processSlot - 1 slots 55.878 us/op 55.878 us/op 1.00
processSlot - 32 slots 3.4430 ms/op 3.4143 ms/op 1.01
getCommitteeAssignments - req 1 vs - 250000 vc 6.2764 ms/op 6.4243 ms/op 0.98
getCommitteeAssignments - req 100 vs - 250000 vc 8.7210 ms/op 8.9435 ms/op 0.98
getCommitteeAssignments - req 1000 vs - 250000 vc 9.3262 ms/op 9.6771 ms/op 0.96
computeProposers - vc 250000 26.656 ms/op 25.424 ms/op 1.05
computeEpochShuffling - vc 250000 240.88 ms/op 219.77 ms/op 1.10
getNextSyncCommittee - vc 250000 403.51 ms/op 411.48 ms/op 0.98
altair processAttestation - 250000 vs - 7PWei normalcase 49.269 ms/op 57.110 ms/op 0.86
altair processAttestation - 250000 vs - 7PWei worstcase 64.786 ms/op 51.327 ms/op 1.26
altair processAttestation - setStatus - 1/6 committees join 13.578 ms/op 13.534 ms/op 1.00
altair processAttestation - setStatus - 1/3 committees join 26.975 ms/op 28.736 ms/op 0.94
altair processAttestation - setStatus - 1/2 committees join 43.275 ms/op 43.149 ms/op 1.00
altair processAttestation - setStatus - 2/3 committees join 54.806 ms/op 57.333 ms/op 0.96
altair processAttestation - setStatus - 4/5 committees join 68.233 ms/op 67.755 ms/op 1.01
altair processAttestation - setStatus - 100% committees join 84.332 ms/op 84.232 ms/op 1.00
altair processAttestation - updateEpochParticipants - 1/6 committees join 15.087 ms/op 14.360 ms/op 1.05
altair processAttestation - updateEpochParticipants - 1/3 committees join 31.853 ms/op 29.895 ms/op 1.07
altair processAttestation - updateEpochParticipants - 1/2 committees join 30.007 ms/op 36.833 ms/op 0.81
altair processAttestation - updateEpochParticipants - 2/3 committees join 29.365 ms/op 32.646 ms/op 0.90
altair processAttestation - updateEpochParticipants - 4/5 committees join 32.481 ms/op 33.559 ms/op 0.97
altair processAttestation - updateEpochParticipants - 100% committees join 39.905 ms/op 35.865 ms/op 1.11
altair processAttestation - updateAllStatus 25.312 ms/op 25.664 ms/op 0.99
altair processBlock - 250000 vs - 7PWei normalcase 51.017 ms/op 58.151 ms/op 0.88
altair processBlock - 250000 vs - 7PWei worstcase 136.00 ms/op 133.41 ms/op 1.02
altair processEpoch - mainnet_e81889 1.2677 s/op 1.2580 s/op 1.01
mainnet_e81889 - altair beforeProcessEpoch 306.47 ms/op 299.00 ms/op 1.02
mainnet_e81889 - altair processJustificationAndFinalization 66.860 us/op 76.742 us/op 0.87
mainnet_e81889 - altair processInactivityUpdates 21.974 ms/op 21.164 ms/op 1.04
mainnet_e81889 - altair processRewardsAndPenalties 269.77 ms/op 268.40 ms/op 1.01
mainnet_e81889 - altair processRegistryUpdates 10.931 us/op 12.344 us/op 0.89
mainnet_e81889 - altair processSlashings 2.7880 us/op 3.7070 us/op 0.75
mainnet_e81889 - altair processEth1DataReset 2.8540 us/op 3.8760 us/op 0.74
mainnet_e81889 - altair processEffectiveBalanceUpdates 13.664 ms/op 13.847 ms/op 0.99
mainnet_e81889 - altair processSlashingsReset 15.357 us/op 20.189 us/op 0.76
mainnet_e81889 - altair processRandaoMixesReset 19.633 us/op 23.555 us/op 0.83
mainnet_e81889 - altair processHistoricalRootsUpdate 4.3740 us/op 5.3710 us/op 0.81
mainnet_e81889 - altair processParticipationFlagUpdates 119.78 ms/op 122.94 ms/op 0.97
mainnet_e81889 - altair processSyncCommitteeUpdates 2.4970 us/op 2.6730 us/op 0.93
mainnet_e81889 - altair afterProcessEpoch 265.37 ms/op 263.91 ms/op 1.01
altair processInactivityUpdates - 250000 normalcase 84.991 ms/op 87.665 ms/op 0.97
altair processInactivityUpdates - 250000 worstcase 86.416 ms/op 87.314 ms/op 0.99
altair processParticipationFlagUpdates - 250000 anycase 110.15 ms/op 105.61 ms/op 1.04
altair processRewardsAndPenalties - 250000 normalcase 175.20 ms/op 175.98 ms/op 1.00
altair processRewardsAndPenalties - 250000 worstcase 177.08 ms/op 176.51 ms/op 1.00
altair processSyncCommitteeUpdates - 250000 443.95 ms/op 432.75 ms/op 1.03
Tree 40 250000 create 945.07 ms/op 1.0065 s/op 0.94
Tree 40 250000 get(125000) 457.66 ns/op 394.14 ns/op 1.16
Tree 40 250000 set(125000) 2.0821 us/op 2.1050 us/op 0.99
Tree 40 250000 toArray() 51.296 ms/op 44.025 ms/op 1.17
Tree 40 250000 iterate all - toArray() + loop 43.371 ms/op 51.649 ms/op 0.84
Tree 40 250000 iterate all - get(i) 139.05 ms/op 140.15 ms/op 0.99
MutableVector 250000 create 25.037 ms/op 25.600 ms/op 0.98
MutableVector 250000 get(125000) 18.035 ns/op 17.196 ns/op 1.05
MutableVector 250000 set(125000) 601.86 ns/op 582.86 ns/op 1.03
MutableVector 250000 toArray() 20.324 ms/op 12.216 ms/op 1.66
MutableVector 250000 iterate all - toArray() + loop 10.434 ms/op 10.266 ms/op 1.02
MutableVector 250000 iterate all - get(i) 4.1106 ms/op 3.7587 ms/op 1.09
Array 250000 create 6.1643 ms/op 6.2053 ms/op 0.99
Array 250000 clone - spread 1.9705 ms/op 2.0703 ms/op 0.95
Array 250000 get(125000) 1.0170 ns/op 1.0250 ns/op 0.99
Array 250000 set(125000) 1.0230 ns/op 1.0170 ns/op 1.01
Array 250000 iterate all - loop 201.50 us/op 201.37 us/op 1.00
aggregationBits - 2048 els - readonlyValues 239.17 us/op 245.79 us/op 0.97
aggregationBits - 2048 els - zipIndexesInBitList 40.500 us/op 41.879 us/op 0.97
regular array get 100000 times 80.910 us/op 80.864 us/op 1.00
wrappedArray get 100000 times 80.935 us/op 80.888 us/op 1.00
arrayWithProxy get 100000 times 34.828 ms/op 33.033 ms/op 1.05
ssz.Root.equals 1.4030 us/op 1.3210 us/op 1.06
ssz.Root.equals with valueOf() 1.5670 us/op 1.6120 us/op 0.97
byteArrayEquals with valueOf() 1.5460 us/op 1.5330 us/op 1.01
phase0 processBlock - 250000 vs - 7PWei normalcase 12.091 ms/op 12.724 ms/op 0.95
phase0 processBlock - 250000 vs - 7PWei worstcase 90.385 ms/op 89.980 ms/op 1.00
phase0 afterProcessEpoch - 250000 vs - 7PWei 246.72 ms/op 241.61 ms/op 1.02
phase0 beforeProcessEpoch - 250000 vs - 7PWei 630.97 ms/op 620.45 ms/op 1.02
phase0 processEpoch - mainnet_e58758 912.85 ms/op 919.67 ms/op 0.99
mainnet_e58758 - phase0 beforeProcessEpoch 561.59 ms/op 571.40 ms/op 0.98
mainnet_e58758 - phase0 processJustificationAndFinalization 71.381 us/op 71.375 us/op 1.00
mainnet_e58758 - phase0 processRewardsAndPenalties 194.10 ms/op 184.94 ms/op 1.05
mainnet_e58758 - phase0 processRegistryUpdates 48.537 us/op 47.089 us/op 1.03
mainnet_e58758 - phase0 processSlashings 3.8860 us/op 3.1300 us/op 1.24
mainnet_e58758 - phase0 processEth1DataReset 2.9490 us/op 3.1780 us/op 0.93
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 10.932 ms/op 11.819 ms/op 0.92
mainnet_e58758 - phase0 processSlashingsReset 15.303 us/op 17.097 us/op 0.90
mainnet_e58758 - phase0 processRandaoMixesReset 18.613 us/op 20.337 us/op 0.92
mainnet_e58758 - phase0 processHistoricalRootsUpdate 3.7010 us/op 3.6440 us/op 1.02
mainnet_e58758 - phase0 processParticipationRecordUpdates 15.243 us/op 15.325 us/op 0.99
mainnet_e58758 - phase0 afterProcessEpoch 226.35 ms/op 213.31 ms/op 1.06
phase0 processEffectiveBalanceUpdates - 250000 normalcase 13.114 ms/op 12.587 ms/op 1.04
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.6900 s/op 1.5506 s/op 1.09
phase0 processRegistryUpdates - 250000 normalcase 51.932 us/op 50.840 us/op 1.02
phase0 processRegistryUpdates - 250000 badcase_full_deposits 3.4452 ms/op 3.4205 ms/op 1.01
phase0 processRegistryUpdates - 250000 worstcase 0.5 2.1797 s/op 2.0647 s/op 1.06
phase0 getAttestationDeltas - 250000 normalcase 42.149 ms/op 45.323 ms/op 0.93
phase0 getAttestationDeltas - 250000 worstcase 45.063 ms/op 42.881 ms/op 1.05
phase0 processSlashings - 250000 worstcase 43.636 ms/op 43.456 ms/op 1.00
shuffle list - 16384 els 32.852 ms/op 17.042 ms/op 1.93
shuffle list - 250000 els 223.96 ms/op 215.83 ms/op 1.04
getEffectiveBalances - 250000 vs - 7PWei 13.698 ms/op 14.259 ms/op 0.96
computeDeltas 4.4120 ms/op 4.4441 ms/op 0.99
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.3035 ms/op 2.7884 ms/op 0.83
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 769.90 us/op 897.68 us/op 0.86
BLS verify - blst-native 2.2324 ms/op 2.2267 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 4.5753 ms/op 4.5709 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst-native 9.8708 ms/op 9.8640 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 35.863 ms/op 35.744 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 46.977 us/op 47.130 us/op 1.00
BLS aggregatePubkeys 128 - blst-native 183.24 us/op 184.25 us/op 0.99
getAttestationsForBlock 89.138 ms/op 97.976 ms/op 0.91
CheckpointStateCache - add get delete 18.496 us/op 18.632 us/op 0.99
validate gossip signedAggregateAndProof - struct 5.3568 ms/op 5.3767 ms/op 1.00
validate gossip signedAggregateAndProof - treeBacked 5.2982 ms/op 5.3115 ms/op 1.00
validate gossip attestation - struct 2.4967 ms/op 2.5063 ms/op 1.00
validate gossip attestation - treeBacked 2.5280 ms/op 2.5261 ms/op 1.00
Object access 1 prop 0.40800 ns/op 0.40600 ns/op 1.00
Map access 1 prop 0.33200 ns/op 0.33200 ns/op 1.00
Object get x1000 20.929 ns/op 20.659 ns/op 1.01
Map get x1000 1.2710 ns/op 1.2910 ns/op 0.98
Object set x1000 130.94 ns/op 133.10 ns/op 0.98
Map set x1000 79.472 ns/op 81.415 ns/op 0.98
Return object 10000 times 0.44570 ns/op 0.44480 ns/op 1.00
Throw Error 10000 times 7.0103 us/op 7.1307 us/op 0.98
enrSubnets - fastDeserialize 64 bits 1.4540 us/op 1.4980 us/op 0.97
enrSubnets - ssz BitVector 64 bits 19.630 us/op 19.635 us/op 1.00
enrSubnets - fastDeserialize 4 bits 533.00 ns/op 541.00 ns/op 0.99
enrSubnets - ssz BitVector 4 bits 3.3560 us/op 3.3610 us/op 1.00
RateTracker 1000000 limit, 1 obj count per request 216.76 ns/op 218.93 ns/op 0.99
RateTracker 1000000 limit, 2 obj count per request 162.66 ns/op 164.59 ns/op 0.99
RateTracker 1000000 limit, 4 obj count per request 134.68 ns/op 137.14 ns/op 0.98
RateTracker 1000000 limit, 8 obj count per request 121.94 ns/op 123.02 ns/op 0.99
RateTracker with prune 4.8400 us/op 4.9050 us/op 0.99

by benchmarkbot/action

@twoeths twoeths marked this pull request as ready for review December 27, 2021 10:30
g11tech
g11tech previously approved these changes Dec 27, 2021
View reviewed changes
@twoeths
Copy link
Contributor Author

twoeths commented Dec 28, 2021

The Sequence Diagram
processRpcMessage drawio

@dapplion dapplion merged commit fcbc459 into master Dec 28, 2021
@dapplion dapplion deleted the tuyen/wrap_processRpcMesasge_in_queue branch December 28, 2021 19:06
twoeths added a commit that referenced this pull request Feb 9, 2022
@twoeths
Revert "Lodestar gossip queues to wrap processRpcMessage() (#3554)"
25510f4 
This reverts commit fcbc459.
twoeths added a commit that referenced this pull request Feb 11, 2022
@twoeths
Revert "Lodestar gossip queues to wrap processRpcMessage() (#3554)"
8d54a70 
This reverts commit fcbc459.
This was referenced Feb 11, 2022
Merged
Closed
wemeetagain pushed a commit that referenced this pull request Feb 12, 2022
@twoeths
Revert "Lodestar gossip queues to wrap processRpcMessage() (#3554)" (#…
c6fe430 
…3745)

This reverts commit fcbc459.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion approved these changes

@g11tech g11tech g11tech left review comments

@mpetrunic mpetrunic Awaiting requested review from mpetrunic

@wemeetagain wemeetagain Awaiting requested review from wemeetagain

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Gossip queue: Wrap _processRpcMessage() function instead of validate() function
3 participants
@twoeths @dapplion @g11tech