Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set udp to enr #5001

Merged
merged 3 commits into from
Jan 14, 2023
Merged

Set udp to enr #5001

merged 3 commits into from
Jan 14, 2023

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Jan 13, 2023 •
edited
Loading

Motivation

Description

  • Root cause of memory issue is @chainsafe/snappy-stream, migrated to 5.1.2
  • When we set udp to enr, node receives so many incoming peers and libp2p prune peers randomly, so setting maxConnection (as our maxPeers) to tcp helps, thanks @dapplion

Closes #4623

TODOs

  • Verify the PR in feat1 group
  • Same work without setting maxConnection to tcp to verify the memory leak issue faster in nogroup 2 3

dapplion
dapplion approved these changes Jan 13, 2023
View reviewed changes
Copy link
Contributor

@dapplion dapplion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-approving pending testing. Thank you so much for the debugging effort @tuyennhv ❤️

@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: b9a8784 Previous: 0582f41 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 902.36 us/op 1.4909 ms/op 0.61
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 58.742 us/op 90.223 us/op 0.65
BLS verify - blst-native 2.1753 ms/op 2.2356 ms/op 0.97
BLS verifyMultipleSignatures 3 - blst-native 4.4841 ms/op 4.5647 ms/op 0.98
BLS verifyMultipleSignatures 8 - blst-native 9.6971 ms/op 9.8529 ms/op 0.98
BLS verifyMultipleSignatures 32 - blst-native 35.264 ms/op 35.702 ms/op 0.99
BLS aggregatePubkeys 32 - blst-native 46.460 us/op 47.037 us/op 0.99
BLS aggregatePubkeys 128 - blst-native 182.35 us/op 184.06 us/op 0.99
getAttestationsForBlock 76.313 ms/op 110.00 ms/op 0.69
isKnown best case - 1 super set check 498.00 ns/op 505.00 ns/op 0.99
isKnown normal case - 2 super set checks 488.00 ns/op 488.00 ns/op 1.00
isKnown worse case - 16 super set checks 493.00 ns/op 493.00 ns/op 1.00
CheckpointStateCache - add get delete 7.9600 us/op 10.031 us/op 0.79
validate gossip signedAggregateAndProof - struct 5.0144 ms/op 5.1320 ms/op 0.98
validate gossip attestation - struct 2.3570 ms/op 2.4126 ms/op 0.98
pickEth1Vote - no votes 2.4657 ms/op 2.6314 ms/op 0.94
pickEth1Vote - max votes 15.775 ms/op 21.599 ms/op 0.73
pickEth1Vote - Eth1Data hashTreeRoot value x2048 12.897 ms/op 14.506 ms/op 0.89
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 20.501 ms/op 25.741 ms/op 0.80
pickEth1Vote - Eth1Data fastSerialize value x2048 1.0827 ms/op 1.4996 ms/op 0.72
pickEth1Vote - Eth1Data fastSerialize tree x2048 9.4314 ms/op 12.527 ms/op 0.75
bytes32 toHexString 845.00 ns/op 1.2780 us/op 0.66
bytes32 Buffer.toString(hex) 654.00 ns/op 727.00 ns/op 0.90
bytes32 Buffer.toString(hex) from Uint8Array 978.00 ns/op 1.1110 us/op 0.88
bytes32 Buffer.toString(hex) + 0x 644.00 ns/op 704.00 ns/op 0.91
Object access 1 prop 0.28400 ns/op 0.34500 ns/op 0.82
Map access 1 prop 0.30000 ns/op 0.31500 ns/op 0.95
Object get x1000 10.758 ns/op 19.599 ns/op 0.55
Map get x1000 0.92700 ns/op 1.2010 ns/op 0.77
Object set x1000 68.003 ns/op 131.36 ns/op 0.52
Map set x1000 40.968 ns/op 83.650 ns/op 0.49
Return object 10000 times 0.43690 ns/op 0.45120 ns/op 0.97
Throw Error 10000 times 6.6480 us/op 8.3535 us/op 0.80
fastMsgIdFn sha256 / 200 bytes 4.8940 us/op 5.2080 us/op 0.94
fastMsgIdFn h32 xxhash / 200 bytes 537.00 ns/op 580.00 ns/op 0.93
fastMsgIdFn h64 xxhash / 200 bytes 708.00 ns/op 789.00 ns/op 0.90
fastMsgIdFn sha256 / 1000 bytes 15.720 us/op 16.338 us/op 0.96
fastMsgIdFn h32 xxhash / 1000 bytes 707.00 ns/op 768.00 ns/op 0.92
fastMsgIdFn h64 xxhash / 1000 bytes 787.00 ns/op 861.00 ns/op 0.91
fastMsgIdFn sha256 / 10000 bytes 136.30 us/op 140.88 us/op 0.97
fastMsgIdFn h32 xxhash / 10000 bytes 2.5950 us/op 2.7970 us/op 0.93
fastMsgIdFn h64 xxhash / 10000 bytes 1.7810 us/op 1.9520 us/op 0.91
enrSubnets - fastDeserialize 64 bits 2.2890 us/op 2.9590 us/op 0.77
enrSubnets - ssz BitVector 64 bits 892.00 ns/op 1.0140 us/op 0.88
enrSubnets - fastDeserialize 4 bits 290.00 ns/op 375.00 ns/op 0.77
enrSubnets - ssz BitVector 4 bits 897.00 ns/op 1.0220 us/op 0.88
prioritizePeers score -10:0 att 32-0.1 sync 2-0 132.17 us/op 217.32 us/op 0.61
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 177.41 us/op 239.01 us/op 0.74
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 260.36 us/op 368.24 us/op 0.71
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 424.93 us/op 631.59 us/op 0.67
prioritizePeers score 0:0 att 64-1 sync 4-1 474.32 us/op 729.23 us/op 0.65
array of 16000 items push then shift 51.600 us/op 3.2461 us/op 15.90
LinkedList of 16000 items push then shift 12.242 ns/op 20.688 ns/op 0.59
array of 16000 items push then pop 183.07 ns/op 247.45 ns/op 0.74
LinkedList of 16000 items push then pop 11.960 ns/op 19.833 ns/op 0.60
array of 24000 items push then shift 77.437 us/op 4.7553 us/op 16.28
LinkedList of 24000 items push then shift 12.509 ns/op 23.963 ns/op 0.52
array of 24000 items push then pop 184.07 ns/op 246.21 ns/op 0.75
LinkedList of 24000 items push then pop 11.864 ns/op 21.004 ns/op 0.56
intersect bitArray bitLen 8 21.669 ns/op 34.937 ns/op 0.62
intersect array and set length 8 120.54 ns/op 205.40 ns/op 0.59
intersect bitArray bitLen 128 70.171 ns/op 143.09 ns/op 0.49
intersect array and set length 128 1.6474 us/op 2.7459 us/op 0.60
Buffer.concat 32 items 5.1650 us/op 6.0900 us/op 0.85
Uint8Array.set 32 items 3.6710 us/op 4.8400 us/op 0.76
pass gossip attestations to forkchoice per slot 3.6254 ms/op 5.1030 ms/op 0.71
computeDeltas 4.5994 ms/op 7.3015 ms/op 0.63
computeProposerBoostScoreFromBalances 2.4954 ms/op 3.0988 ms/op 0.81
altair processAttestation - 250000 vs - 7PWei normalcase 3.3648 ms/op 4.6858 ms/op 0.72
altair processAttestation - 250000 vs - 7PWei worstcase 5.2194 ms/op 7.7649 ms/op 0.67
altair processAttestation - setStatus - 1/6 committees join 174.83 us/op 250.70 us/op 0.70
altair processAttestation - setStatus - 1/3 committees join 356.65 us/op 443.68 us/op 0.80
altair processAttestation - setStatus - 1/2 committees join 515.89 us/op 701.68 us/op 0.74
altair processAttestation - setStatus - 2/3 committees join 689.75 us/op 826.75 us/op 0.83
altair processAttestation - setStatus - 4/5 committees join 951.13 us/op 1.1992 ms/op 0.79
altair processAttestation - setStatus - 100% committees join 1.1416 ms/op 1.4221 ms/op 0.80
altair processBlock - 250000 vs - 7PWei normalcase 22.869 ms/op 31.587 ms/op 0.72
altair processBlock - 250000 vs - 7PWei normalcase hashState 34.657 ms/op 43.392 ms/op 0.80
altair processBlock - 250000 vs - 7PWei worstcase 80.138 ms/op 88.127 ms/op 0.91
altair processBlock - 250000 vs - 7PWei worstcase hashState 100.12 ms/op 113.40 ms/op 0.88
phase0 processBlock - 250000 vs - 7PWei normalcase 3.1236 ms/op 4.1637 ms/op 0.75
phase0 processBlock - 250000 vs - 7PWei worstcase 50.245 ms/op 56.038 ms/op 0.90
altair processEth1Data - 250000 vs - 7PWei normalcase 586.72 us/op 969.42 us/op 0.61
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 9.6810 us/op 14.498 us/op 0.67
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 35.024 us/op 35.699 us/op 0.98
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 14.441 us/op 18.351 us/op 0.79
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 11.019 us/op 16.221 us/op 0.68
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 148.71 us/op 133.14 us/op 1.12
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 995.74 us/op 1.2179 ms/op 0.82
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 1.2961 ms/op 1.6627 ms/op 0.78
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 1.2550 ms/op 1.6806 ms/op 0.75
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 4.0192 ms/op 5.0592 ms/op 0.79
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 2.5110 ms/op 3.3659 ms/op 0.75
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 7.1773 ms/op 8.1766 ms/op 0.88
Tree 40 250000 create 588.96 ms/op 830.47 ms/op 0.71
Tree 40 250000 get(125000) 265.34 ns/op 332.49 ns/op 0.80
Tree 40 250000 set(125000) 1.7294 us/op 2.3475 us/op 0.74
Tree 40 250000 toArray() 25.230 ms/op 35.981 ms/op 0.70
Tree 40 250000 iterate all - toArray() + loop 25.750 ms/op 35.961 ms/op 0.72
Tree 40 250000 iterate all - get(i) 101.32 ms/op 122.94 ms/op 0.82
MutableVector 250000 create 11.993 ms/op 17.709 ms/op 0.68
MutableVector 250000 get(125000) 10.765 ns/op 14.928 ns/op 0.72
MutableVector 250000 set(125000) 483.59 ns/op 690.09 ns/op 0.70
MutableVector 250000 toArray() 5.0636 ms/op 7.2194 ms/op 0.70
MutableVector 250000 iterate all - toArray() + loop 5.3355 ms/op 8.7802 ms/op 0.61
MutableVector 250000 iterate all - get(i) 3.1782 ms/op 4.1514 ms/op 0.77
Array 250000 create 5.3598 ms/op 6.9614 ms/op 0.77
Array 250000 clone - spread 1.8035 ms/op 2.4874 ms/op 0.73
Array 250000 get(125000) 0.92400 ns/op 1.1920 ns/op 0.78
Array 250000 set(125000) 0.92700 ns/op 1.2950 ns/op 0.72
Array 250000 iterate all - loop 150.92 us/op 224.45 us/op 0.67
effectiveBalanceIncrements clone Uint8Array 300000 31.673 us/op 90.445 us/op 0.35
effectiveBalanceIncrements clone MutableVector 300000 525.00 ns/op 768.00 ns/op 0.68
effectiveBalanceIncrements rw all Uint8Array 300000 244.05 us/op 303.30 us/op 0.80
effectiveBalanceIncrements rw all MutableVector 300000 130.69 ms/op 198.72 ms/op 0.66
phase0 afterProcessEpoch - 250000 vs - 7PWei 185.86 ms/op 222.41 ms/op 0.84
phase0 beforeProcessEpoch - 250000 vs - 7PWei 55.476 ms/op 85.271 ms/op 0.65
altair processEpoch - mainnet_e81889 548.95 ms/op 574.30 ms/op 0.96
mainnet_e81889 - altair beforeProcessEpoch 108.56 ms/op 174.57 ms/op 0.62
mainnet_e81889 - altair processJustificationAndFinalization 26.841 us/op 35.610 us/op 0.75
mainnet_e81889 - altair processInactivityUpdates 9.0106 ms/op 13.057 ms/op 0.69
mainnet_e81889 - altair processRewardsAndPenalties 75.449 ms/op 103.50 ms/op 0.73
mainnet_e81889 - altair processRegistryUpdates 2.1230 us/op 8.3310 us/op 0.25
mainnet_e81889 - altair processSlashings 723.00 ns/op 1.6180 us/op 0.45
mainnet_e81889 - altair processEth1DataReset 795.00 ns/op 1.9870 us/op 0.40
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.0281 ms/op 2.9451 ms/op 0.69
mainnet_e81889 - altair processSlashingsReset 5.0180 us/op 12.692 us/op 0.40
mainnet_e81889 - altair processRandaoMixesReset 5.8690 us/op 11.718 us/op 0.50
mainnet_e81889 - altair processHistoricalRootsUpdate 1.1420 us/op 1.9900 us/op 0.57
mainnet_e81889 - altair processParticipationFlagUpdates 3.1690 us/op 7.1470 us/op 0.44
mainnet_e81889 - altair processSyncCommitteeUpdates 986.00 ns/op 1.8240 us/op 0.54
mainnet_e81889 - altair afterProcessEpoch 197.09 ms/op 240.58 ms/op 0.82
phase0 processEpoch - mainnet_e58758 475.86 ms/op 630.53 ms/op 0.75
mainnet_e58758 - phase0 beforeProcessEpoch 183.47 ms/op 273.44 ms/op 0.67
mainnet_e58758 - phase0 processJustificationAndFinalization 26.621 us/op 34.286 us/op 0.78
mainnet_e58758 - phase0 processRewardsAndPenalties 100.94 ms/op 158.25 ms/op 0.64
mainnet_e58758 - phase0 processRegistryUpdates 10.072 us/op 18.600 us/op 0.54
mainnet_e58758 - phase0 processSlashings 783.00 ns/op 1.5190 us/op 0.52
mainnet_e58758 - phase0 processEth1DataReset 839.00 ns/op 1.5150 us/op 0.55
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.7701 ms/op 2.9608 ms/op 0.60
mainnet_e58758 - phase0 processSlashingsReset 4.4720 us/op 9.6350 us/op 0.46
mainnet_e58758 - phase0 processRandaoMixesReset 6.3830 us/op 12.625 us/op 0.51
mainnet_e58758 - phase0 processHistoricalRootsUpdate 699.00 ns/op 1.8840 us/op 0.37
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.7420 us/op 11.470 us/op 0.41
mainnet_e58758 - phase0 afterProcessEpoch 162.39 ms/op 193.46 ms/op 0.84
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.9234 ms/op 3.6107 ms/op 0.53
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.2355 ms/op 4.0560 ms/op 0.55
altair processInactivityUpdates - 250000 normalcase 31.355 ms/op 45.109 ms/op 0.70
altair processInactivityUpdates - 250000 worstcase 51.006 ms/op 45.642 ms/op 1.12
phase0 processRegistryUpdates - 250000 normalcase 9.5920 us/op 14.883 us/op 0.64
phase0 processRegistryUpdates - 250000 badcase_full_deposits 489.12 us/op 550.94 us/op 0.89
phase0 processRegistryUpdates - 250000 worstcase 0.5 170.89 ms/op 253.48 ms/op 0.67
altair processRewardsAndPenalties - 250000 normalcase 74.507 ms/op 102.06 ms/op 0.73
altair processRewardsAndPenalties - 250000 worstcase 99.189 ms/op 145.06 ms/op 0.68
phase0 getAttestationDeltas - 250000 normalcase 10.942 ms/op 15.039 ms/op 0.73
phase0 getAttestationDeltas - 250000 worstcase 11.207 ms/op 15.003 ms/op 0.75
phase0 processSlashings - 250000 worstcase 5.1589 ms/op 6.7688 ms/op 0.76
altair processSyncCommitteeUpdates - 250000 290.76 ms/op 345.75 ms/op 0.84
BeaconState.hashTreeRoot - No change 491.00 ns/op 512.00 ns/op 0.96
BeaconState.hashTreeRoot - 1 full validator 71.311 us/op 75.861 us/op 0.94
BeaconState.hashTreeRoot - 32 full validator 649.16 us/op 808.29 us/op 0.80
BeaconState.hashTreeRoot - 512 full validator 6.8228 ms/op 7.9607 ms/op 0.86
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 85.444 us/op 96.077 us/op 0.89
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2934 ms/op 1.4201 ms/op 0.91
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.610 ms/op 19.369 ms/op 0.86
BeaconState.hashTreeRoot - 1 balances 69.157 us/op 72.253 us/op 0.96
BeaconState.hashTreeRoot - 32 balances 605.18 us/op 671.30 us/op 0.90
BeaconState.hashTreeRoot - 512 balances 7.8861 ms/op 7.2890 ms/op 1.08
BeaconState.hashTreeRoot - 250000 balances 95.558 ms/op 106.94 ms/op 0.89
aggregationBits - 2048 els - zipIndexesInBitList 21.610 us/op 34.947 us/op 0.62
regular array get 100000 times 60.547 us/op 82.183 us/op 0.74
wrappedArray get 100000 times 60.562 us/op 83.727 us/op 0.72
arrayWithProxy get 100000 times 26.767 ms/op 33.828 ms/op 0.79
ssz.Root.equals 961.00 ns/op 1.3480 us/op 0.71
byteArrayEquals 952.00 ns/op 1.3260 us/op 0.72
shuffle list - 16384 els 11.492 ms/op 13.664 ms/op 0.84
shuffle list - 250000 els 167.89 ms/op 198.83 ms/op 0.84
processSlot - 1 slots 13.131 us/op 14.927 us/op 0.88
processSlot - 32 slots 1.9279 ms/op 2.1894 ms/op 0.88
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 349.74 us/op 402.74 us/op 0.87
getCommitteeAssignments - req 1 vs - 250000 vc 5.4605 ms/op 7.2722 ms/op 0.75
getCommitteeAssignments - req 100 vs - 250000 vc 7.9568 ms/op 8.6450 ms/op 0.92
getCommitteeAssignments - req 1000 vs - 250000 vc 8.5154 ms/op 9.4191 ms/op 0.90
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 7.4900 ns/op 11.840 ns/op 0.63
state getBlockRootAtSlot - 250000 vs - 7PWei 1.0395 us/op 1.4715 us/op 0.71
computeProposers - vc 250000 17.576 ms/op 20.101 ms/op 0.87
computeEpochShuffling - vc 250000 171.35 ms/op 203.62 ms/op 0.84
getNextSyncCommittee - vc 250000 291.89 ms/op 327.37 ms/op 0.89

by benchmarkbot/action

@wemeetagain wemeetagain mentioned this pull request Jan 13, 2023
Closed
wemeetagain
wemeetagain approved these changes Jan 13, 2023
View reviewed changes
Copy link
Member

@wemeetagain wemeetagain left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The metrics on this look really nice. 👍

Same work without setting maxConnection to tcp to verify the memory leak issue faster in nogroup 2 3

I recommend we do more experimentation in a future PR and merge this as-is

@twoeths twoeths marked this pull request as ready for review January 14, 2023 06:44
@twoeths twoeths requested a review from a team as a code owner January 14, 2023 06:44
@twoeths twoeths merged commit 5e756d7 into unstable Jan 14, 2023
@twoeths twoeths deleted the tuyen/set_udp_to_enr_2 branch January 14, 2023 06:44
@twoeths
Copy link
Contributor Author

twoeths commented Jan 14, 2023

CPU usage is reduced from 145% (as in unstable 1k ) to 120% - 130%

Screen Shot 2023-01-14 at 13 56 36

@twoeths twoeths mentioned this pull request Jan 14, 2023
Merged
@twoeths
Copy link
Contributor Author

twoeths commented Jan 15, 2023

works on nogroup 2 & 3 nodes:

Screen Shot 2023-01-15 at 16 11 30

Screen Shot 2023-01-15 at 16 12 18

@twoeths twoeths mentioned this pull request Jan 16, 2023
Closed
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

@dapplion dapplion dapplion approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Lodestar does not add UDP to ENR
3 participants
@twoeths @wemeetagain @dapplion