Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tag relevant peer #4997

Merged
merged 2 commits into from
Jan 15, 2023
Merged

Tag relevant peer #4997

merged 2 commits into from
Jan 15, 2023

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Jan 10, 2023

Motivation

Description

  • Tag relevants peers in use, untag when we disconnect peers
  • Keep maxConnections option in libp2p's connection manager

found it from #4623

Test result

  • Deployed to feat1 and "Peer connection length` metric improved

Screen Shot 2023-01-10 at 11 15 40

@twoeths twoeths requested a review from a team as a code owner January 10, 2023 04:16
@github-actions
Copy link
Contributor

github-actions bot commented Jan 10, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 7a84abe Previous: da01673 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.2794 ms/op 1.2727 ms/op 1.01
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 81.126 us/op 77.710 us/op 1.04
BLS verify - blst-native 2.1641 ms/op 1.8629 ms/op 1.16
BLS verifyMultipleSignatures 3 - blst-native 4.4044 ms/op 3.8089 ms/op 1.16
BLS verifyMultipleSignatures 8 - blst-native 9.6574 ms/op 8.2140 ms/op 1.18
BLS verifyMultipleSignatures 32 - blst-native 34.732 ms/op 29.751 ms/op 1.17
BLS aggregatePubkeys 32 - blst-native 46.249 us/op 39.844 us/op 1.16
BLS aggregatePubkeys 128 - blst-native 178.15 us/op 154.12 us/op 1.16
getAttestationsForBlock 104.18 ms/op 90.314 ms/op 1.15
isKnown best case - 1 super set check 500.00 ns/op 433.00 ns/op 1.15
isKnown normal case - 2 super set checks 482.00 ns/op 416.00 ns/op 1.16
isKnown worse case - 16 super set checks 480.00 ns/op 416.00 ns/op 1.15
CheckpointStateCache - add get delete 10.079 us/op 8.5950 us/op 1.17
validate gossip signedAggregateAndProof - struct 5.0501 ms/op 4.2678 ms/op 1.18
validate gossip attestation - struct 2.3495 ms/op 2.0271 ms/op 1.16
pickEth1Vote - no votes 2.5584 ms/op 2.2044 ms/op 1.16
pickEth1Vote - max votes 20.662 ms/op 15.873 ms/op 1.30
pickEth1Vote - Eth1Data hashTreeRoot value x2048 14.018 ms/op 11.379 ms/op 1.23
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 24.424 ms/op 20.128 ms/op 1.21
pickEth1Vote - Eth1Data fastSerialize value x2048 1.3996 ms/op 1.1865 ms/op 1.18
pickEth1Vote - Eth1Data fastSerialize tree x2048 11.432 ms/op 9.1059 ms/op 1.26
bytes32 toHexString 1.2760 us/op 1.0340 us/op 1.23
bytes32 Buffer.toString(hex) 687.00 ns/op 610.00 ns/op 1.13
bytes32 Buffer.toString(hex) from Uint8Array 1.0950 us/op 908.00 ns/op 1.21
bytes32 Buffer.toString(hex) + 0x 706.00 ns/op 607.00 ns/op 1.16
Object access 1 prop 0.33100 ns/op 0.27800 ns/op 1.19
Map access 1 prop 0.33700 ns/op 0.25000 ns/op 1.35
Object get x1000 20.692 ns/op 15.967 ns/op 1.30
Map get x1000 1.1530 ns/op 0.99600 ns/op 1.16
Object set x1000 127.99 ns/op 103.89 ns/op 1.23
Map set x1000 82.496 ns/op 64.072 ns/op 1.29
Return object 10000 times 0.43510 ns/op 0.37400 ns/op 1.16
Throw Error 10000 times 8.0984 us/op 6.8984 us/op 1.17
fastMsgIdFn sha256 / 200 bytes 5.1890 us/op 4.2560 us/op 1.22
fastMsgIdFn h32 xxhash / 200 bytes 559.00 ns/op 481.00 ns/op 1.16
fastMsgIdFn h64 xxhash / 200 bytes 740.00 ns/op 636.00 ns/op 1.16
fastMsgIdFn sha256 / 1000 bytes 16.037 us/op 13.612 us/op 1.18
fastMsgIdFn h32 xxhash / 1000 bytes 727.00 ns/op 634.00 ns/op 1.15
fastMsgIdFn h64 xxhash / 1000 bytes 861.00 ns/op 720.00 ns/op 1.20
fastMsgIdFn sha256 / 10000 bytes 135.87 us/op 116.92 us/op 1.16
fastMsgIdFn h32 xxhash / 10000 bytes 2.8130 us/op 2.3290 us/op 1.21
fastMsgIdFn h64 xxhash / 10000 bytes 2.0310 us/op 1.6710 us/op 1.22
enrSubnets - fastDeserialize 64 bits 2.8000 us/op 2.2640 us/op 1.24
enrSubnets - ssz BitVector 64 bits 990.00 ns/op 797.00 ns/op 1.24
enrSubnets - fastDeserialize 4 bits 383.00 ns/op 305.00 ns/op 1.26
enrSubnets - ssz BitVector 4 bits 1.0050 us/op 813.00 ns/op 1.24
prioritizePeers score -10:0 att 32-0.1 sync 2-0 202.55 us/op 175.86 us/op 1.15
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 244.49 us/op 201.04 us/op 1.22
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 365.44 us/op 310.18 us/op 1.18
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 632.95 us/op 531.74 us/op 1.19
prioritizePeers score 0:0 att 64-1 sync 4-1 694.04 us/op 605.48 us/op 1.15
array of 16000 items push then shift 3.2566 us/op 2.7794 us/op 1.17
LinkedList of 16000 items push then shift 19.911 ns/op 17.256 ns/op 1.15
array of 16000 items push then pop 251.86 ns/op 215.64 ns/op 1.17
LinkedList of 16000 items push then pop 19.146 ns/op 16.469 ns/op 1.16
array of 24000 items push then shift 4.6521 us/op 3.9924 us/op 1.17
LinkedList of 24000 items push then shift 22.921 ns/op 20.142 ns/op 1.14
array of 24000 items push then pop 238.78 ns/op 211.25 ns/op 1.13
LinkedList of 24000 items push then pop 20.763 ns/op 17.876 ns/op 1.16
intersect bitArray bitLen 8 35.011 ns/op 28.975 ns/op 1.21
intersect array and set length 8 205.11 ns/op 161.53 ns/op 1.27
intersect bitArray bitLen 128 128.25 ns/op 118.16 ns/op 1.09
intersect array and set length 128 2.6762 us/op 2.2461 us/op 1.19
Buffer.concat 32 items 6.0000 us/op 5.1020 us/op 1.18
Uint8Array.set 32 items 4.6880 us/op 4.1490 us/op 1.13
pass gossip attestations to forkchoice per slot 4.8790 ms/op 4.2310 ms/op 1.15
computeDeltas 6.8455 ms/op 5.8375 ms/op 1.17
computeProposerBoostScoreFromBalances 3.0953 ms/op 2.8424 ms/op 1.09
altair processAttestation - 250000 vs - 7PWei normalcase 4.3832 ms/op 3.6652 ms/op 1.20
altair processAttestation - 250000 vs - 7PWei worstcase 7.3171 ms/op 6.4883 ms/op 1.13
altair processAttestation - setStatus - 1/6 committees join 248.20 us/op 202.72 us/op 1.22
altair processAttestation - setStatus - 1/3 committees join 466.78 us/op 390.37 us/op 1.20
altair processAttestation - setStatus - 1/2 committees join 654.85 us/op 551.92 us/op 1.19
altair processAttestation - setStatus - 2/3 committees join 847.22 us/op 709.49 us/op 1.19
altair processAttestation - setStatus - 4/5 committees join 1.1664 ms/op 983.40 us/op 1.19
altair processAttestation - setStatus - 100% committees join 1.3998 ms/op 1.1596 ms/op 1.21
altair processBlock - 250000 vs - 7PWei normalcase 28.744 ms/op 25.681 ms/op 1.12
altair processBlock - 250000 vs - 7PWei normalcase hashState 42.374 ms/op 39.970 ms/op 1.06
altair processBlock - 250000 vs - 7PWei worstcase 95.276 ms/op 78.162 ms/op 1.22
altair processBlock - 250000 vs - 7PWei worstcase hashState 110.94 ms/op 106.15 ms/op 1.05
phase0 processBlock - 250000 vs - 7PWei normalcase 4.2056 ms/op 3.3358 ms/op 1.26
phase0 processBlock - 250000 vs - 7PWei worstcase 54.632 ms/op 46.175 ms/op 1.18
altair processEth1Data - 250000 vs - 7PWei normalcase 986.21 us/op 769.72 us/op 1.28
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 13.231 us/op 13.410 us/op 0.99
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 32.397 us/op 33.787 us/op 0.96
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 17.810 us/op 20.833 us/op 0.85
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 15.013 us/op 10.432 us/op 1.44
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 125.51 us/op 150.63 us/op 0.83
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 1.2562 ms/op 1.0293 ms/op 1.22
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 1.7060 ms/op 1.3908 ms/op 1.23
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 1.6479 ms/op 1.3409 ms/op 1.23
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 5.7865 ms/op 4.3667 ms/op 1.33
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 3.4801 ms/op 3.0441 ms/op 1.14
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 8.7911 ms/op 7.0098 ms/op 1.25
Tree 40 250000 create 716.44 ms/op 641.21 ms/op 1.12
Tree 40 250000 get(125000) 335.42 ns/op 289.75 ns/op 1.16
Tree 40 250000 set(125000) 2.3064 us/op 2.0345 us/op 1.13
Tree 40 250000 toArray() 35.948 ms/op 31.152 ms/op 1.15
Tree 40 250000 iterate all - toArray() + loop 36.487 ms/op 31.244 ms/op 1.17
Tree 40 250000 iterate all - get(i) 123.44 ms/op 105.95 ms/op 1.17
MutableVector 250000 create 17.387 ms/op 15.880 ms/op 1.09
MutableVector 250000 get(125000) 15.206 ns/op 13.473 ns/op 1.13
MutableVector 250000 set(125000) 665.83 ns/op 609.19 ns/op 1.09
MutableVector 250000 toArray() 7.2573 ms/op 6.6184 ms/op 1.10
MutableVector 250000 iterate all - toArray() + loop 7.4734 ms/op 7.3112 ms/op 1.02
MutableVector 250000 iterate all - get(i) 4.1837 ms/op 3.5797 ms/op 1.17
Array 250000 create 7.1404 ms/op 5.8760 ms/op 1.22
Array 250000 clone - spread 2.9289 ms/op 2.1025 ms/op 1.39
Array 250000 get(125000) 1.3030 ns/op 0.91800 ns/op 1.42
Array 250000 set(125000) 1.2800 ns/op 0.92800 ns/op 1.38
Array 250000 iterate all - loop 199.71 us/op 167.92 us/op 1.19
effectiveBalanceIncrements clone Uint8Array 300000 86.681 us/op 63.780 us/op 1.36
effectiveBalanceIncrements clone MutableVector 300000 897.00 ns/op 588.00 ns/op 1.53
effectiveBalanceIncrements rw all Uint8Array 300000 292.82 us/op 252.78 us/op 1.16
effectiveBalanceIncrements rw all MutableVector 300000 218.01 ms/op 159.65 ms/op 1.37
phase0 afterProcessEpoch - 250000 vs - 7PWei 217.55 ms/op 187.55 ms/op 1.16
phase0 beforeProcessEpoch - 250000 vs - 7PWei 110.18 ms/op 70.960 ms/op 1.55
altair processEpoch - mainnet_e81889 655.13 ms/op 490.75 ms/op 1.33
mainnet_e81889 - altair beforeProcessEpoch 176.40 ms/op 136.72 ms/op 1.29
mainnet_e81889 - altair processJustificationAndFinalization 26.974 us/op 27.081 us/op 1.00
mainnet_e81889 - altair processInactivityUpdates 12.502 ms/op 10.364 ms/op 1.21
mainnet_e81889 - altair processRewardsAndPenalties 105.89 ms/op 87.952 ms/op 1.20
mainnet_e81889 - altair processRegistryUpdates 6.1580 us/op 4.6300 us/op 1.33
mainnet_e81889 - altair processSlashings 1.6690 us/op 1.0160 us/op 1.64
mainnet_e81889 - altair processEth1DataReset 1.6140 us/op 924.00 ns/op 1.75
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.6611 ms/op 2.4420 ms/op 1.09
mainnet_e81889 - altair processSlashingsReset 11.195 us/op 5.7410 us/op 1.95
mainnet_e81889 - altair processRandaoMixesReset 11.255 us/op 9.9020 us/op 1.14
mainnet_e81889 - altair processHistoricalRootsUpdate 1.9290 us/op 1.0150 us/op 1.90
mainnet_e81889 - altair processParticipationFlagUpdates 7.1260 us/op 3.8770 us/op 1.84
mainnet_e81889 - altair processSyncCommitteeUpdates 1.6020 us/op 1.0720 us/op 1.49
mainnet_e81889 - altair afterProcessEpoch 229.66 ms/op 195.85 ms/op 1.17
phase0 processEpoch - mainnet_e58758 597.48 ms/op 518.26 ms/op 1.15
mainnet_e58758 - phase0 beforeProcessEpoch 254.03 ms/op 219.42 ms/op 1.16
mainnet_e58758 - phase0 processJustificationAndFinalization 25.776 us/op 24.879 us/op 1.04
mainnet_e58758 - phase0 processRewardsAndPenalties 86.682 ms/op 133.41 ms/op 0.65
mainnet_e58758 - phase0 processRegistryUpdates 15.441 us/op 10.685 us/op 1.45
mainnet_e58758 - phase0 processSlashings 1.4920 us/op 1.0590 us/op 1.41
mainnet_e58758 - phase0 processEth1DataReset 1.6240 us/op 1.1070 us/op 1.47
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.2951 ms/op 1.9135 ms/op 1.20
mainnet_e58758 - phase0 processSlashingsReset 7.9070 us/op 7.7440 us/op 1.02
mainnet_e58758 - phase0 processRandaoMixesReset 10.725 us/op 8.1730 us/op 1.31
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.8560 us/op 981.00 ns/op 1.89
mainnet_e58758 - phase0 processParticipationRecordUpdates 10.166 us/op 5.3660 us/op 1.89
mainnet_e58758 - phase0 afterProcessEpoch 188.14 ms/op 159.90 ms/op 1.18
phase0 processEffectiveBalanceUpdates - 250000 normalcase 3.3053 ms/op 2.7939 ms/op 1.18
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.8287 ms/op 3.2654 ms/op 1.17
altair processInactivityUpdates - 250000 normalcase 44.245 ms/op 36.622 ms/op 1.21
altair processInactivityUpdates - 250000 worstcase 53.663 ms/op 47.447 ms/op 1.13
phase0 processRegistryUpdates - 250000 normalcase 14.030 us/op 10.574 us/op 1.33
phase0 processRegistryUpdates - 250000 badcase_full_deposits 501.38 us/op 436.63 us/op 1.15
phase0 processRegistryUpdates - 250000 worstcase 0.5 246.61 ms/op 205.93 ms/op 1.20
altair processRewardsAndPenalties - 250000 normalcase 143.88 ms/op 129.17 ms/op 1.11
altair processRewardsAndPenalties - 250000 worstcase 95.747 ms/op 83.981 ms/op 1.14
phase0 getAttestationDeltas - 250000 normalcase 14.750 ms/op 12.485 ms/op 1.18
phase0 getAttestationDeltas - 250000 worstcase 14.879 ms/op 12.559 ms/op 1.18
phase0 processSlashings - 250000 worstcase 6.3014 ms/op 5.6148 ms/op 1.12
altair processSyncCommitteeUpdates - 250000 312.57 ms/op 269.37 ms/op 1.16
BeaconState.hashTreeRoot - No change 567.00 ns/op 440.00 ns/op 1.29
BeaconState.hashTreeRoot - 1 full validator 69.205 us/op 63.702 us/op 1.09
BeaconState.hashTreeRoot - 32 full validator 1.1044 ms/op 561.13 us/op 1.97
BeaconState.hashTreeRoot - 512 full validator 7.4852 ms/op 7.9080 ms/op 0.95
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 88.085 us/op 77.338 us/op 1.14
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.3085 ms/op 1.1628 ms/op 1.13
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 18.220 ms/op 15.039 ms/op 1.21
BeaconState.hashTreeRoot - 1 balances 67.830 us/op 58.711 us/op 1.16
BeaconState.hashTreeRoot - 32 balances 712.66 us/op 561.42 us/op 1.27
BeaconState.hashTreeRoot - 512 balances 6.8787 ms/op 5.7201 ms/op 1.20
BeaconState.hashTreeRoot - 250000 balances 101.70 ms/op 90.058 ms/op 1.13
aggregationBits - 2048 els - zipIndexesInBitList 35.988 us/op 29.588 us/op 1.22
regular array get 100000 times 80.007 us/op 67.325 us/op 1.19
wrappedArray get 100000 times 79.221 us/op 67.302 us/op 1.18
arrayWithProxy get 100000 times 36.184 ms/op 33.114 ms/op 1.09
ssz.Root.equals 1.3010 us/op 1.1260 us/op 1.16
byteArrayEquals 1.2920 us/op 1.1100 us/op 1.16
shuffle list - 16384 els 13.515 ms/op 11.260 ms/op 1.20
shuffle list - 250000 els 197.36 ms/op 165.60 ms/op 1.19
processSlot - 1 slots 14.431 us/op 11.592 us/op 1.24
processSlot - 32 slots 2.0258 ms/op 1.6820 ms/op 1.20
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 388.66 us/op 331.04 us/op 1.17
getCommitteeAssignments - req 1 vs - 250000 vc 6.5954 ms/op 5.5956 ms/op 1.18
getCommitteeAssignments - req 100 vs - 250000 vc 8.9171 ms/op 7.6031 ms/op 1.17
getCommitteeAssignments - req 1000 vs - 250000 vc 9.5257 ms/op 8.1060 ms/op 1.18
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 11.740 ns/op 9.5900 ns/op 1.22
state getBlockRootAtSlot - 250000 vs - 7PWei 1.3937 us/op 1.1267 us/op 1.24
computeProposers - vc 250000 18.477 ms/op 16.159 ms/op 1.14
computeEpochShuffling - vc 250000 198.99 ms/op 170.53 ms/op 1.17
getNextSyncCommittee - vc 250000 309.37 ms/op 266.02 ms/op 1.16

by benchmarkbot/action

@maschad
Copy link
Contributor

maschad commented Jan 10, 2023

I think this duplicates the work of ChainSafe/js-libp2p-gossipsub#380

the plan outlined here is to have the option to configure priority peers to prevent the connection manager pruning connections to high-value peers.

in the interim we could just leave maxConnections and https://github.com/ChainSafe/lodestar/blob/unstable/packages/beacon-node/src/network/nodejs/util.ts#L73 as the default (both Infinity) to prevent pruning for now (until libp2p/js-libp2p#369 is released)

@twoeths twoeths marked this pull request as draft January 12, 2023 07:45
@wemeetagain
Copy link
Member

@maschad @tuyennhv

Was thinking about this recently, I think we can do both this PR and ChainSafe/js-libp2p-gossipsub#380

While they both tag peers, I think each has its own merit.

Tagging mesh peers in gossipsub is helpful for any users of gossipsub to avoid disconnecting these important peers.
Likewise, its ok for libp2p applications to tag important peers from their perspective.

If both of these PRs were implemented, if libp2p peer manager were to disconnect a peer, hopefully both tags would work together to give better behavior:

  1. first disconnect irrelevant peers
  2. next disconnect relevant but non-mesh peers
  3. next disconnect relevent mesh peers

@maschad
Copy link
Contributor

maschad commented Jan 12, 2023

@wemeetagain good point, I suppose as long as there isn't the prospect of a performance penalty as suggested here then I agree that having both tags would serve as a safeguard against this type of disconnection.

@twoeths
Copy link
Contributor Author

twoeths commented Jan 13, 2023

thanks @wemeetagain @maschad for your feeback

@dapplion also suggested to use maxConnection option in @libp2p/tcp as our maxPeers https://github.com/libp2p/js-libp2p-tcp/blob/b93f4c0a921923401264aa560ae259afc9430a93/src/index.ts#L38, so we limit our peers to 55 at the transport layer and libp2p never has to prune based on tags, so tagging peers is optional in this case, any thoughts?

we can also do both and add tcp.maxConnections feature flag

@twoeths
Copy link
Contributor Author

twoeths commented Jan 14, 2023

#5001 was merged which set maxConnection to tcp, however it only affect inbound connections, still need the tagging solution here @wemeetagain , otherwise we may have too many peers at some points

Screen Shot 2023-01-15 at 05 39 06

@twoeths twoeths marked this pull request as ready for review January 14, 2023 22:39
@twoeths twoeths merged commit 399559d into unstable Jan 15, 2023
@twoeths twoeths deleted the tuyen/tag_relevant_peer_2 branch January 15, 2023 08:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants