Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

EIP-4844: Dynamic topic lists for ReqResp and Gossip #4848

Merged
merged 4 commits into from
Dec 7, 2022
Merged

EIP-4844: Dynamic topic lists for ReqResp and Gossip #4848

merged 4 commits into from
Dec 7, 2022

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Dec 5, 2022

Motivation

Our current implementation of ReqResp protocols is static. Starting with EIP4844, we must register and un-register protocols at fork boundaries

Description

  • Implement dynamic list of ReqResp protocols + register / un-register logic at fork boundaries
  • De-duplicate dynamic list of gossip topics

@dapplion dapplion requested a review from a team as a code owner December 5, 2022 08:57
@dapplion dapplion changed the title Dynamic topic lists for ReqResp and Gossip EIP-4844: Dynamic topic lists for ReqResp and Gossip Dec 5, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Dec 5, 2022 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 8120b55 Previous: b7c7910 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.6370 ms/op 2.5877 ms/op 0.63
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 60.561 us/op 81.423 us/op 0.74
BLS verify - blst-native 2.1701 ms/op 1.8565 ms/op 1.17
BLS verifyMultipleSignatures 3 - blst-native 4.4956 ms/op 3.8159 ms/op 1.18
BLS verifyMultipleSignatures 8 - blst-native 9.7108 ms/op 8.2060 ms/op 1.18
BLS verifyMultipleSignatures 32 - blst-native 35.265 ms/op 29.733 ms/op 1.19
BLS aggregatePubkeys 32 - blst-native 46.479 us/op 39.504 us/op 1.18
BLS aggregatePubkeys 128 - blst-native 181.89 us/op 153.36 us/op 1.19
getAttestationsForBlock 77.144 ms/op 103.03 ms/op 0.75
isKnown best case - 1 super set check 484.00 ns/op 430.00 ns/op 1.13
isKnown normal case - 2 super set checks 474.00 ns/op 417.00 ns/op 1.14
isKnown worse case - 16 super set checks 470.00 ns/op 411.00 ns/op 1.14
CheckpointStateCache - add get delete 8.6600 us/op 9.5090 us/op 0.91
validate gossip signedAggregateAndProof - struct 5.0791 ms/op 4.2848 ms/op 1.19
validate gossip attestation - struct 2.3827 ms/op 2.0383 ms/op 1.17
pickEth1Vote - no votes 2.2612 ms/op 2.1611 ms/op 1.05
pickEth1Vote - max votes 17.716 ms/op 24.238 ms/op 0.73
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.855 ms/op 11.713 ms/op 1.01
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 19.022 ms/op 23.254 ms/op 0.82
pickEth1Vote - Eth1Data fastSerialize value x2048 1.4298 ms/op 1.6071 ms/op 0.89
pickEth1Vote - Eth1Data fastSerialize tree x2048 12.110 ms/op 15.907 ms/op 0.76
bytes32 toHexString 911.00 ns/op 1.2620 us/op 0.72
bytes32 Buffer.toString(hex) 704.00 ns/op 715.00 ns/op 0.98
bytes32 Buffer.toString(hex) from Uint8Array 975.00 ns/op 944.00 ns/op 1.03
bytes32 Buffer.toString(hex) + 0x 747.00 ns/op 714.00 ns/op 1.05
Object access 1 prop 0.34400 ns/op 0.38200 ns/op 0.90
Map access 1 prop 0.30700 ns/op 0.29300 ns/op 1.05
Object get x1000 10.957 ns/op 17.018 ns/op 0.64
Map get x1000 1.0150 ns/op 0.97800 ns/op 1.04
Object set x1000 72.055 ns/op 130.67 ns/op 0.55
Map set x1000 47.304 ns/op 79.062 ns/op 0.60
Return object 10000 times 0.43300 ns/op 0.36670 ns/op 1.18
Throw Error 10000 times 6.1709 us/op 5.9525 us/op 1.04
fastMsgIdFn sha256 / 200 bytes 4.8080 us/op 4.1650 us/op 1.15
fastMsgIdFn h32 xxhash / 200 bytes 539.00 ns/op 604.00 ns/op 0.89
fastMsgIdFn h64 xxhash / 200 bytes 765.00 ns/op 854.00 ns/op 0.90
fastMsgIdFn sha256 / 1000 bytes 15.379 us/op 13.251 us/op 1.16
fastMsgIdFn h32 xxhash / 1000 bytes 698.00 ns/op 742.00 ns/op 0.94
fastMsgIdFn h64 xxhash / 1000 bytes 902.00 ns/op 843.00 ns/op 1.07
fastMsgIdFn sha256 / 10000 bytes 134.99 us/op 112.17 us/op 1.20
fastMsgIdFn h32 xxhash / 10000 bytes 2.5740 us/op 2.4360 us/op 1.06
fastMsgIdFn h64 xxhash / 10000 bytes 1.9820 us/op 1.8000 us/op 1.10
enrSubnets - fastDeserialize 64 bits 2.6360 us/op 3.3100 us/op 0.80
enrSubnets - ssz BitVector 64 bits 827.00 ns/op 835.00 ns/op 0.99
enrSubnets - fastDeserialize 4 bits 360.00 ns/op 442.00 ns/op 0.81
enrSubnets - ssz BitVector 4 bits 793.00 ns/op 823.00 ns/op 0.96
prioritizePeers score -10:0 att 32-0.1 sync 2-0 83.614 us/op 103.37 us/op 0.81
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 127.73 us/op 146.94 us/op 0.87
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 197.88 us/op 250.52 us/op 0.79
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 343.22 us/op 393.58 us/op 0.87
prioritizePeers score 0:0 att 64-1 sync 4-1 417.68 us/op 469.36 us/op 0.89
RateTracker 1000000 limit, 1 obj count per request 186.82 ns/op 196.35 ns/op 0.95
RateTracker 1000000 limit, 2 obj count per request 135.18 ns/op 150.19 ns/op 0.90
RateTracker 1000000 limit, 4 obj count per request 110.29 ns/op 128.72 ns/op 0.86
RateTracker 1000000 limit, 8 obj count per request 97.185 ns/op 109.89 ns/op 0.88
RateTracker with prune 3.8370 us/op 4.8460 us/op 0.79
array of 16000 items push then shift 51.598 us/op 2.8415 us/op 18.16
LinkedList of 16000 items push then shift 12.085 ns/op 19.277 ns/op 0.63
array of 16000 items push then pop 201.17 ns/op 260.85 ns/op 0.77
LinkedList of 16000 items push then pop 11.992 ns/op 17.984 ns/op 0.67
array of 24000 items push then shift 77.366 us/op 3.9780 us/op 19.45
LinkedList of 24000 items push then shift 12.365 ns/op 23.653 ns/op 0.52
array of 24000 items push then pop 191.03 ns/op 211.55 ns/op 0.90
LinkedList of 24000 items push then pop 12.166 ns/op 20.118 ns/op 0.60
intersect bitArray bitLen 8 10.742 ns/op 11.766 ns/op 0.91
intersect array and set length 8 133.25 ns/op 185.99 ns/op 0.72
intersect bitArray bitLen 128 57.928 ns/op 62.042 ns/op 0.93
intersect array and set length 128 1.7266 us/op 2.3980 us/op 0.72
Buffer.concat 32 items 1.7380 ns/op 1.9770 ns/op 0.88
pass gossip attestations to forkchoice per slot 3.6027 ms/op 4.0803 ms/op 0.88
computeDeltas 4.7424 ms/op 6.2317 ms/op 0.76
computeProposerBoostScoreFromBalances 803.64 us/op 921.41 us/op 0.87
altair processAttestation - 250000 vs - 7PWei normalcase 3.2775 ms/op 4.5081 ms/op 0.73
altair processAttestation - 250000 vs - 7PWei worstcase 5.7258 ms/op 7.3138 ms/op 0.78
altair processAttestation - setStatus - 1/6 committees join 182.55 us/op 222.00 us/op 0.82
altair processAttestation - setStatus - 1/3 committees join 355.48 us/op 404.62 us/op 0.88
altair processAttestation - setStatus - 1/2 committees join 509.33 us/op 566.28 us/op 0.90
altair processAttestation - setStatus - 2/3 committees join 676.98 us/op 736.59 us/op 0.92
altair processAttestation - setStatus - 4/5 committees join 944.02 us/op 1.0146 ms/op 0.93
altair processAttestation - setStatus - 100% committees join 1.1212 ms/op 1.1839 ms/op 0.95
altair processBlock - 250000 vs - 7PWei normalcase 24.822 ms/op 29.120 ms/op 0.85
altair processBlock - 250000 vs - 7PWei normalcase hashState 39.556 ms/op 37.982 ms/op 1.04
altair processBlock - 250000 vs - 7PWei worstcase 80.146 ms/op 92.568 ms/op 0.87
altair processBlock - 250000 vs - 7PWei worstcase hashState 110.67 ms/op 99.039 ms/op 1.12
phase0 processBlock - 250000 vs - 7PWei normalcase 3.0934 ms/op 4.1782 ms/op 0.74
phase0 processBlock - 250000 vs - 7PWei worstcase 50.603 ms/op 48.293 ms/op 1.05
altair processEth1Data - 250000 vs - 7PWei normalcase 681.90 us/op 1.0606 ms/op 0.64
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.8240 us/op 9.2200 us/op 0.74
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 20.695 us/op 28.183 us/op 0.73
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 9.3090 us/op 12.031 us/op 0.77
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 6.7660 us/op 9.4450 us/op 0.72
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 90.165 us/op 104.62 us/op 0.86
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 903.05 us/op 1.0622 ms/op 0.85
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 141069 11.749 ms/op 15.718 ms/op 0.75
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 250000 21.049 ms/op 25.608 ms/op 0.82
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 250000 70.459 ms/op 88.093 ms/op 0.80
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 250000 38.229 ms/op 49.227 ms/op 0.78
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 250000 117.32 ms/op 136.89 ms/op 0.86
Tree 40 250000 create 689.73 ms/op 937.19 ms/op 0.74
Tree 40 250000 get(125000) 222.53 ns/op 301.47 ns/op 0.74
Tree 40 250000 set(125000) 2.0605 us/op 3.2061 us/op 0.64
Tree 40 250000 toArray() 25.771 ms/op 36.314 ms/op 0.71
Tree 40 250000 iterate all - toArray() + loop 26.491 ms/op 36.712 ms/op 0.72
Tree 40 250000 iterate all - get(i) 108.60 ms/op 118.45 ms/op 0.92
MutableVector 250000 create 12.427 ms/op 19.028 ms/op 0.65
MutableVector 250000 get(125000) 11.023 ns/op 14.792 ns/op 0.75
MutableVector 250000 set(125000) 555.24 ns/op 763.93 ns/op 0.73
MutableVector 250000 toArray() 5.8076 ms/op 8.4590 ms/op 0.69
MutableVector 250000 iterate all - toArray() + loop 5.9957 ms/op 8.6017 ms/op 0.70
MutableVector 250000 iterate all - get(i) 3.2454 ms/op 3.4432 ms/op 0.94
Array 250000 create 5.6722 ms/op 7.9429 ms/op 0.71
Array 250000 clone - spread 3.2052 ms/op 4.1764 ms/op 0.77
Array 250000 get(125000) 1.4380 ns/op 1.6680 ns/op 0.86
Array 250000 set(125000) 1.4430 ns/op 1.7050 ns/op 0.85
Array 250000 iterate all - loop 153.51 us/op 167.97 us/op 0.91
effectiveBalanceIncrements clone Uint8Array 300000 161.31 us/op 101.08 us/op 1.60
effectiveBalanceIncrements clone MutableVector 300000 685.00 ns/op 1.2870 us/op 0.53
effectiveBalanceIncrements rw all Uint8Array 300000 247.55 us/op 252.63 us/op 0.98
effectiveBalanceIncrements rw all MutableVector 300000 143.76 ms/op 241.39 ms/op 0.60
phase0 afterProcessEpoch - 250000 vs - 7PWei 189.14 ms/op 184.41 ms/op 1.03
phase0 beforeProcessEpoch - 250000 vs - 7PWei 58.168 ms/op 86.023 ms/op 0.68
altair processEpoch - mainnet_e81889 486.46 ms/op 605.39 ms/op 0.80
mainnet_e81889 - altair beforeProcessEpoch 116.87 ms/op 158.48 ms/op 0.74
mainnet_e81889 - altair processJustificationAndFinalization 16.213 us/op 31.467 us/op 0.52
mainnet_e81889 - altair processInactivityUpdates 8.7347 ms/op 12.448 ms/op 0.70
mainnet_e81889 - altair processRewardsAndPenalties 77.561 ms/op 99.155 ms/op 0.78
mainnet_e81889 - altair processRegistryUpdates 2.4830 us/op 3.9850 us/op 0.62
mainnet_e81889 - altair processSlashings 572.00 ns/op 803.00 ns/op 0.71
mainnet_e81889 - altair processEth1DataReset 622.00 ns/op 1.0090 us/op 0.62
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.3500 ms/op 2.2845 ms/op 1.03
mainnet_e81889 - altair processSlashingsReset 4.0170 us/op 6.4180 us/op 0.63
mainnet_e81889 - altair processRandaoMixesReset 4.1020 us/op 7.9700 us/op 0.51
mainnet_e81889 - altair processHistoricalRootsUpdate 693.00 ns/op 1.0240 us/op 0.68
mainnet_e81889 - altair processParticipationFlagUpdates 2.2970 us/op 4.9790 us/op 0.46
mainnet_e81889 - altair processSyncCommitteeUpdates 629.00 ns/op 1.1650 us/op 0.54
mainnet_e81889 - altair afterProcessEpoch 201.28 ms/op 193.77 ms/op 1.04
phase0 processEpoch - mainnet_e58758 483.81 ms/op 556.99 ms/op 0.87
mainnet_e58758 - phase0 beforeProcessEpoch 198.45 ms/op 264.38 ms/op 0.75
mainnet_e58758 - phase0 processJustificationAndFinalization 14.952 us/op 43.321 us/op 0.35
mainnet_e58758 - phase0 processRewardsAndPenalties 67.374 ms/op 149.36 ms/op 0.45
mainnet_e58758 - phase0 processRegistryUpdates 7.8810 us/op 13.455 us/op 0.59
mainnet_e58758 - phase0 processSlashings 611.00 ns/op 1.0440 us/op 0.59
mainnet_e58758 - phase0 processEth1DataReset 599.00 ns/op 1.1900 us/op 0.50
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.9192 ms/op 2.0219 ms/op 0.95
mainnet_e58758 - phase0 processSlashingsReset 3.5460 us/op 6.5420 us/op 0.54
mainnet_e58758 - phase0 processRandaoMixesReset 4.1370 us/op 8.2140 us/op 0.50
mainnet_e58758 - phase0 processHistoricalRootsUpdate 688.00 ns/op 1.2710 us/op 0.54
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.3950 us/op 8.7730 us/op 0.39
mainnet_e58758 - phase0 afterProcessEpoch 166.26 ms/op 158.26 ms/op 1.05
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.9771 ms/op 2.7282 ms/op 0.72
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.2128 ms/op 3.4648 ms/op 0.64
altair processInactivityUpdates - 250000 normalcase 32.079 ms/op 44.098 ms/op 0.73
altair processInactivityUpdates - 250000 worstcase 41.744 ms/op 51.752 ms/op 0.81
phase0 processRegistryUpdates - 250000 normalcase 6.1280 us/op 8.9190 us/op 0.69
phase0 processRegistryUpdates - 250000 badcase_full_deposits 368.47 us/op 414.94 us/op 0.89
phase0 processRegistryUpdates - 250000 worstcase 0.5 172.99 ms/op 224.83 ms/op 0.77
altair processRewardsAndPenalties - 250000 normalcase 75.818 ms/op 136.97 ms/op 0.55
altair processRewardsAndPenalties - 250000 worstcase 75.391 ms/op 90.593 ms/op 0.83
phase0 getAttestationDeltas - 250000 normalcase 11.959 ms/op 13.331 ms/op 0.90
phase0 getAttestationDeltas - 250000 worstcase 12.338 ms/op 13.855 ms/op 0.89
phase0 processSlashings - 250000 worstcase 5.6405 ms/op 5.8982 ms/op 0.96
altair processSyncCommitteeUpdates - 250000 284.03 ms/op 284.02 ms/op 1.00
BeaconState.hashTreeRoot - No change 550.00 ns/op 489.00 ns/op 1.12
BeaconState.hashTreeRoot - 1 full validator 72.713 us/op 65.841 us/op 1.10
BeaconState.hashTreeRoot - 32 full validator 715.05 us/op 628.26 us/op 1.14
BeaconState.hashTreeRoot - 512 full validator 6.9483 ms/op 8.4741 ms/op 0.82
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 98.291 us/op 78.248 us/op 1.26
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2365 ms/op 1.1797 ms/op 1.05
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.485 ms/op 15.529 ms/op 1.06
BeaconState.hashTreeRoot - 1 balances 67.718 us/op 60.743 us/op 1.11
BeaconState.hashTreeRoot - 32 balances 642.06 us/op 563.28 us/op 1.14
BeaconState.hashTreeRoot - 512 balances 6.2216 ms/op 5.8055 ms/op 1.07
BeaconState.hashTreeRoot - 250000 balances 101.58 ms/op 96.024 ms/op 1.06
aggregationBits - 2048 els - zipIndexesInBitList 28.824 us/op 38.020 us/op 0.76
regular array get 100000 times 60.586 us/op 67.491 us/op 0.90
wrappedArray get 100000 times 60.572 us/op 67.412 us/op 0.90
arrayWithProxy get 100000 times 27.651 ms/op 28.484 ms/op 0.97
ssz.Root.equals 485.00 ns/op 531.00 ns/op 0.91
byteArrayEquals 476.00 ns/op 484.00 ns/op 0.98
shuffle list - 16384 els 11.672 ms/op 10.913 ms/op 1.07
shuffle list - 250000 els 170.81 ms/op 161.08 ms/op 1.06
processSlot - 1 slots 13.371 us/op 11.524 us/op 1.16
processSlot - 32 slots 1.9143 ms/op 1.7281 ms/op 1.11
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 347.25 us/op 395.12 us/op 0.88
getCommitteeAssignments - req 1 vs - 250000 vc 5.3441 ms/op 5.3237 ms/op 1.00
getCommitteeAssignments - req 100 vs - 250000 vc 7.8675 ms/op 7.3329 ms/op 1.07
getCommitteeAssignments - req 1000 vs - 250000 vc 8.4352 ms/op 7.7879 ms/op 1.08
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 8.0100 ns/op 10.590 ns/op 0.76
state getBlockRootAtSlot - 250000 vs - 7PWei 1.0528 us/op 1.0595 us/op 0.99
computeProposers - vc 250000 16.848 ms/op 16.796 ms/op 1.00
computeEpochShuffling - vc 250000 173.48 ms/op 166.12 ms/op 1.04
getNextSyncCommittee - vc 250000 282.67 ms/op 272.66 ms/op 1.04

by benchmarkbot/action

@dapplion dapplion added the spec-deneb Issues targeting the Deneb spec version label Dec 5, 2022
@dapplion dapplion mentioned this pull request Dec 6, 2022
Merged
@twoeths twoeths merged commit eb61572 into unstable Dec 7, 2022
@twoeths twoeths deleted the dapplion/dynamic-reqresp branch December 7, 2022 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@twoeths twoeths twoeths approved these changes

Assignees
No one assigned
Labels
spec-deneb Issues targeting the Deneb spec version
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@dapplion @twoeths