Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: namespace getExpectedWithdrawals benchmark #5530

Merged
merged 1 commit into from May 24, 2023
Merged

chore: namespace getExpectedWithdrawals benchmark #5530

merged 1 commit into from May 24, 2023

Conversation

dapplion
Copy link
Contributor

Motivation

it blocks of benchmarks must be fully qualified. This example below does not have enough information in the name

    ✔ vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15                         20304.57 ops/s    49.25000 us/op   x6.472       1246 runs   7.16 s
    ✔ vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219                3745.837 ops/s    266.9630 us/op   x9.680       1393 runs   7.43 s
    ✔ vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42                 12449.58 ops/s    80.32400 us/op   x8.091       2167 runs   7.81 s
    ✔ vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18                 21150.14 ops/s    47.28100 us/op   x5.419       1175 runs   6.69 s
    ✔ vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020                   547.5639 ops/s    1.826271 ms/op  x18.489        275 runs   5.87 s
    ✔ vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777                72.51992 ops/s    13.78932 ms/op  x11.016         72 runs   5.81 s
    ✔ vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384                45.21641 ops/s    22.11587 ms/op  x24.413         16 runs   5.15 s
    ✔ vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384                      52.17905 ops/s    19.16478 ms/op  x13.449        125 runs   7.17 s
    ✔ vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384              26.93323 ops/s    37.12886 ms/op  x15.540         31 runs   5.53 s
    ✔ vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384                      42.02287 ops/s    23.79657 ms/op  x12.279         48 runs   5.88 s
    ✔ vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384              20.92901 ops/s    47.78057 ms/op  x11.428         29 runs   6.38 s

Description

Prefix getExpectedWithdrawals benchmarks with getExpectedWithdrawals

@dapplion dapplion requested a review from a team as a code owner May 24, 2023 10:03
@dapplion dapplion changed the title Namespace getExpectedWithdrawals benchmark chore: namespace getExpectedWithdrawals benchmark May 24, 2023
@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 08b6806 Previous: 5a91e39 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 599.30 us/op 635.25 us/op 0.94
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 51.775 us/op 50.875 us/op 1.02
BLS verify - blst-native 1.2108 ms/op 1.3002 ms/op 0.93
BLS verifyMultipleSignatures 3 - blst-native 2.4576 ms/op 2.6853 ms/op 0.92
BLS verifyMultipleSignatures 8 - blst-native 5.3464 ms/op 5.9925 ms/op 0.89
BLS verifyMultipleSignatures 32 - blst-native 19.334 ms/op 21.457 ms/op 0.90
BLS aggregatePubkeys 32 - blst-native 25.917 us/op 28.300 us/op 0.92
BLS aggregatePubkeys 128 - blst-native 100.95 us/op 110.94 us/op 0.91
getAttestationsForBlock 60.073 ms/op 61.894 ms/op 0.97
isKnown best case - 1 super set check 262.00 ns/op 272.00 ns/op 0.96
isKnown normal case - 2 super set checks 255.00 ns/op 262.00 ns/op 0.97
isKnown worse case - 16 super set checks 253.00 ns/op 275.00 ns/op 0.92
CheckpointStateCache - add get delete 5.8560 us/op 5.7200 us/op 1.02
validate gossip signedAggregateAndProof - struct 2.7982 ms/op 2.9509 ms/op 0.95
validate gossip attestation - struct 1.3453 ms/op 1.4184 ms/op 0.95
pickEth1Vote - no votes 1.3802 ms/op 1.5244 ms/op 0.91
pickEth1Vote - max votes 9.3741 ms/op 11.508 ms/op 0.81
pickEth1Vote - Eth1Data hashTreeRoot value x2048 9.4876 ms/op 9.8749 ms/op 0.96
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 16.197 ms/op 16.041 ms/op 1.01
pickEth1Vote - Eth1Data fastSerialize value x2048 745.59 us/op 750.78 us/op 0.99
pickEth1Vote - Eth1Data fastSerialize tree x2048 6.8276 ms/op 7.1551 ms/op 0.95
bytes32 toHexString 625.00 ns/op 509.00 ns/op 1.23
bytes32 Buffer.toString(hex) 436.00 ns/op 350.00 ns/op 1.25
bytes32 Buffer.toString(hex) from Uint8Array 635.00 ns/op 551.00 ns/op 1.15
bytes32 Buffer.toString(hex) + 0x 398.00 ns/op 385.00 ns/op 1.03
Object access 1 prop 0.18000 ns/op 0.17500 ns/op 1.03
Map access 1 prop 0.16700 ns/op 0.16700 ns/op 1.00
Object get x1000 7.1670 ns/op 6.6390 ns/op 1.08
Map get x1000 0.66100 ns/op 0.62200 ns/op 1.06
Object set x1000 72.770 ns/op 54.042 ns/op 1.35
Map set x1000 51.005 ns/op 45.972 ns/op 1.11
Return object 10000 times 0.27810 ns/op 0.24510 ns/op 1.13
Throw Error 10000 times 4.3562 us/op 4.3139 us/op 1.01
fastMsgIdFn sha256 / 200 bytes 3.6410 us/op 3.5730 us/op 1.02
fastMsgIdFn h32 xxhash / 200 bytes 328.00 ns/op 297.00 ns/op 1.10
fastMsgIdFn h64 xxhash / 200 bytes 494.00 ns/op 411.00 ns/op 1.20
fastMsgIdFn sha256 / 1000 bytes 12.460 us/op 11.822 us/op 1.05
fastMsgIdFn h32 xxhash / 1000 bytes 483.00 ns/op 434.00 ns/op 1.11
fastMsgIdFn h64 xxhash / 1000 bytes 592.00 ns/op 478.00 ns/op 1.24
fastMsgIdFn sha256 / 10000 bytes 107.85 us/op 104.30 us/op 1.03
fastMsgIdFn h32 xxhash / 10000 bytes 2.0690 us/op 1.9830 us/op 1.04
fastMsgIdFn h64 xxhash / 10000 bytes 1.5390 us/op 1.4720 us/op 1.05
enrSubnets - fastDeserialize 64 bits 2.0800 us/op 1.3870 us/op 1.50
enrSubnets - ssz BitVector 64 bits 691.00 ns/op 539.00 ns/op 1.28
enrSubnets - fastDeserialize 4 bits 232.00 ns/op 181.00 ns/op 1.28
enrSubnets - ssz BitVector 4 bits 719.00 ns/op 530.00 ns/op 1.36
prioritizePeers score -10:0 att 32-0.1 sync 2-0 144.53 us/op 107.17 us/op 1.35
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 190.53 us/op 144.65 us/op 1.32
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 214.04 us/op 181.00 us/op 1.18
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 439.56 us/op 318.35 us/op 1.38
prioritizePeers score 0:0 att 64-1 sync 4-1 502.50 us/op 380.42 us/op 1.32
array of 16000 items push then shift 1.7842 us/op 1.7179 us/op 1.04
LinkedList of 16000 items push then shift 9.4700 ns/op 9.5240 ns/op 0.99
array of 16000 items push then pop 115.93 ns/op 88.867 ns/op 1.30
LinkedList of 16000 items push then pop 8.9160 ns/op 8.5740 ns/op 1.04
array of 24000 items push then shift 2.3841 us/op 2.4118 us/op 0.99
LinkedList of 24000 items push then shift 8.8900 ns/op 9.1480 ns/op 0.97
array of 24000 items push then pop 102.05 ns/op 76.055 ns/op 1.34
LinkedList of 24000 items push then pop 10.462 ns/op 8.5830 ns/op 1.22
intersect bitArray bitLen 8 17.539 ns/op 13.712 ns/op 1.28
intersect array and set length 8 126.50 ns/op 80.066 ns/op 1.58
intersect bitArray bitLen 128 59.385 ns/op 46.082 ns/op 1.29
intersect array and set length 128 1.5158 us/op 1.0935 us/op 1.39
Buffer.concat 32 items 3.8800 us/op 3.0020 us/op 1.29
Uint8Array.set 32 items 3.0440 us/op 2.4830 us/op 1.23
pass gossip attestations to forkchoice per slot 2.7998 ms/op 2.3738 ms/op 1.18
computeDeltas 3.3348 ms/op 3.5069 ms/op 0.95
computeProposerBoostScoreFromBalances 1.8534 ms/op 1.7896 ms/op 1.04
altair processAttestation - 250000 vs - 7PWei normalcase 2.4210 ms/op 2.0978 ms/op 1.15
altair processAttestation - 250000 vs - 7PWei worstcase 4.3784 ms/op 3.5109 ms/op 1.25
altair processAttestation - setStatus - 1/6 committees join 144.45 us/op 146.64 us/op 0.99
altair processAttestation - setStatus - 1/3 committees join 303.91 us/op 285.98 us/op 1.06
altair processAttestation - setStatus - 1/2 committees join 426.62 us/op 398.78 us/op 1.07
altair processAttestation - setStatus - 2/3 committees join 504.13 us/op 489.64 us/op 1.03
altair processAttestation - setStatus - 4/5 committees join 674.70 us/op 676.56 us/op 1.00
altair processAttestation - setStatus - 100% committees join 848.59 us/op 798.50 us/op 1.06
altair processBlock - 250000 vs - 7PWei normalcase 20.484 ms/op 17.146 ms/op 1.19
altair processBlock - 250000 vs - 7PWei normalcase hashState 37.898 ms/op 25.569 ms/op 1.48
altair processBlock - 250000 vs - 7PWei worstcase 58.840 ms/op 53.651 ms/op 1.10
altair processBlock - 250000 vs - 7PWei worstcase hashState 91.708 ms/op 72.495 ms/op 1.27
phase0 processBlock - 250000 vs - 7PWei normalcase 2.4395 ms/op 2.3827 ms/op 1.02
phase0 processBlock - 250000 vs - 7PWei worstcase 31.591 ms/op 30.228 ms/op 1.05
altair processEth1Data - 250000 vs - 7PWei normalcase 698.43 us/op 521.86 us/op 1.34
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 14.015 us/op
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 39.250 us/op
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 16.701 us/op
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 13.472 us/op
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 123.66 us/op
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 841.03 us/op
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.1875 ms/op
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 920.13 us/op
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.7480 ms/op
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.9543 ms/op
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 7.7075 ms/op
Tree 40 250000 create 594.82 ms/op 332.96 ms/op 1.79
Tree 40 250000 get(125000) 206.84 ns/op 203.17 ns/op 1.02
Tree 40 250000 set(125000) 3.0676 us/op 1.0511 us/op 2.92
Tree 40 250000 toArray() 39.331 ms/op 22.611 ms/op 1.74
Tree 40 250000 iterate all - toArray() + loop 39.724 ms/op 23.969 ms/op 1.66
Tree 40 250000 iterate all - get(i) 104.17 ms/op 81.877 ms/op 1.27
MutableVector 250000 create 17.916 ms/op 12.625 ms/op 1.42
MutableVector 250000 get(125000) 6.6030 ns/op 6.9830 ns/op 0.95
MutableVector 250000 set(125000) 871.34 ns/op 363.67 ns/op 2.40
MutableVector 250000 toArray() 6.5070 ms/op 3.8059 ms/op 1.71
MutableVector 250000 iterate all - toArray() + loop 5.4296 ms/op 3.4116 ms/op 1.59
MutableVector 250000 iterate all - get(i) 1.7684 ms/op 1.6547 ms/op 1.07
Array 250000 create 6.3941 ms/op 3.1459 ms/op 2.03
Array 250000 clone - spread 6.2923 ms/op 1.2314 ms/op 5.11
Array 250000 get(125000) 2.0750 ns/op 0.60500 ns/op 3.43
Array 250000 set(125000) 1.9740 ns/op 0.70700 ns/op 2.79
Array 250000 iterate all - loop 88.789 us/op 88.428 us/op 1.00
effectiveBalanceIncrements clone Uint8Array 300000 84.662 us/op 37.303 us/op 2.27
effectiveBalanceIncrements clone MutableVector 300000 449.00 ns/op 368.00 ns/op 1.22
effectiveBalanceIncrements rw all Uint8Array 300000 183.90 us/op 189.98 us/op 0.97
effectiveBalanceIncrements rw all MutableVector 300000 288.19 ms/op 92.735 ms/op 3.11
phase0 afterProcessEpoch - 250000 vs - 7PWei 132.64 ms/op 130.49 ms/op 1.02
phase0 beforeProcessEpoch - 250000 vs - 7PWei 96.683 ms/op 47.815 ms/op 2.02
altair processEpoch - mainnet_e81889 575.20 ms/op 355.35 ms/op 1.62
mainnet_e81889 - altair beforeProcessEpoch 173.23 ms/op 72.703 ms/op 2.38
mainnet_e81889 - altair processJustificationAndFinalization 50.278 us/op 22.522 us/op 2.23
mainnet_e81889 - altair processInactivityUpdates 17.867 ms/op 6.4110 ms/op 2.79
mainnet_e81889 - altair processRewardsAndPenalties 145.91 ms/op 56.646 ms/op 2.58
mainnet_e81889 - altair processRegistryUpdates 9.0540 us/op 2.9110 us/op 3.11
mainnet_e81889 - altair processSlashings 2.2660 us/op 508.00 ns/op 4.46
mainnet_e81889 - altair processEth1DataReset 2.3060 us/op 775.00 ns/op 2.98
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.5688 ms/op 1.3259 ms/op 1.94
mainnet_e81889 - altair processSlashingsReset 13.550 us/op 5.2450 us/op 2.58
mainnet_e81889 - altair processRandaoMixesReset 13.936 us/op 7.4490 us/op 1.87
mainnet_e81889 - altair processHistoricalRootsUpdate 1.7430 us/op 1.1720 us/op 1.49
mainnet_e81889 - altair processParticipationFlagUpdates 9.1070 us/op 4.0000 us/op 2.28
mainnet_e81889 - altair processSyncCommitteeUpdates 1.9470 us/op 1.5320 us/op 1.27
mainnet_e81889 - altair afterProcessEpoch 138.71 ms/op 143.67 ms/op 0.97
phase0 processEpoch - mainnet_e58758 339.80 ms/op 501.13 ms/op 0.68
mainnet_e58758 - phase0 beforeProcessEpoch 182.29 ms/op 198.85 ms/op 0.92
mainnet_e58758 - phase0 processJustificationAndFinalization 33.099 us/op 29.180 us/op 1.13
mainnet_e58758 - phase0 processRewardsAndPenalties 69.878 ms/op 79.914 ms/op 0.87
mainnet_e58758 - phase0 processRegistryUpdates 12.824 us/op 17.874 us/op 0.72
mainnet_e58758 - phase0 processSlashings 1.2370 us/op 1.6020 us/op 0.77
mainnet_e58758 - phase0 processEth1DataReset 1.2770 us/op 1.6270 us/op 0.78
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.0475 ms/op 1.4159 ms/op 1.45
mainnet_e58758 - phase0 processSlashingsReset 4.1790 us/op 12.364 us/op 0.34
mainnet_e58758 - phase0 processRandaoMixesReset 6.6750 us/op 13.605 us/op 0.49
mainnet_e58758 - phase0 processHistoricalRootsUpdate 815.00 ns/op 2.0650 us/op 0.39
mainnet_e58758 - phase0 processParticipationRecordUpdates 8.7090 us/op 9.4680 us/op 0.92
mainnet_e58758 - phase0 afterProcessEpoch 98.182 ms/op 125.00 ms/op 0.79
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2891 ms/op 1.6068 ms/op 0.80
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.0067 ms/op 1.8986 ms/op 1.06
altair processInactivityUpdates - 250000 normalcase 23.232 ms/op 31.809 ms/op 0.73
altair processInactivityUpdates - 250000 worstcase 27.969 ms/op 24.545 ms/op 1.14
phase0 processRegistryUpdates - 250000 normalcase 17.729 us/op 8.1010 us/op 2.19
phase0 processRegistryUpdates - 250000 badcase_full_deposits 371.79 us/op 311.01 us/op 1.20
phase0 processRegistryUpdates - 250000 worstcase 0.5 193.98 ms/op 137.40 ms/op 1.41
altair processRewardsAndPenalties - 250000 normalcase 76.951 ms/op 74.113 ms/op 1.04
altair processRewardsAndPenalties - 250000 worstcase 93.249 ms/op 75.349 ms/op 1.24
phase0 getAttestationDeltas - 250000 normalcase 14.253 ms/op 10.788 ms/op 1.32
phase0 getAttestationDeltas - 250000 worstcase 16.776 ms/op 10.787 ms/op 1.56
phase0 processSlashings - 250000 worstcase 9.0400 ms/op 4.8886 ms/op 1.85
altair processSyncCommitteeUpdates - 250000 250.91 ms/op 226.76 ms/op 1.11
BeaconState.hashTreeRoot - No change 361.00 ns/op 338.00 ns/op 1.07
BeaconState.hashTreeRoot - 1 full validator 59.973 us/op 58.919 us/op 1.02
BeaconState.hashTreeRoot - 32 full validator 680.32 us/op 689.43 us/op 0.99
BeaconState.hashTreeRoot - 512 full validator 8.0852 ms/op 6.8242 ms/op 1.18
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 83.466 us/op 77.194 us/op 1.08
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.1659 ms/op 1.0648 ms/op 1.09
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 14.770 ms/op 13.896 ms/op 1.06
BeaconState.hashTreeRoot - 1 balances 63.969 us/op 55.827 us/op 1.15
BeaconState.hashTreeRoot - 32 balances 623.43 us/op 480.26 us/op 1.30
BeaconState.hashTreeRoot - 512 balances 6.9893 ms/op 5.0239 ms/op 1.39
BeaconState.hashTreeRoot - 250000 balances 88.188 ms/op 86.590 ms/op 1.02
aggregationBits - 2048 els - zipIndexesInBitList 25.715 us/op 25.253 us/op 1.02
regular array get 100000 times 49.957 us/op 45.123 us/op 1.11
wrappedArray get 100000 times 49.396 us/op 38.039 us/op 1.30
arrayWithProxy get 100000 times 16.609 ms/op 19.589 ms/op 0.85
ssz.Root.equals 615.00 ns/op 678.00 ns/op 0.91
byteArrayEquals 631.00 ns/op 663.00 ns/op 0.95
shuffle list - 16384 els 7.9366 ms/op 7.5599 ms/op 1.05
shuffle list - 250000 els 114.02 ms/op 116.00 ms/op 0.98
processSlot - 1 slots 11.856 us/op 9.8310 us/op 1.21
processSlot - 32 slots 1.4906 ms/op 1.5472 ms/op 0.96
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 39.148 ms/op 44.524 ms/op 0.88
getCommitteeAssignments - req 1 vs - 250000 vc 3.0031 ms/op 3.0746 ms/op 0.98
getCommitteeAssignments - req 100 vs - 250000 vc 4.2621 ms/op 5.4820 ms/op 0.78
getCommitteeAssignments - req 1000 vs - 250000 vc 4.6807 ms/op 5.3130 ms/op 0.88
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.8700 ns/op 5.6900 ns/op 0.86
state getBlockRootAtSlot - 250000 vs - 7PWei 1.0918 us/op 1.0291 us/op 1.06
computeProposers - vc 250000 11.395 ms/op 14.607 ms/op 0.78
computeEpochShuffling - vc 250000 105.00 ms/op 119.73 ms/op 0.88
getNextSyncCommittee - vc 250000 181.19 ms/op 229.64 ms/op 0.79
computeSigningRoot for AttestationData 14.699 us/op 16.361 us/op 0.90
hash AttestationData serialized data then Buffer.toString(base64) 2.5662 us/op 3.1634 us/op 0.81
toHexString serialized data 1.3751 us/op 2.3190 us/op 0.59
Buffer.toString(base64) 384.81 ns/op 455.39 ns/op 0.85

by benchmarkbot/action

@wemeetagain wemeetagain merged commit 3855341 into unstable May 24, 2023
10 of 12 checks passed
@wemeetagain wemeetagain deleted the dapplion/bench-getExpectedWithdrawals branch May 24, 2023 16:17
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.9.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@dapplion @wemeetagain