Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip PrepareNextSlot scheduler for forkchoice spec tests #4327

Merged
merged 1 commit into from Jul 19, 2022

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Jul 19, 2022

Motivation

See a lot of errors thrown from "PrepareNextSlotScheduler" when running forkchoice spec tests

Description

Add a flag to disable/skip it when running fork_choice spec tests

@twoeths twoeths requested a review from a team as a code owner July 19, 2022 04:26
@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: dacfc5b Previous: 938032c Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.1899 ms/op 2.6371 ms/op 0.83
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 81.738 us/op 87.224 us/op 0.94
BLS verify - blst-native 2.1942 ms/op 2.7314 ms/op 0.80
BLS verifyMultipleSignatures 3 - blst-native 4.4404 ms/op 5.6623 ms/op 0.78
BLS verifyMultipleSignatures 8 - blst-native 9.5590 ms/op 12.261 ms/op 0.78
BLS verifyMultipleSignatures 32 - blst-native 34.491 ms/op 43.951 ms/op 0.78
BLS aggregatePubkeys 32 - blst-native 45.433 us/op 60.360 us/op 0.75
BLS aggregatePubkeys 128 - blst-native 178.21 us/op 238.12 us/op 0.75
getAttestationsForBlock 52.805 ms/op 57.360 ms/op 0.92
isKnown best case - 1 super set check 557.00 ns/op 519.00 ns/op 1.07
isKnown normal case - 2 super set checks 546.00 ns/op 517.00 ns/op 1.06
isKnown worse case - 16 super set checks 549.00 ns/op 506.00 ns/op 1.08
CheckpointStateCache - add get delete 10.485 us/op 11.152 us/op 0.94
validate gossip signedAggregateAndProof - struct 4.9791 ms/op 6.4334 ms/op 0.77
validate gossip attestation - struct 2.3574 ms/op 3.0239 ms/op 0.78
altair verifyImport mainnet_s3766816:31 13.359 s/op 16.107 s/op 0.83
pickEth1Vote - no votes 2.6670 ms/op 2.6911 ms/op 0.99
pickEth1Vote - max votes 27.000 ms/op 28.506 ms/op 0.95
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.912 ms/op 15.770 ms/op 0.88
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 30.984 ms/op 25.940 ms/op 1.19
pickEth1Vote - Eth1Data fastSerialize value x2048 1.8013 ms/op 2.0085 ms/op 0.90
pickEth1Vote - Eth1Data fastSerialize tree x2048 16.199 ms/op 21.145 ms/op 0.77
bytes32 toHexString 1.2600 us/op 1.4820 us/op 0.85
bytes32 Buffer.toString(hex) 830.00 ns/op 912.00 ns/op 0.91
bytes32 Buffer.toString(hex) from Uint8Array 1.0410 us/op 1.2960 us/op 0.80
bytes32 Buffer.toString(hex) + 0x 815.00 ns/op 928.00 ns/op 0.88
Object access 1 prop 0.39600 ns/op 0.49500 ns/op 0.80
Map access 1 prop 0.33900 ns/op 0.35800 ns/op 0.95
Object get x1000 21.772 ns/op 16.730 ns/op 1.30
Map get x1000 1.3010 ns/op 1.0430 ns/op 1.25
Object set x1000 133.48 ns/op 131.33 ns/op 1.02
Map set x1000 79.787 ns/op 91.622 ns/op 0.87
Return object 10000 times 0.43100 ns/op 0.43680 ns/op 0.99
Throw Error 10000 times 6.7003 us/op 7.8854 us/op 0.85
enrSubnets - fastDeserialize 64 bits 2.7210 us/op 3.6870 us/op 0.74
enrSubnets - ssz BitVector 64 bits 808.00 ns/op 989.00 ns/op 0.82
enrSubnets - fastDeserialize 4 bits 405.00 ns/op 531.00 ns/op 0.76
enrSubnets - ssz BitVector 4 bits 772.00 ns/op 1.0590 us/op 0.73
prioritizePeers score -10:0 att 32-0.1 sync 2-0 106.50 us/op 135.19 us/op 0.79
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 144.22 us/op 153.92 us/op 0.94
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 247.49 us/op 304.59 us/op 0.81
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 521.41 us/op 675.07 us/op 0.77
prioritizePeers score 0:0 att 64-1 sync 4-1 537.88 us/op 653.21 us/op 0.82
RateTracker 1000000 limit, 1 obj count per request 223.18 ns/op 241.98 ns/op 0.92
RateTracker 1000000 limit, 2 obj count per request 169.93 ns/op 182.22 ns/op 0.93
RateTracker 1000000 limit, 4 obj count per request 141.35 ns/op 145.69 ns/op 0.97
RateTracker 1000000 limit, 8 obj count per request 134.50 ns/op 131.51 ns/op 1.02
RateTracker with prune 5.0540 us/op 6.3470 us/op 0.80
array of 16000 items push then shift 3.7247 us/op 5.6035 us/op 0.66
LinkedList of 16000 items push then shift 29.228 ns/op 20.435 ns/op 1.43
array of 16000 items push then pop 258.82 ns/op 291.78 ns/op 0.89
LinkedList of 16000 items push then pop 23.797 ns/op 19.032 ns/op 1.25
array of 24000 items push then shift 5.3082 us/op 7.7749 us/op 0.68
LinkedList of 24000 items push then shift 29.055 ns/op 19.724 ns/op 1.47
array of 24000 items push then pop 219.34 ns/op 223.75 ns/op 0.98
LinkedList of 24000 items push then pop 24.065 ns/op 17.959 ns/op 1.34
intersect bitArray bitLen 8 14.332 ns/op 12.310 ns/op 1.16
intersect array and set length 8 187.06 ns/op 224.61 ns/op 0.83
intersect bitArray bitLen 128 84.727 ns/op 72.533 ns/op 1.17
intersect array and set length 128 2.6108 us/op 2.6206 us/op 1.00
pass gossip attestations to forkchoice per slot 3.6607 ms/op 4.2991 ms/op 0.85
computeDeltas 4.3255 ms/op 3.8365 ms/op 1.13
computeProposerBoostScoreFromBalances 1.0771 ms/op 896.95 us/op 1.20
altair processAttestation - 250000 vs - 7PWei normalcase 4.7258 ms/op 5.3794 ms/op 0.88
altair processAttestation - 250000 vs - 7PWei worstcase 6.8347 ms/op 8.0710 ms/op 0.85
altair processAttestation - setStatus - 1/6 committees join 242.52 us/op 249.79 us/op 0.97
altair processAttestation - setStatus - 1/3 committees join 462.11 us/op 460.72 us/op 1.00
altair processAttestation - setStatus - 1/2 committees join 659.81 us/op 653.61 us/op 1.01
altair processAttestation - setStatus - 2/3 committees join 836.47 us/op 913.45 us/op 0.92
altair processAttestation - setStatus - 4/5 committees join 1.1742 ms/op 1.2128 ms/op 0.97
altair processAttestation - setStatus - 100% committees join 1.4124 ms/op 1.4423 ms/op 0.98
altair processBlock - 250000 vs - 7PWei normalcase 34.917 ms/op 33.006 ms/op 1.06
altair processBlock - 250000 vs - 7PWei normalcase hashState 46.466 ms/op 52.596 ms/op 0.88
altair processBlock - 250000 vs - 7PWei worstcase 95.225 ms/op 110.57 ms/op 0.86
altair processBlock - 250000 vs - 7PWei worstcase hashState 115.57 ms/op 129.78 ms/op 0.89
phase0 processBlock - 250000 vs - 7PWei normalcase 5.7830 ms/op 5.8963 ms/op 0.98
phase0 processBlock - 250000 vs - 7PWei worstcase 55.965 ms/op 76.931 ms/op 0.73
altair processEth1Data - 250000 vs - 7PWei normalcase 1.0120 ms/op 1.0379 ms/op 0.98
Tree 40 250000 create 847.88 ms/op 1.1055 s/op 0.77
Tree 40 250000 get(125000) 335.04 ns/op 351.62 ns/op 0.95
Tree 40 250000 set(125000) 2.6380 us/op 3.9365 us/op 0.67
Tree 40 250000 toArray() 35.015 ms/op 38.752 ms/op 0.90
Tree 40 250000 iterate all - toArray() + loop 35.828 ms/op 38.830 ms/op 0.92
Tree 40 250000 iterate all - get(i) 126.29 ms/op 138.35 ms/op 0.91
MutableVector 250000 create 17.349 ms/op 19.757 ms/op 0.88
MutableVector 250000 get(125000) 17.884 ns/op 13.935 ns/op 1.28
MutableVector 250000 set(125000) 810.60 ns/op 1.0324 us/op 0.79
MutableVector 250000 toArray() 8.0850 ms/op 8.6825 ms/op 0.93
MutableVector 250000 iterate all - toArray() + loop 7.9912 ms/op 8.4122 ms/op 0.95
MutableVector 250000 iterate all - get(i) 3.9287 ms/op 3.5789 ms/op 1.10
Array 250000 create 10.088 ms/op 7.8648 ms/op 1.28
Array 250000 clone - spread 2.7552 ms/op 5.8545 ms/op 0.47
Array 250000 get(125000) 1.2790 ns/op 2.2770 ns/op 0.56
Array 250000 set(125000) 1.2540 ns/op 2.4650 ns/op 0.51
Array 250000 iterate all - loop 188.35 us/op 146.18 us/op 1.29
effectiveBalanceIncrements clone Uint8Array 300000 69.632 us/op 162.74 us/op 0.43
effectiveBalanceIncrements clone MutableVector 300000 862.00 ns/op 878.00 ns/op 0.98
effectiveBalanceIncrements rw all Uint8Array 300000 280.98 us/op 310.48 us/op 0.90
effectiveBalanceIncrements rw all MutableVector 300000 179.15 ms/op 236.77 ms/op 0.76
phase0 afterProcessEpoch - 250000 vs - 7PWei 207.86 ms/op 226.14 ms/op 0.92
phase0 beforeProcessEpoch - 250000 vs - 7PWei 87.261 ms/op 80.225 ms/op 1.09
altair processEpoch - mainnet_e81889 570.20 ms/op 684.85 ms/op 0.83
mainnet_e81889 - altair beforeProcessEpoch 199.91 ms/op 202.20 ms/op 0.99
mainnet_e81889 - altair processJustificationAndFinalization 34.154 us/op 79.467 us/op 0.43
mainnet_e81889 - altair processInactivityUpdates 12.322 ms/op 12.234 ms/op 1.01
mainnet_e81889 - altair processRewardsAndPenalties 110.29 ms/op 115.27 ms/op 0.96
mainnet_e81889 - altair processRegistryUpdates 7.6340 us/op 19.758 us/op 0.39
mainnet_e81889 - altair processSlashings 1.7250 us/op 4.4510 us/op 0.39
mainnet_e81889 - altair processEth1DataReset 2.3030 us/op 4.6080 us/op 0.50
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.7836 ms/op 2.5865 ms/op 1.08
mainnet_e81889 - altair processSlashingsReset 13.455 us/op 30.541 us/op 0.44
mainnet_e81889 - altair processRandaoMixesReset 13.681 us/op 28.013 us/op 0.49
mainnet_e81889 - altair processHistoricalRootsUpdate 2.6760 us/op 5.6530 us/op 0.47
mainnet_e81889 - altair processParticipationFlagUpdates 6.9440 us/op 15.462 us/op 0.45
mainnet_e81889 - altair processSyncCommitteeUpdates 1.6450 us/op 4.1550 us/op 0.40
mainnet_e81889 - altair afterProcessEpoch 226.22 ms/op 220.94 ms/op 1.02
phase0 processEpoch - mainnet_e58758 591.25 ms/op 663.16 ms/op 0.89
mainnet_e58758 - phase0 beforeProcessEpoch 258.57 ms/op 302.84 ms/op 0.85
mainnet_e58758 - phase0 processJustificationAndFinalization 39.871 us/op 67.848 us/op 0.59
mainnet_e58758 - phase0 processRewardsAndPenalties 136.00 ms/op 166.71 ms/op 0.82
mainnet_e58758 - phase0 processRegistryUpdates 21.805 us/op 36.963 us/op 0.59
mainnet_e58758 - phase0 processSlashings 2.0190 us/op 4.0280 us/op 0.50
mainnet_e58758 - phase0 processEth1DataReset 2.0310 us/op 4.5200 us/op 0.45
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.3079 ms/op 2.7466 ms/op 0.84
mainnet_e58758 - phase0 processSlashingsReset 11.474 us/op 23.227 us/op 0.49
mainnet_e58758 - phase0 processRandaoMixesReset 14.556 us/op 26.051 us/op 0.56
mainnet_e58758 - phase0 processHistoricalRootsUpdate 2.6760 us/op 4.5900 us/op 0.58
mainnet_e58758 - phase0 processParticipationRecordUpdates 14.912 us/op 25.168 us/op 0.59
mainnet_e58758 - phase0 afterProcessEpoch 184.55 ms/op 182.13 ms/op 1.01
phase0 processEffectiveBalanceUpdates - 250000 normalcase 3.0093 ms/op 2.5749 ms/op 1.17
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.4588 ms/op 2.7694 ms/op 1.25
altair processInactivityUpdates - 250000 normalcase 47.337 ms/op 52.517 ms/op 0.90
altair processInactivityUpdates - 250000 worstcase 56.257 ms/op 65.673 ms/op 0.86
phase0 processRegistryUpdates - 250000 normalcase 14.373 us/op 30.274 us/op 0.47
phase0 processRegistryUpdates - 250000 badcase_full_deposits 497.77 us/op 518.79 us/op 0.96
phase0 processRegistryUpdates - 250000 worstcase 0.5 249.45 ms/op 246.09 ms/op 1.01
altair processRewardsAndPenalties - 250000 normalcase 125.09 ms/op 154.36 ms/op 0.81
altair processRewardsAndPenalties - 250000 worstcase 144.87 ms/op 101.06 ms/op 1.43
phase0 getAttestationDeltas - 250000 normalcase 14.444 ms/op 15.639 ms/op 0.92
phase0 getAttestationDeltas - 250000 worstcase 14.628 ms/op 18.134 ms/op 0.81
phase0 processSlashings - 250000 worstcase 6.3769 ms/op 7.0374 ms/op 0.91
altair processSyncCommitteeUpdates - 250000 323.67 ms/op 384.33 ms/op 0.84
BeaconState.hashTreeRoot - No change 545.00 ns/op 639.00 ns/op 0.85
BeaconState.hashTreeRoot - 1 full validator 71.582 us/op 87.264 us/op 0.82
BeaconState.hashTreeRoot - 32 full validator 715.05 us/op 873.39 us/op 0.82
BeaconState.hashTreeRoot - 512 full validator 6.7386 ms/op 8.0152 ms/op 0.84
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 94.582 us/op 110.48 us/op 0.86
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.3599 ms/op 1.6329 ms/op 0.83
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 17.041 ms/op 21.175 ms/op 0.80
BeaconState.hashTreeRoot - 1 balances 61.465 us/op 80.193 us/op 0.77
BeaconState.hashTreeRoot - 32 balances 672.16 us/op 764.14 us/op 0.88
BeaconState.hashTreeRoot - 512 balances 6.8653 ms/op 7.6298 ms/op 0.90
BeaconState.hashTreeRoot - 250000 balances 116.34 ms/op 133.00 ms/op 0.87
aggregationBits - 2048 els - zipIndexesInBitList 32.802 us/op 39.288 us/op 0.83
regular array get 100000 times 79.739 us/op 59.441 us/op 1.34
wrappedArray get 100000 times 79.710 us/op 58.375 us/op 1.37
arrayWithProxy get 100000 times 34.824 ms/op 36.834 ms/op 0.95
ssz.Root.equals 536.00 ns/op 666.00 ns/op 0.80
byteArrayEquals 521.00 ns/op 633.00 ns/op 0.82
shuffle list - 16384 els 12.971 ms/op 12.507 ms/op 1.04
shuffle list - 250000 els 192.52 ms/op 181.66 ms/op 1.06
processSlot - 1 slots 14.796 us/op 18.930 us/op 0.78
processSlot - 32 slots 1.9963 ms/op 2.5224 ms/op 0.79
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 462.07 us/op 497.22 us/op 0.93
getCommitteeAssignments - req 1 vs - 250000 vc 6.2464 ms/op 5.5399 ms/op 1.13
getCommitteeAssignments - req 100 vs - 250000 vc 8.4712 ms/op 7.8958 ms/op 1.07
getCommitteeAssignments - req 1000 vs - 250000 vc 9.1090 ms/op 8.6038 ms/op 1.06
computeProposers - vc 250000 20.175 ms/op 23.774 ms/op 0.85
computeEpochShuffling - vc 250000 196.41 ms/op 187.07 ms/op 1.05
getNextSyncCommittee - vc 250000 308.71 ms/op 361.09 ms/op 0.85

by benchmarkbot/action

@dapplion dapplion merged commit 30e400b into unstable Jul 19, 2022
@dapplion dapplion deleted the tuyen/skip-prepareNextSlot branch July 19, 2022 06:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants