Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase Tree benchmark timeout #3529

Merged
merged 1 commit into from
Dec 16, 2021
Merged

Conversation

dapplion
Copy link
Contributor

Motivation

Fix broken CI in https://github.com/ChainSafe/lodestar/runs/4544729064

Description

Increase Tree benchmark timeout

@codeclimate
Copy link

codeclimate bot commented Dec 16, 2021

Code Climate has analyzed commit 579d74d and detected 0 issues on this pull request.

View more on Code Climate.

@codecov
Copy link

codecov bot commented Dec 16, 2021

Codecov Report

Merging #3529 (579d74d) into master (eba2f2a) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3529   +/-   ##
=======================================
  Coverage   37.57%   37.57%           
=======================================
  Files         310      310           
  Lines        8251     8251           
  Branches     1272     1272           
=======================================
  Hits         3100     3100           
  Misses       5003     5003           
  Partials      148      148           

@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 6454f49 Previous: 7f4a590 Ratio
BeaconState.hashTreeRoot - No change 481.00 ns/op 747.00 ns/op 0.64
BeaconState.hashTreeRoot - 1 full validator 112.28 us/op 101.39 us/op 1.11
BeaconState.hashTreeRoot - 32 full validator 1.7949 ms/op 1.5942 ms/op 1.13
BeaconState.hashTreeRoot - 512 full validator 23.783 ms/op 17.514 ms/op 1.36
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 114.46 us/op 102.07 us/op 1.12
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.8465 ms/op 1.6999 ms/op 1.09
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 24.712 ms/op 20.869 ms/op 1.18
BeaconState.hashTreeRoot - 1 balances 83.638 us/op 72.691 us/op 1.15
BeaconState.hashTreeRoot - 32 balances 683.87 us/op 631.21 us/op 1.08
BeaconState.hashTreeRoot - 512 balances 6.6013 ms/op 5.9742 ms/op 1.10
BeaconState.hashTreeRoot - 250000 balances 123.19 ms/op 108.93 ms/op 1.13
processSlot - 1 slots 41.785 us/op 52.405 us/op 0.80
processSlot - 32 slots 2.5162 ms/op 2.7395 ms/op 0.92
getCommitteeAssignments - req 1 vs - 250000 vc 4.8255 ms/op 5.4960 ms/op 0.88
getCommitteeAssignments - req 100 vs - 250000 vc 6.4351 ms/op 7.7299 ms/op 0.83
getCommitteeAssignments - req 1000 vs - 250000 vc 7.3080 ms/op 8.3916 ms/op 0.87
computeProposers - vc 250000 18.617 ms/op 26.135 ms/op 0.71
computeEpochShuffling - vc 250000 170.99 ms/op 209.02 ms/op 0.82
getNextSyncCommittee - vc 250000 310.12 ms/op 426.04 ms/op 0.73
altair processAttestation - 250000 vs - 7PWei normalcase 46.536 ms/op 41.906 ms/op 1.11
altair processAttestation - 250000 vs - 7PWei worstcase 42.694 ms/op 55.100 ms/op 0.77
altair processAttestation - setStatus - 1/6 committees join 10.105 ms/op 11.750 ms/op 0.86
altair processAttestation - setStatus - 1/3 committees join 20.740 ms/op 24.060 ms/op 0.86
altair processAttestation - setStatus - 1/2 committees join 31.268 ms/op 36.570 ms/op 0.86
altair processAttestation - setStatus - 2/3 committees join 41.742 ms/op 49.870 ms/op 0.84
altair processAttestation - setStatus - 4/5 committees join 51.149 ms/op 60.453 ms/op 0.85
altair processAttestation - setStatus - 100% committees join 57.059 ms/op 75.301 ms/op 0.76
altair processAttestation - updateEpochParticipants - 1/6 committees join 11.272 ms/op 13.320 ms/op 0.85
altair processAttestation - updateEpochParticipants - 1/3 committees join 19.232 ms/op 25.175 ms/op 0.76
altair processAttestation - updateEpochParticipants - 1/2 committees join 23.167 ms/op 30.594 ms/op 0.76
altair processAttestation - updateEpochParticipants - 2/3 committees join 31.315 ms/op 31.784 ms/op 0.99
altair processAttestation - updateEpochParticipants - 4/5 committees join 28.611 ms/op 38.761 ms/op 0.74
altair processAttestation - updateEpochParticipants - 100% committees join 29.863 ms/op 34.982 ms/op 0.85
altair processAttestation - updateAllStatus 21.091 ms/op 24.959 ms/op 0.85
altair processBlock - 250000 vs - 7PWei normalcase 41.260 ms/op 48.488 ms/op 0.85
altair processBlock - 250000 vs - 7PWei worstcase 123.52 ms/op 140.04 ms/op 0.88
altair processEpoch - mainnet_e81889 1.0268 s/op 1.0733 s/op 0.96
mainnet_e81889 - altair beforeProcessEpoch 270.37 ms/op 377.32 ms/op 0.72
mainnet_e81889 - altair processJustificationAndFinalization 124.54 us/op 110.58 us/op 1.13
mainnet_e81889 - altair processInactivityUpdates 16.892 ms/op 18.787 ms/op 0.90
mainnet_e81889 - altair processRewardsAndPenalties 233.57 ms/op 158.99 ms/op 1.47
mainnet_e81889 - altair processRegistryUpdates 18.350 us/op 19.352 us/op 0.95
mainnet_e81889 - altair processSlashings 4.8910 us/op 4.9270 us/op 0.99
mainnet_e81889 - altair processEth1DataReset 5.1910 us/op 4.2350 us/op 1.23
mainnet_e81889 - altair processEffectiveBalanceUpdates 10.530 ms/op 14.836 ms/op 0.71
mainnet_e81889 - altair processSlashingsReset 26.592 us/op 27.913 us/op 0.95
mainnet_e81889 - altair processRandaoMixesReset 36.105 us/op 32.992 us/op 1.09
mainnet_e81889 - altair processHistoricalRootsUpdate 6.5580 us/op 5.4030 us/op 1.21
mainnet_e81889 - altair processParticipationFlagUpdates 99.283 ms/op 167.08 ms/op 0.59
mainnet_e81889 - altair processSyncCommitteeUpdates 4.1800 us/op 3.9560 us/op 1.06
mainnet_e81889 - altair afterProcessEpoch 204.86 ms/op 248.90 ms/op 0.82
altair processInactivityUpdates - 250000 normalcase 61.448 ms/op 72.425 ms/op 0.85
altair processInactivityUpdates - 250000 worstcase 63.619 ms/op 71.646 ms/op 0.89
altair processParticipationFlagUpdates - 250000 anycase 103.19 ms/op 96.734 ms/op 1.07
altair processRewardsAndPenalties - 250000 normalcase 142.03 ms/op 123.53 ms/op 1.15
altair processRewardsAndPenalties - 250000 worstcase 143.58 ms/op 140.39 ms/op 1.02
altair processSyncCommitteeUpdates - 250000 319.99 ms/op 445.55 ms/op 0.72
Tree 40 250000 create 736.26 ms/op 582.40 ms/op 1.26
Tree 40 250000 get(125000) 289.13 ns/op 340.26 ns/op 0.85
Tree 40 250000 set(125000) 1.7404 us/op 1.9669 us/op 0.88
Tree 40 250000 toArray() 39.823 ms/op 45.182 ms/op 0.88
Tree 40 250000 iterate all - toArray() + loop 44.092 ms/op 44.514 ms/op 0.99
Tree 40 250000 iterate all - get(i) 104.73 ms/op 145.48 ms/op 0.72
MutableVector 250000 create 17.981 ms/op 26.650 ms/op 0.67
MutableVector 250000 get(125000) 13.060 ns/op 15.803 ns/op 0.83
MutableVector 250000 set(125000) 433.05 ns/op 696.44 ns/op 0.62
MutableVector 250000 toArray() 7.4699 ms/op 9.1563 ms/op 0.82
MutableVector 250000 iterate all - toArray() + loop 7.2338 ms/op 9.2517 ms/op 0.78
MutableVector 250000 iterate all - get(i) 3.0894 ms/op 3.7452 ms/op 0.82
Array 250000 create 4.3704 ms/op 5.5231 ms/op 0.79
Array 250000 clone - spread 1.9947 ms/op 2.3035 ms/op 0.87
Array 250000 get(125000) 0.91300 ns/op 1.1570 ns/op 0.79
Array 250000 set(125000) 0.91100 ns/op 1.2190 ns/op 0.75
Array 250000 iterate all - loop 151.03 us/op 143.53 us/op 1.05
aggregationBits - 2048 els - readonlyValues 211.89 us/op 240.30 us/op 0.88
aggregationBits - 2048 els - zipIndexesInBitList 42.794 us/op 41.592 us/op 1.03
regular array get 100000 times 62.411 us/op 58.684 us/op 1.06
wrappedArray get 100000 times 63.189 us/op 58.184 us/op 1.09
arrayWithProxy get 100000 times 33.104 ms/op 36.223 ms/op 0.91
ssz.Root.equals 1.1900 us/op 1.2950 us/op 0.92
ssz.Root.equals with valueOf() 1.4340 us/op 1.6480 us/op 0.87
byteArrayEquals with valueOf() 1.3400 us/op 1.6170 us/op 0.83
phase0 processBlock - 250000 vs - 7PWei normalcase 10.204 ms/op 12.986 ms/op 0.79
phase0 processBlock - 250000 vs - 7PWei worstcase 67.971 ms/op 100.28 ms/op 0.68
phase0 afterProcessEpoch - 250000 vs - 7PWei 196.94 ms/op 239.00 ms/op 0.82
phase0 beforeProcessEpoch - 250000 vs - 7PWei 498.78 ms/op 759.52 ms/op 0.66
phase0 processEpoch - mainnet_e58758 716.24 ms/op 952.70 ms/op 0.75
mainnet_e58758 - phase0 beforeProcessEpoch 467.75 ms/op 602.43 ms/op 0.78
mainnet_e58758 - phase0 processJustificationAndFinalization 99.157 us/op 108.52 us/op 0.91
mainnet_e58758 - phase0 processRewardsAndPenalties 102.04 ms/op 109.14 ms/op 0.93
mainnet_e58758 - phase0 processRegistryUpdates 83.191 us/op 76.334 us/op 1.09
mainnet_e58758 - phase0 processSlashings 5.3420 us/op 6.0340 us/op 0.89
mainnet_e58758 - phase0 processEth1DataReset 4.9170 us/op 5.0200 us/op 0.98
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 8.1807 ms/op 11.530 ms/op 0.71
mainnet_e58758 - phase0 processSlashingsReset 23.472 us/op 25.608 us/op 0.92
mainnet_e58758 - phase0 processRandaoMixesReset 26.083 us/op 30.221 us/op 0.86
mainnet_e58758 - phase0 processHistoricalRootsUpdate 5.2320 us/op 5.2710 us/op 0.99
mainnet_e58758 - phase0 processParticipationRecordUpdates 21.902 us/op 23.803 us/op 0.92
mainnet_e58758 - phase0 afterProcessEpoch 162.37 ms/op 206.87 ms/op 0.78
phase0 processEffectiveBalanceUpdates - 250000 normalcase 10.569 ms/op 12.923 ms/op 0.82
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.1155 s/op 1.4349 s/op 0.78
phase0 processRegistryUpdates - 250000 normalcase 86.185 us/op 82.817 us/op 1.04
phase0 processRegistryUpdates - 250000 badcase_full_deposits 2.6891 ms/op 3.7039 ms/op 0.73
phase0 processRegistryUpdates - 250000 worstcase 0.5 1.5482 s/op 2.0451 s/op 0.76
phase0 getAttestationDeltas - 250000 normalcase 31.932 ms/op 37.754 ms/op 0.85
phase0 getAttestationDeltas - 250000 worstcase 32.167 ms/op 37.173 ms/op 0.87
phase0 processSlashings - 250000 worstcase 33.820 ms/op 35.781 ms/op 0.95
shuffle list - 16384 els 10.998 ms/op 15.499 ms/op 0.71
shuffle list - 250000 els 164.01 ms/op 205.21 ms/op 0.80
getEffectiveBalances - 250000 vs - 7PWei 9.1893 ms/op 11.746 ms/op 0.78
computeDeltas 3.2796 ms/op 3.9897 ms/op 0.82
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.8046 ms/op 2.3614 ms/op 0.76
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 609.49 us/op 813.12 us/op 0.75
BLS verify - blst-native 1.6849 ms/op 2.6945 ms/op 0.63
BLS verifyMultipleSignatures 3 - blst-native 3.4872 ms/op 5.5311 ms/op 0.63
BLS verifyMultipleSignatures 8 - blst-native 7.2199 ms/op 12.065 ms/op 0.60
BLS verifyMultipleSignatures 32 - blst-native 31.495 ms/op 43.454 ms/op 0.72
BLS aggregatePubkeys 32 - blst-native 39.460 us/op 59.093 us/op 0.67
BLS aggregatePubkeys 128 - blst-native 141.58 us/op 231.06 us/op 0.61
getAttestationsForBlock 75.291 ms/op 102.54 ms/op 0.73
CheckpointStateCache - add get delete 13.817 us/op 19.281 us/op 0.72
validate gossip signedAggregateAndProof - struct 4.0790 ms/op 6.5816 ms/op 0.62
validate gossip signedAggregateAndProof - treeBacked 4.0301 ms/op 6.4257 ms/op 0.63
validate gossip attestation - struct 1.9155 ms/op 3.0024 ms/op 0.64
validate gossip attestation - treeBacked 1.8556 ms/op 3.1147 ms/op 0.60
Object access 1 prop 0.27300 ns/op 0.39200 ns/op 0.70
Map access 1 prop 0.25500 ns/op 0.34400 ns/op 0.74
Object get x1000 16.387 ns/op 16.462 ns/op 1.00
Map get x1000 0.92500 ns/op 1.0360 ns/op 0.89
Object set x1000 87.866 ns/op 104.19 ns/op 0.84
Map set x1000 53.533 ns/op 72.418 ns/op 0.74
Return object 10000 times 0.33360 ns/op 0.42730 ns/op 0.78
Throw Error 10000 times 5.3971 us/op 7.0624 us/op 0.76
RateTracker 1000000 limit, 1 obj count per request 160.94 ns/op
RateTracker 1000000 limit, 2 obj count per request 118.86 ns/op
RateTracker 1000000 limit, 4 obj count per request 102.00 ns/op
RateTracker 1000000 limit, 8 obj count per request 89.823 ns/op
RateTracker with prune 3.3410 us/op

by benchmarkbot/action

@dapplion dapplion merged commit 9e08eff into master Dec 16, 2021
@dapplion dapplion deleted the dapplion/extend-tree-timeout branch December 16, 2021 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants