Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Separate E2E and Browser github workflow tests #4356

Merged
merged 4 commits into from Aug 2, 2022
Merged

Separate E2E and Browser github workflow tests #4356

merged 4 commits into from Aug 2, 2022

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Aug 2, 2022 •
edited

Separate E2E and Browser github workflow tests

Closes #4352

@g11tech g11tech requested a review from a team as a code owner August 2, 2022 13:46
@g11tech g11tech enabled auto-merge (squash) August 2, 2022 13:50
@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2022 •
edited

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 4d3be50 Previous: b6c1e2b Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.0603 ms/op 1.5773 ms/op 1.31
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 66.181 us/op 63.628 us/op 1.04
BLS verify - blst-native 2.1682 ms/op 2.1647 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 4.4730 ms/op 4.4711 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst-native 9.6706 ms/op 9.6785 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 35.143 ms/op 35.177 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 46.585 us/op 46.682 us/op 1.00
BLS aggregatePubkeys 128 - blst-native 182.17 us/op 182.24 us/op 1.00
getAttestationsForBlock 164.56 ms/op 141.62 ms/op 1.16
isKnown best case - 1 super set check 488.00 ns/op 484.00 ns/op 1.01
isKnown normal case - 2 super set checks 477.00 ns/op 465.00 ns/op 1.03
isKnown worse case - 16 super set checks 478.00 ns/op 464.00 ns/op 1.03
CheckpointStateCache - add get delete 9.3580 us/op 8.5570 us/op 1.09
validate gossip signedAggregateAndProof - struct 5.0242 ms/op 5.0141 ms/op 1.00
validate gossip attestation - struct 2.3842 ms/op 2.3711 ms/op 1.01
altair verifyImport mainnet_s3766816:31 8.8703 s/op 8.6806 s/op 1.02
pickEth1Vote - no votes 2.1284 ms/op 2.0140 ms/op 1.06
pickEth1Vote - max votes 23.214 ms/op 20.245 ms/op 1.15
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.740 ms/op 12.534 ms/op 1.10
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 22.242 ms/op 19.867 ms/op 1.12
pickEth1Vote - Eth1Data fastSerialize value x2048 1.6533 ms/op 1.4238 ms/op 1.16
pickEth1Vote - Eth1Data fastSerialize tree x2048 16.172 ms/op 15.085 ms/op 1.07
bytes32 toHexString 1.2860 us/op 998.00 ns/op 1.29
bytes32 Buffer.toString(hex) 853.00 ns/op 766.00 ns/op 1.11
bytes32 Buffer.toString(hex) from Uint8Array 1.1130 us/op 1.0220 us/op 1.09
bytes32 Buffer.toString(hex) + 0x 857.00 ns/op 769.00 ns/op 1.11
Object access 1 prop 0.45000 ns/op 0.41500 ns/op 1.08
Map access 1 prop 0.30500 ns/op 0.30000 ns/op 1.02
Object get x1000 11.080 ns/op 10.760 ns/op 1.03
Map get x1000 0.93500 ns/op 1.0010 ns/op 0.93
Object set x1000 90.394 ns/op 73.958 ns/op 1.22
Map set x1000 58.605 ns/op 48.537 ns/op 1.21
Return object 10000 times 0.44520 ns/op 0.43890 ns/op 1.01
Throw Error 10000 times 6.0487 us/op 6.0712 us/op 1.00
enrSubnets - fastDeserialize 64 bits 3.3650 us/op 2.7160 us/op 1.24
enrSubnets - ssz BitVector 64 bits 911.00 ns/op 817.00 ns/op 1.12
enrSubnets - fastDeserialize 4 bits 490.00 ns/op 421.00 ns/op 1.16
enrSubnets - ssz BitVector 4 bits 916.00 ns/op 798.00 ns/op 1.15
prioritizePeers score -10:0 att 32-0.1 sync 2-0 97.015 us/op 84.167 us/op 1.15
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 114.98 us/op 124.92 us/op 0.92
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 234.22 us/op 203.52 us/op 1.15
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 457.87 us/op 341.25 us/op 1.34
prioritizePeers score 0:0 att 64-1 sync 4-1 409.29 us/op 413.89 us/op 0.99
RateTracker 1000000 limit, 1 obj count per request 208.21 ns/op 186.95 ns/op 1.11
RateTracker 1000000 limit, 2 obj count per request 151.90 ns/op 136.59 ns/op 1.11
RateTracker 1000000 limit, 4 obj count per request 128.42 ns/op 110.89 ns/op 1.16
RateTracker 1000000 limit, 8 obj count per request 115.16 ns/op 99.134 ns/op 1.16
RateTracker with prune 4.9520 us/op 4.2440 us/op 1.17
array of 16000 items push then shift 51.594 us/op 51.601 us/op 1.00
LinkedList of 16000 items push then shift 16.859 ns/op 16.416 ns/op 1.03
array of 16000 items push then pop 244.46 ns/op 220.89 ns/op 1.11
LinkedList of 16000 items push then pop 14.577 ns/op 14.215 ns/op 1.03
array of 24000 items push then shift 77.394 us/op 77.360 us/op 1.00
LinkedList of 24000 items push then shift 22.124 ns/op 18.968 ns/op 1.17
array of 24000 items push then pop 195.46 ns/op 195.99 ns/op 1.00
LinkedList of 24000 items push then pop 16.014 ns/op 15.434 ns/op 1.04
intersect bitArray bitLen 8 10.629 ns/op 10.944 ns/op 0.97
intersect array and set length 8 157.40 ns/op 150.91 ns/op 1.04
intersect bitArray bitLen 128 57.853 ns/op 55.556 ns/op 1.04
intersect array and set length 128 2.0091 us/op 1.7990 us/op 1.12
Buffer.concat 32 items 1.8290 ns/op 1.8520 ns/op 0.99
pass gossip attestations to forkchoice per slot 6.7225 ms/op 3.1714 ms/op 2.12
computeDeltas 3.2660 ms/op 3.1452 ms/op 1.04
computeProposerBoostScoreFromBalances 803.92 us/op 804.23 us/op 1.00
altair processAttestation - 250000 vs - 7PWei normalcase 3.7274 ms/op 3.4597 ms/op 1.08
altair processAttestation - 250000 vs - 7PWei worstcase 5.6072 ms/op 5.1736 ms/op 1.08
altair processAttestation - setStatus - 1/6 committees join 177.75 us/op 177.00 us/op 1.00
altair processAttestation - setStatus - 1/3 committees join 340.68 us/op 343.88 us/op 0.99
altair processAttestation - setStatus - 1/2 committees join 487.96 us/op 493.70 us/op 0.99
altair processAttestation - setStatus - 2/3 committees join 638.51 us/op 640.28 us/op 1.00
altair processAttestation - setStatus - 4/5 committees join 898.11 us/op 902.49 us/op 1.00
altair processAttestation - setStatus - 100% committees join 1.0802 ms/op 1.0848 ms/op 1.00
altair processBlock - 250000 vs - 7PWei normalcase 25.746 ms/op 24.665 ms/op 1.04
altair processBlock - 250000 vs - 7PWei normalcase hashState 35.760 ms/op 33.189 ms/op 1.08
altair processBlock - 250000 vs - 7PWei worstcase 79.524 ms/op 81.926 ms/op 0.97
altair processBlock - 250000 vs - 7PWei worstcase hashState 102.80 ms/op 114.49 ms/op 0.90
phase0 processBlock - 250000 vs - 7PWei normalcase 4.2537 ms/op 3.8391 ms/op 1.11
phase0 processBlock - 250000 vs - 7PWei worstcase 53.538 ms/op 51.671 ms/op 1.04
altair processEth1Data - 250000 vs - 7PWei normalcase 797.35 us/op 716.64 us/op 1.11
Tree 40 250000 create 746.09 ms/op 703.57 ms/op 1.06
Tree 40 250000 get(125000) 239.19 ns/op 227.40 ns/op 1.05
Tree 40 250000 set(125000) 2.5409 us/op 2.1374 us/op 1.19
Tree 40 250000 toArray() 28.708 ms/op 26.176 ms/op 1.10
Tree 40 250000 iterate all - toArray() + loop 28.896 ms/op 27.035 ms/op 1.07
Tree 40 250000 iterate all - get(i) 113.85 ms/op 109.25 ms/op 1.04
MutableVector 250000 create 15.734 ms/op 12.328 ms/op 1.28
MutableVector 250000 get(125000) 11.236 ns/op 11.198 ns/op 1.00
MutableVector 250000 set(125000) 564.66 ns/op 539.47 ns/op 1.05
MutableVector 250000 toArray() 6.4508 ms/op 6.1410 ms/op 1.05
MutableVector 250000 iterate all - toArray() + loop 6.5068 ms/op 6.3134 ms/op 1.03
MutableVector 250000 iterate all - get(i) 2.6096 ms/op 2.6896 ms/op 0.97
Array 250000 create 6.5254 ms/op 6.1805 ms/op 1.06
Array 250000 clone - spread 3.5513 ms/op 3.5662 ms/op 1.00
Array 250000 get(125000) 1.6010 ns/op 1.5780 ns/op 1.01
Array 250000 set(125000) 1.5740 ns/op 1.5430 ns/op 1.02
Array 250000 iterate all - loop 153.00 us/op 153.21 us/op 1.00
effectiveBalanceIncrements clone Uint8Array 300000 53.838 us/op 91.362 us/op 0.59
effectiveBalanceIncrements clone MutableVector 300000 1.1300 us/op 643.00 ns/op 1.76
effectiveBalanceIncrements rw all Uint8Array 300000 247.39 us/op 248.62 us/op 1.00
effectiveBalanceIncrements rw all MutableVector 300000 180.90 ms/op 126.84 ms/op 1.43
phase0 afterProcessEpoch - 250000 vs - 7PWei 190.33 ms/op 199.38 ms/op 0.95
phase0 beforeProcessEpoch - 250000 vs - 7PWei 60.040 ms/op 58.330 ms/op 1.03
altair processEpoch - mainnet_e81889 547.97 ms/op 548.93 ms/op 1.00
mainnet_e81889 - altair beforeProcessEpoch 149.83 ms/op 132.21 ms/op 1.13
mainnet_e81889 - altair processJustificationAndFinalization 21.061 us/op 17.025 us/op 1.24
mainnet_e81889 - altair processInactivityUpdates 8.9449 ms/op 8.8722 ms/op 1.01
mainnet_e81889 - altair processRewardsAndPenalties 81.676 ms/op 81.061 ms/op 1.01
mainnet_e81889 - altair processRegistryUpdates 3.7560 us/op 2.6010 us/op 1.44
mainnet_e81889 - altair processSlashings 898.00 ns/op 546.00 ns/op 1.64
mainnet_e81889 - altair processEth1DataReset 898.00 ns/op 571.00 ns/op 1.57
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.1809 ms/op 2.1679 ms/op 1.01
mainnet_e81889 - altair processSlashingsReset 6.0450 us/op 4.2120 us/op 1.44
mainnet_e81889 - altair processRandaoMixesReset 6.1220 us/op 4.0880 us/op 1.50
mainnet_e81889 - altair processHistoricalRootsUpdate 1.0840 us/op 694.00 ns/op 1.56
mainnet_e81889 - altair processParticipationFlagUpdates 4.2020 us/op 7.6730 us/op 0.55
mainnet_e81889 - altair processSyncCommitteeUpdates 1.1220 us/op 1.5920 us/op 0.70
mainnet_e81889 - altair afterProcessEpoch 218.32 ms/op 220.94 ms/op 0.99
phase0 processEpoch - mainnet_e58758 492.15 ms/op 615.00 ms/op 0.80
mainnet_e58758 - phase0 beforeProcessEpoch 182.78 ms/op 238.16 ms/op 0.77
mainnet_e58758 - phase0 processJustificationAndFinalization 19.961 us/op 17.001 us/op 1.17
mainnet_e58758 - phase0 processRewardsAndPenalties 117.55 ms/op 132.61 ms/op 0.89
mainnet_e58758 - phase0 processRegistryUpdates 9.6490 us/op 7.6200 us/op 1.27
mainnet_e58758 - phase0 processSlashings 629.00 ns/op 572.00 ns/op 1.10
mainnet_e58758 - phase0 processEth1DataReset 870.00 ns/op 565.00 ns/op 1.54
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.8194 ms/op 1.7763 ms/op 1.02
mainnet_e58758 - phase0 processSlashingsReset 5.8410 us/op 4.6000 us/op 1.27
mainnet_e58758 - phase0 processRandaoMixesReset 6.6430 us/op 4.0810 us/op 1.63
mainnet_e58758 - phase0 processHistoricalRootsUpdate 877.00 ns/op 629.00 ns/op 1.39
mainnet_e58758 - phase0 processParticipationRecordUpdates 5.1770 us/op 3.8180 us/op 1.36
mainnet_e58758 - phase0 afterProcessEpoch 162.01 ms/op 164.78 ms/op 0.98
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.9703 ms/op 1.9986 ms/op 0.99
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.2624 ms/op 2.2360 ms/op 1.01
altair processInactivityUpdates - 250000 normalcase 39.567 ms/op 51.982 ms/op 0.76
altair processInactivityUpdates - 250000 worstcase 40.417 ms/op 56.523 ms/op 0.72
phase0 processRegistryUpdates - 250000 normalcase 6.6070 us/op 6.9270 us/op 0.95
phase0 processRegistryUpdates - 250000 badcase_full_deposits 393.15 us/op 449.75 us/op 0.87
phase0 processRegistryUpdates - 250000 worstcase 0.5 178.36 ms/op 223.38 ms/op 0.80
altair processRewardsAndPenalties - 250000 normalcase 109.27 ms/op 133.30 ms/op 0.82
altair processRewardsAndPenalties - 250000 worstcase 76.560 ms/op 134.96 ms/op 0.57
phase0 getAttestationDeltas - 250000 normalcase 12.848 ms/op 11.821 ms/op 1.09
phase0 getAttestationDeltas - 250000 worstcase 13.296 ms/op 11.953 ms/op 1.11
phase0 processSlashings - 250000 worstcase 5.0659 ms/op 5.2206 ms/op 0.97
altair processSyncCommitteeUpdates - 250000 294.15 ms/op 287.26 ms/op 1.02
BeaconState.hashTreeRoot - No change 546.00 ns/op 668.00 ns/op 0.82
BeaconState.hashTreeRoot - 1 full validator 72.610 us/op 69.693 us/op 1.04
BeaconState.hashTreeRoot - 32 full validator 760.12 us/op 685.67 us/op 1.11
BeaconState.hashTreeRoot - 512 full validator 8.4736 ms/op 8.9847 ms/op 0.94
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 90.988 us/op 90.442 us/op 1.01
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.3320 ms/op 1.4560 ms/op 0.91
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 17.697 ms/op 16.312 ms/op 1.08
BeaconState.hashTreeRoot - 1 balances 71.364 us/op 66.082 us/op 1.08
BeaconState.hashTreeRoot - 32 balances 640.94 us/op 677.34 us/op 0.95
BeaconState.hashTreeRoot - 512 balances 6.3118 ms/op 6.1355 ms/op 1.03
BeaconState.hashTreeRoot - 250000 balances 100.84 ms/op 101.79 ms/op 0.99
aggregationBits - 2048 els - zipIndexesInBitList 32.990 us/op 24.667 us/op 1.34
regular array get 100000 times 64.026 us/op 60.561 us/op 1.06
wrappedArray get 100000 times 61.610 us/op 60.563 us/op 1.02
arrayWithProxy get 100000 times 28.833 ms/op 32.169 ms/op 0.90
ssz.Root.equals 567.00 ns/op 604.00 ns/op 0.94
byteArrayEquals 560.00 ns/op 492.00 ns/op 1.14
shuffle list - 16384 els 11.209 ms/op 11.631 ms/op 0.96
shuffle list - 250000 els 171.26 ms/op 169.25 ms/op 1.01
processSlot - 1 slots 14.881 us/op 12.907 us/op 1.15
processSlot - 32 slots 1.9950 ms/op 2.1029 ms/op 0.95
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 375.89 us/op 401.50 us/op 0.94
getCommitteeAssignments - req 1 vs - 250000 vc 5.4755 ms/op 5.3850 ms/op 1.02
getCommitteeAssignments - req 100 vs - 250000 vc 7.9993 ms/op 7.8727 ms/op 1.02
getCommitteeAssignments - req 1000 vs - 250000 vc 8.7352 ms/op 8.4098 ms/op 1.04
computeProposers - vc 250000 17.435 ms/op 18.634 ms/op 0.94
computeEpochShuffling - vc 250000 185.02 ms/op 173.70 ms/op 1.07
getNextSyncCommittee - vc 250000 290.57 ms/op 283.07 ms/op 1.03

by benchmarkbot/action

@g11tech g11tech merged commit 78c93f7 into unstable Aug 2, 2022
@g11tech g11tech deleted the g11tech/separate-e2e-browser-workflows branch August 2, 2022 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Duplicate actions in the CI pipeline
2 participants
@g11tech @wemeetagain