Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace geth binary in sim merge tests with docker #4629

Merged
merged 6 commits into from Oct 3, 2022
Merged

Replace geth binary in sim merge tests with docker #4629

merged 6 commits into from Oct 3, 2022

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Oct 3, 2022 •
edited

Building geth takes time and resources (3-4 minutes of cloning + build), whill pulling geth image is so much easier. This would significantly reduce the merge sim time and will be followed up with nethermind build replacement if success.

geth build time:
image
image pull time:
image

similar for nethermind (will be addressed in a separate PR as docker setup for nethermind needs to be added)

@github-actions
Copy link
Contributor

github-actions bot commented Oct 3, 2022 •
edited

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 4469772 Previous: 07eb658 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.3058 ms/op 1.7634 ms/op 1.31
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 89.265 us/op 64.090 us/op 1.39
BLS verify - blst-native 2.3249 ms/op 2.1641 ms/op 1.07
BLS verifyMultipleSignatures 3 - blst-native 4.7655 ms/op 4.4728 ms/op 1.07
BLS verifyMultipleSignatures 8 - blst-native 10.268 ms/op 9.6648 ms/op 1.06
BLS verifyMultipleSignatures 32 - blst-native 37.268 ms/op 35.127 ms/op 1.06
BLS aggregatePubkeys 32 - blst-native 49.279 us/op 46.784 us/op 1.05
BLS aggregatePubkeys 128 - blst-native 191.34 us/op 182.35 us/op 1.05
getAttestationsForBlock 114.45 ms/op 73.302 ms/op 1.56
isKnown best case - 1 super set check 583.00 ns/op 469.00 ns/op 1.24
isKnown normal case - 2 super set checks 558.00 ns/op 460.00 ns/op 1.21
isKnown worse case - 16 super set checks 543.00 ns/op 462.00 ns/op 1.18
CheckpointStateCache - add get delete 10.986 us/op 8.6060 us/op 1.28
validate gossip signedAggregateAndProof - struct 5.3383 ms/op 5.0126 ms/op 1.06
validate gossip attestation - struct 2.5222 ms/op 2.3756 ms/op 1.06
pickEth1Vote - no votes 2.7200 ms/op 2.1011 ms/op 1.29
pickEth1Vote - max votes 24.064 ms/op 16.962 ms/op 1.42
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.507 ms/op 12.026 ms/op 1.12
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 26.332 ms/op 19.676 ms/op 1.34
pickEth1Vote - Eth1Data fastSerialize value x2048 2.0480 ms/op 1.4500 ms/op 1.41
pickEth1Vote - Eth1Data fastSerialize tree x2048 17.689 ms/op 12.777 ms/op 1.38
bytes32 toHexString 1.4020 us/op 1.0390 us/op 1.35
bytes32 Buffer.toString(hex) 947.00 ns/op 777.00 ns/op 1.22
bytes32 Buffer.toString(hex) from Uint8Array 1.2940 us/op 1.0180 us/op 1.27
bytes32 Buffer.toString(hex) + 0x 938.00 ns/op 783.00 ns/op 1.20
Object access 1 prop 0.49100 ns/op 0.37200 ns/op 1.32
Map access 1 prop 0.39300 ns/op 0.31000 ns/op 1.27
Object get x1000 21.547 ns/op 10.843 ns/op 1.99
Map get x1000 1.2720 ns/op 0.93500 ns/op 1.36
Object set x1000 143.57 ns/op 74.087 ns/op 1.94
Map set x1000 86.253 ns/op 50.387 ns/op 1.71
Return object 10000 times 0.45760 ns/op 0.43920 ns/op 1.04
Throw Error 10000 times 7.4816 us/op 5.9921 us/op 1.25
enrSubnets - fastDeserialize 64 bits 3.2920 us/op 2.5490 us/op 1.29
enrSubnets - ssz BitVector 64 bits 891.00 ns/op 820.00 ns/op 1.09
enrSubnets - fastDeserialize 4 bits 482.00 ns/op 383.00 ns/op 1.26
enrSubnets - ssz BitVector 4 bits 932.00 ns/op 802.00 ns/op 1.16
prioritizePeers score -10:0 att 32-0.1 sync 2-0 119.61 us/op 82.853 us/op 1.44
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 169.85 us/op 124.71 us/op 1.36
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 277.10 us/op 209.24 us/op 1.32
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 487.65 us/op 332.89 us/op 1.46
prioritizePeers score 0:0 att 64-1 sync 4-1 573.94 us/op 406.72 us/op 1.41
RateTracker 1000000 limit, 1 obj count per request 269.44 ns/op 181.00 ns/op 1.49
RateTracker 1000000 limit, 2 obj count per request 199.03 ns/op 131.84 ns/op 1.51
RateTracker 1000000 limit, 4 obj count per request 169.31 ns/op 109.11 ns/op 1.55
RateTracker 1000000 limit, 8 obj count per request 147.91 ns/op 96.671 ns/op 1.53
RateTracker with prune 5.3940 us/op 4.0320 us/op 1.34
array of 16000 items push then shift 3.8489 us/op 51.581 us/op 0.07
LinkedList of 16000 items push then shift 20.899 ns/op 12.498 ns/op 1.67
array of 16000 items push then pop 313.77 ns/op 219.09 ns/op 1.43
LinkedList of 16000 items push then pop 22.543 ns/op 11.949 ns/op 1.89
array of 24000 items push then shift 5.6835 us/op 77.381 us/op 0.07
LinkedList of 24000 items push then shift 23.527 ns/op 12.926 ns/op 1.82
array of 24000 items push then pop 264.24 ns/op 197.78 ns/op 1.34
LinkedList of 24000 items push then pop 22.930 ns/op 12.152 ns/op 1.89
intersect bitArray bitLen 8 15.792 ns/op 10.812 ns/op 1.46
intersect array and set length 8 218.64 ns/op 147.25 ns/op 1.48
intersect bitArray bitLen 128 95.030 ns/op 55.618 ns/op 1.71
intersect array and set length 128 2.8609 us/op 1.9621 us/op 1.46
Buffer.concat 32 items 2.3790 ns/op 1.7850 ns/op 1.33
pass gossip attestations to forkchoice per slot 5.0566 ms/op 3.6205 ms/op 1.40
computeDeltas 7.4806 ms/op 4.9079 ms/op 1.52
computeProposerBoostScoreFromBalances 1.1299 ms/op 806.62 us/op 1.40
altair processAttestation - 250000 vs - 7PWei normalcase 4.2157 ms/op 3.6434 ms/op 1.16
altair processAttestation - 250000 vs - 7PWei worstcase 7.5023 ms/op 5.5216 ms/op 1.36
altair processAttestation - setStatus - 1/6 committees join 264.21 us/op 196.19 us/op 1.35
altair processAttestation - setStatus - 1/3 committees join 524.17 us/op 363.55 us/op 1.44
altair processAttestation - setStatus - 1/2 committees join 722.64 us/op 523.26 us/op 1.38
altair processAttestation - setStatus - 2/3 committees join 953.52 us/op 679.29 us/op 1.40
altair processAttestation - setStatus - 4/5 committees join 1.2721 ms/op 942.80 us/op 1.35
altair processAttestation - setStatus - 100% committees join 1.5142 ms/op 1.1478 ms/op 1.32
altair processBlock - 250000 vs - 7PWei normalcase 36.381 ms/op 26.100 ms/op 1.39
altair processBlock - 250000 vs - 7PWei normalcase hashState 46.945 ms/op 40.654 ms/op 1.15
altair processBlock - 250000 vs - 7PWei worstcase 123.73 ms/op 77.288 ms/op 1.60
altair processBlock - 250000 vs - 7PWei worstcase hashState 124.01 ms/op 103.45 ms/op 1.20
phase0 processBlock - 250000 vs - 7PWei normalcase 4.6720 ms/op 3.3250 ms/op 1.41
phase0 processBlock - 250000 vs - 7PWei worstcase 64.544 ms/op 50.435 ms/op 1.28
altair processEth1Data - 250000 vs - 7PWei normalcase 1.0573 ms/op 672.80 us/op 1.57
Tree 40 250000 create 1.0514 s/op 668.08 ms/op 1.57
Tree 40 250000 get(125000) 394.29 ns/op 253.31 ns/op 1.56
Tree 40 250000 set(125000) 3.2581 us/op 2.3162 us/op 1.41
Tree 40 250000 toArray() 42.560 ms/op 27.543 ms/op 1.55
Tree 40 250000 iterate all - toArray() + loop 43.503 ms/op 27.350 ms/op 1.59
Tree 40 250000 iterate all - get(i) 140.52 ms/op 113.66 ms/op 1.24
MutableVector 250000 create 19.000 ms/op 13.965 ms/op 1.36
MutableVector 250000 get(125000) 16.331 ns/op 11.970 ns/op 1.36
MutableVector 250000 set(125000) 839.68 ns/op 444.90 ns/op 1.89
MutableVector 250000 toArray() 9.4637 ms/op 5.9190 ms/op 1.60
MutableVector 250000 iterate all - toArray() + loop 9.1425 ms/op 5.9859 ms/op 1.53
MutableVector 250000 iterate all - get(i) 4.1051 ms/op 2.6220 ms/op 1.57
Array 250000 create 8.9199 ms/op 5.3496 ms/op 1.67
Array 250000 clone - spread 5.0586 ms/op 2.4267 ms/op 2.08
Array 250000 get(125000) 2.0890 ns/op 1.1720 ns/op 1.78
Array 250000 set(125000) 2.1560 ns/op 1.1650 ns/op 1.85
Array 250000 iterate all - loop 209.82 us/op 154.00 us/op 1.36
effectiveBalanceIncrements clone Uint8Array 300000 108.00 us/op 37.550 us/op 2.88
effectiveBalanceIncrements clone MutableVector 300000 902.00 ns/op 741.00 ns/op 1.22
effectiveBalanceIncrements rw all Uint8Array 300000 315.81 us/op 248.52 us/op 1.27
effectiveBalanceIncrements rw all MutableVector 300000 195.48 ms/op 142.45 ms/op 1.37
phase0 afterProcessEpoch - 250000 vs - 7PWei 265.39 ms/op 186.15 ms/op 1.43
phase0 beforeProcessEpoch - 250000 vs - 7PWei 90.398 ms/op 60.598 ms/op 1.49
altair processEpoch - mainnet_e81889 762.77 ms/op 573.47 ms/op 1.33
mainnet_e81889 - altair beforeProcessEpoch 190.66 ms/op 107.88 ms/op 1.77
mainnet_e81889 - altair processJustificationAndFinalization 46.101 us/op 16.936 us/op 2.72
mainnet_e81889 - altair processInactivityUpdates 14.169 ms/op 8.4487 ms/op 1.68
mainnet_e81889 - altair processRewardsAndPenalties 179.13 ms/op 76.585 ms/op 2.34
mainnet_e81889 - altair processRegistryUpdates 10.722 us/op 2.4960 us/op 4.30
mainnet_e81889 - altair processSlashings 2.8780 us/op 513.00 ns/op 5.61
mainnet_e81889 - altair processEth1DataReset 2.6830 us/op 613.00 ns/op 4.38
mainnet_e81889 - altair processEffectiveBalanceUpdates 3.6725 ms/op 1.9309 ms/op 1.90
mainnet_e81889 - altair processSlashingsReset 14.971 us/op 4.8170 us/op 3.11
mainnet_e81889 - altair processRandaoMixesReset 16.995 us/op 3.8420 us/op 4.42
mainnet_e81889 - altair processHistoricalRootsUpdate 2.9130 us/op 607.00 ns/op 4.80
mainnet_e81889 - altair processParticipationFlagUpdates 12.348 us/op 2.0420 us/op 6.05
mainnet_e81889 - altair processSyncCommitteeUpdates 1.8940 us/op 610.00 ns/op 3.10
mainnet_e81889 - altair afterProcessEpoch 244.43 ms/op 196.70 ms/op 1.24
phase0 processEpoch - mainnet_e58758 766.54 ms/op 480.53 ms/op 1.60
mainnet_e58758 - phase0 beforeProcessEpoch 288.37 ms/op 175.63 ms/op 1.64
mainnet_e58758 - phase0 processJustificationAndFinalization 53.886 us/op 16.940 us/op 3.18
mainnet_e58758 - phase0 processRewardsAndPenalties 163.41 ms/op 96.498 ms/op 1.69
mainnet_e58758 - phase0 processRegistryUpdates 25.466 us/op 8.1530 us/op 3.12
mainnet_e58758 - phase0 processSlashings 2.1590 us/op 549.00 ns/op 3.93
mainnet_e58758 - phase0 processEth1DataReset 2.8160 us/op 558.00 ns/op 5.05
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.9863 ms/op 1.7314 ms/op 1.72
mainnet_e58758 - phase0 processSlashingsReset 12.263 us/op 3.6980 us/op 3.32
mainnet_e58758 - phase0 processRandaoMixesReset 18.433 us/op 4.2550 us/op 4.33
mainnet_e58758 - phase0 processHistoricalRootsUpdate 2.5280 us/op 691.00 ns/op 3.66
mainnet_e58758 - phase0 processParticipationRecordUpdates 14.149 us/op 3.3680 us/op 4.20
mainnet_e58758 - phase0 afterProcessEpoch 211.92 ms/op 161.83 ms/op 1.31
phase0 processEffectiveBalanceUpdates - 250000 normalcase 3.2702 ms/op 1.9459 ms/op 1.68
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 4.6660 ms/op 2.2227 ms/op 2.10
altair processInactivityUpdates - 250000 normalcase 55.237 ms/op 40.220 ms/op 1.37
altair processInactivityUpdates - 250000 worstcase 71.413 ms/op 32.717 ms/op 2.18
phase0 processRegistryUpdates - 250000 normalcase 17.162 us/op 6.0430 us/op 2.84
phase0 processRegistryUpdates - 250000 badcase_full_deposits 582.36 us/op 382.11 us/op 1.52
phase0 processRegistryUpdates - 250000 worstcase 0.5 273.46 ms/op 190.41 ms/op 1.44
altair processRewardsAndPenalties - 250000 normalcase 181.38 ms/op 72.778 ms/op 2.49
altair processRewardsAndPenalties - 250000 worstcase 110.23 ms/op 106.14 ms/op 1.04
phase0 getAttestationDeltas - 250000 normalcase 17.404 ms/op 11.351 ms/op 1.53
phase0 getAttestationDeltas - 250000 worstcase 17.548 ms/op 11.919 ms/op 1.47
phase0 processSlashings - 250000 worstcase 7.2695 ms/op 4.9920 ms/op 1.46
altair processSyncCommitteeUpdates - 250000 377.86 ms/op 287.40 ms/op 1.31
BeaconState.hashTreeRoot - No change 642.00 ns/op 542.00 ns/op 1.18
BeaconState.hashTreeRoot - 1 full validator 78.979 us/op 65.876 us/op 1.20
BeaconState.hashTreeRoot - 32 full validator 858.21 us/op 729.80 us/op 1.18
BeaconState.hashTreeRoot - 512 full validator 18.284 ms/op 6.9788 ms/op 2.62
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 94.363 us/op 88.774 us/op 1.06
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.5642 ms/op 1.2759 ms/op 1.23
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 20.379 ms/op 17.690 ms/op 1.15
BeaconState.hashTreeRoot - 1 balances 78.059 us/op 71.950 us/op 1.08
BeaconState.hashTreeRoot - 32 balances 714.51 us/op 645.71 us/op 1.11
BeaconState.hashTreeRoot - 512 balances 7.4847 ms/op 6.4848 ms/op 1.15
BeaconState.hashTreeRoot - 250000 balances 119.20 ms/op 105.70 ms/op 1.13
aggregationBits - 2048 els - zipIndexesInBitList 46.190 us/op 25.712 us/op 1.80
regular array get 100000 times 85.810 us/op 61.203 us/op 1.40
wrappedArray get 100000 times 84.262 us/op 60.549 us/op 1.39
arrayWithProxy get 100000 times 37.170 ms/op 28.793 ms/op 1.29
ssz.Root.equals 612.00 ns/op 480.00 ns/op 1.27
byteArrayEquals 600.00 ns/op 471.00 ns/op 1.27
shuffle list - 16384 els 15.439 ms/op 11.449 ms/op 1.35
shuffle list - 250000 els 224.69 ms/op 166.22 ms/op 1.35
processSlot - 1 slots 19.047 us/op 13.517 us/op 1.41
processSlot - 32 slots 2.2592 ms/op 1.9816 ms/op 1.14
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 470.86 us/op 395.03 us/op 1.19
getCommitteeAssignments - req 1 vs - 250000 vc 6.9896 ms/op 5.5712 ms/op 1.25
getCommitteeAssignments - req 100 vs - 250000 vc 9.2633 ms/op 8.1910 ms/op 1.13
getCommitteeAssignments - req 1000 vs - 250000 vc 9.8200 ms/op 8.6811 ms/op 1.13
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 13.220 ns/op 8.8700 ns/op 1.49
state getBlockRootAtSlot - 250000 vs - 7PWei 1.3454 us/op 1.0911 us/op 1.23
computeProposers - vc 250000 21.304 ms/op 17.210 ms/op 1.24
computeEpochShuffling - vc 250000 219.31 ms/op 170.28 ms/op 1.29
getNextSyncCommittee - vc 250000 348.70 ms/op 287.20 ms/op 1.21

by benchmarkbot/action

@g11tech g11tech marked this pull request as ready for review October 3, 2022 20:04
@g11tech g11tech requested a review from a team as a code owner October 3, 2022 20:04
@g11tech @dapplion
icnorporate suggestion
d7d9645 
Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
@g11tech g11tech enabled auto-merge (squash) October 3, 2022 20:26
@g11tech g11tech merged commit 19b5fe1 into unstable Oct 3, 2022
@g11tech g11tech deleted the g11tech/sim-merge-geth-docker branch October 3, 2022 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@g11tech @dapplion