Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove testcontainers dependency for web3signer test #4567

Closed
wants to merge 3 commits into from
Closed

Remove testcontainers dependency for web3signer test #4567

wants to merge 3 commits into from

Conversation

dapplion
Copy link
Contributor

Motivation

I really want to start decreasing our dependencies exposure, not increasing it unless required. We have rolled our docker runners in the past and should continue to do so.

Description

  • Use adhoc docker

@dadepo I've removed the use of --network=host. If you have to run this tests locally try to understand the incompatibilities and fix them together without 3rd party libraries

@dapplion dapplion requested a review from a team as a code owner September 19, 2022 12:22
@dapplion dapplion changed the title Use adhoc docker Remove testcontainers dependency for web3signer test Sep 19, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Sep 19, 2022 •
edited

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 9ef5034 Previous: 8709e3a Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.0222 ms/op 2.2726 ms/op 0.89
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 64.627 us/op 85.034 us/op 0.76
BLS verify - blst-native 1.6398 ms/op 1.8825 ms/op 0.87
BLS verifyMultipleSignatures 3 - blst-native 3.3554 ms/op 3.8568 ms/op 0.87
BLS verifyMultipleSignatures 8 - blst-native 7.2232 ms/op 8.3030 ms/op 0.87
BLS verifyMultipleSignatures 32 - blst-native 26.180 ms/op 30.109 ms/op 0.87
BLS aggregatePubkeys 32 - blst-native 34.879 us/op 39.878 us/op 0.87
BLS aggregatePubkeys 128 - blst-native 135.23 us/op 154.58 us/op 0.87
getAttestationsForBlock 82.174 ms/op 98.900 ms/op 0.83
isKnown best case - 1 super set check 435.00 ns/op 440.00 ns/op 0.99
isKnown normal case - 2 super set checks 425.00 ns/op 419.00 ns/op 1.01
isKnown worse case - 16 super set checks 424.00 ns/op 420.00 ns/op 1.01
CheckpointStateCache - add get delete 8.9270 us/op 9.4480 us/op 0.94
validate gossip signedAggregateAndProof - struct 4.2520 ms/op 4.3875 ms/op 0.97
validate gossip attestation - struct 2.0347 ms/op 2.0799 ms/op 0.98
pickEth1Vote - no votes 2.1913 ms/op 2.9606 ms/op 0.74
pickEth1Vote - max votes 19.643 ms/op 25.381 ms/op 0.77
pickEth1Vote - Eth1Data hashTreeRoot value x2048 10.082 ms/op 11.991 ms/op 0.84
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 18.514 ms/op 23.462 ms/op 0.79
pickEth1Vote - Eth1Data fastSerialize value x2048 1.3556 ms/op 1.6052 ms/op 0.84
pickEth1Vote - Eth1Data fastSerialize tree x2048 13.362 ms/op 16.664 ms/op 0.80
bytes32 toHexString 965.00 ns/op 1.2770 us/op 0.76
bytes32 Buffer.toString(hex) 634.00 ns/op 767.00 ns/op 0.83
bytes32 Buffer.toString(hex) from Uint8Array 852.00 ns/op 1.0340 us/op 0.82
bytes32 Buffer.toString(hex) + 0x 631.00 ns/op 789.00 ns/op 0.80
Object access 1 prop 0.30800 ns/op 0.42600 ns/op 0.72
Map access 1 prop 0.25700 ns/op 0.28900 ns/op 0.89
Object get x1000 15.999 ns/op 17.974 ns/op 0.89
Map get x1000 0.89200 ns/op 0.99200 ns/op 0.90
Object set x1000 105.45 ns/op 134.97 ns/op 0.78
Map set x1000 63.959 ns/op 84.185 ns/op 0.76
Return object 10000 times 0.33380 ns/op 0.37530 ns/op 0.89
Throw Error 10000 times 5.1964 us/op 5.9733 us/op 0.87
enrSubnets - fastDeserialize 64 bits 2.3550 us/op 3.2570 us/op 0.72
enrSubnets - ssz BitVector 64 bits 661.00 ns/op 834.00 ns/op 0.79
enrSubnets - fastDeserialize 4 bits 352.00 ns/op 431.00 ns/op 0.82
enrSubnets - ssz BitVector 4 bits 669.00 ns/op 858.00 ns/op 0.78
prioritizePeers score -10:0 att 32-0.1 sync 2-0 81.743 us/op 106.93 us/op 0.76
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 117.89 us/op 142.90 us/op 0.82
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 195.46 us/op 253.84 us/op 0.77
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 341.72 us/op 392.85 us/op 0.87
prioritizePeers score 0:0 att 64-1 sync 4-1 407.04 us/op 464.72 us/op 0.88
RateTracker 1000000 limit, 1 obj count per request 180.84 ns/op 214.54 ns/op 0.84
RateTracker 1000000 limit, 2 obj count per request 141.45 ns/op 167.47 ns/op 0.84
RateTracker 1000000 limit, 4 obj count per request 122.22 ns/op 146.13 ns/op 0.84
RateTracker 1000000 limit, 8 obj count per request 112.73 ns/op 132.82 ns/op 0.85
RateTracker with prune 4.2450 us/op 5.7180 us/op 0.74
array of 16000 items push then shift 2.7287 us/op 3.2262 us/op 0.85
LinkedList of 16000 items push then shift 15.950 ns/op 18.030 ns/op 0.88
array of 16000 items push then pop 226.01 ns/op 266.67 ns/op 0.85
LinkedList of 16000 items push then pop 14.858 ns/op 17.137 ns/op 0.87
array of 24000 items push then shift 4.0358 us/op 4.6542 us/op 0.87
LinkedList of 24000 items push then shift 18.618 ns/op 21.458 ns/op 0.87
array of 24000 items push then pop 203.86 ns/op 236.47 ns/op 0.86
LinkedList of 24000 items push then pop 18.274 ns/op 18.616 ns/op 0.98
intersect bitArray bitLen 8 11.770 ns/op 11.679 ns/op 1.01
intersect array and set length 8 174.24 ns/op 199.97 ns/op 0.87
intersect bitArray bitLen 128 61.923 ns/op 72.499 ns/op 0.85
intersect array and set length 128 2.3202 us/op 2.4437 us/op 0.95
Buffer.concat 32 items 2.2380 ns/op 1.9930 ns/op 1.12
pass gossip attestations to forkchoice per slot 3.1519 ms/op 3.6377 ms/op 0.87
computeDeltas 3.4072 ms/op 3.1617 ms/op 1.08
computeProposerBoostScoreFromBalances 812.76 us/op 914.51 us/op 0.89
altair processAttestation - 250000 vs - 7PWei normalcase 3.2025 ms/op 4.4580 ms/op 0.72
altair processAttestation - 250000 vs - 7PWei worstcase 4.9510 ms/op 6.7126 ms/op 0.74
altair processAttestation - setStatus - 1/6 committees join 185.69 us/op 228.91 us/op 0.81
altair processAttestation - setStatus - 1/3 committees join 355.94 us/op 425.20 us/op 0.84
altair processAttestation - setStatus - 1/2 committees join 556.44 us/op 594.43 us/op 0.94
altair processAttestation - setStatus - 2/3 committees join 727.53 us/op 801.28 us/op 0.91
altair processAttestation - setStatus - 4/5 committees join 991.44 us/op 1.0664 ms/op 0.93
altair processAttestation - setStatus - 100% committees join 1.1714 ms/op 1.2616 ms/op 0.93
altair processBlock - 250000 vs - 7PWei normalcase 25.769 ms/op 30.163 ms/op 0.85
altair processBlock - 250000 vs - 7PWei normalcase hashState 39.630 ms/op 37.941 ms/op 1.04
altair processBlock - 250000 vs - 7PWei worstcase 75.837 ms/op 93.205 ms/op 0.81
altair processBlock - 250000 vs - 7PWei worstcase hashState 106.22 ms/op 99.096 ms/op 1.07
phase0 processBlock - 250000 vs - 7PWei normalcase 3.4381 ms/op 4.2678 ms/op 0.81
phase0 processBlock - 250000 vs - 7PWei worstcase 45.243 ms/op 48.020 ms/op 0.94
altair processEth1Data - 250000 vs - 7PWei normalcase 770.12 us/op 1.0748 ms/op 0.72
Tree 40 250000 create 754.12 ms/op 1.1181 s/op 0.67
Tree 40 250000 get(125000) 253.01 ns/op 294.40 ns/op 0.86
Tree 40 250000 set(125000) 2.4956 us/op 3.5923 us/op 0.69
Tree 40 250000 toArray() 30.045 ms/op 33.267 ms/op 0.90
Tree 40 250000 iterate all - toArray() + loop 30.566 ms/op 32.901 ms/op 0.93
Tree 40 250000 iterate all - get(i) 100.42 ms/op 112.08 ms/op 0.90
MutableVector 250000 create 16.145 ms/op 16.934 ms/op 0.95
MutableVector 250000 get(125000) 13.110 ns/op 14.920 ns/op 0.88
MutableVector 250000 set(125000) 642.38 ns/op 907.45 ns/op 0.71
MutableVector 250000 toArray() 7.2206 ms/op 7.9888 ms/op 0.90
MutableVector 250000 iterate all - toArray() + loop 7.3013 ms/op 8.1623 ms/op 0.89
MutableVector 250000 iterate all - get(i) 3.4414 ms/op 3.4788 ms/op 0.99
Array 250000 create 6.8358 ms/op 7.3719 ms/op 0.93
Array 250000 clone - spread 4.0637 ms/op 5.7128 ms/op 0.71
Array 250000 get(125000) 1.7400 ns/op 1.7920 ns/op 0.97
Array 250000 set(125000) 1.7370 ns/op 1.7590 ns/op 0.99
Array 250000 iterate all - loop 170.39 us/op 170.56 us/op 1.00
effectiveBalanceIncrements clone Uint8Array 300000 84.277 us/op 93.430 us/op 0.90
effectiveBalanceIncrements clone MutableVector 300000 706.00 ns/op 1.3360 us/op 0.53
effectiveBalanceIncrements rw all Uint8Array 300000 252.38 us/op 254.72 us/op 0.99
effectiveBalanceIncrements rw all MutableVector 300000 156.87 ms/op 240.64 ms/op 0.65
phase0 afterProcessEpoch - 250000 vs - 7PWei 197.08 ms/op 202.99 ms/op 0.97
phase0 beforeProcessEpoch - 250000 vs - 7PWei 71.213 ms/op 77.585 ms/op 0.92
altair processEpoch - mainnet_e81889 554.07 ms/op 608.57 ms/op 0.91
mainnet_e81889 - altair beforeProcessEpoch 140.93 ms/op 150.38 ms/op 0.94
mainnet_e81889 - altair processJustificationAndFinalization 23.247 us/op 25.842 us/op 0.90
mainnet_e81889 - altair processInactivityUpdates 10.422 ms/op 10.876 ms/op 0.96
mainnet_e81889 - altair processRewardsAndPenalties 92.069 ms/op 97.267 ms/op 0.95
mainnet_e81889 - altair processRegistryUpdates 4.8210 us/op 4.5440 us/op 1.06
mainnet_e81889 - altair processSlashings 913.00 ns/op 1.0210 us/op 0.89
mainnet_e81889 - altair processEth1DataReset 984.00 ns/op 1.0870 us/op 0.91
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.4372 ms/op 2.3044 ms/op 1.06
mainnet_e81889 - altair processSlashingsReset 6.7470 us/op 7.0060 us/op 0.96
mainnet_e81889 - altair processRandaoMixesReset 7.3830 us/op 7.8360 us/op 0.94
mainnet_e81889 - altair processHistoricalRootsUpdate 830.00 ns/op 993.00 ns/op 0.84
mainnet_e81889 - altair processParticipationFlagUpdates 3.5930 us/op 4.2250 us/op 0.85
mainnet_e81889 - altair processSyncCommitteeUpdates 860.00 ns/op 861.00 ns/op 1.00
mainnet_e81889 - altair afterProcessEpoch 174.50 ms/op 200.51 ms/op 0.87
phase0 processEpoch - mainnet_e58758 501.60 ms/op 549.18 ms/op 0.91
mainnet_e58758 - phase0 beforeProcessEpoch 222.04 ms/op 247.21 ms/op 0.90
mainnet_e58758 - phase0 processJustificationAndFinalization 26.124 us/op 26.690 us/op 0.98
mainnet_e58758 - phase0 processRewardsAndPenalties 134.02 ms/op 146.34 ms/op 0.92
mainnet_e58758 - phase0 processRegistryUpdates 12.292 us/op 12.111 us/op 1.01
mainnet_e58758 - phase0 processSlashings 1.0840 us/op 903.00 ns/op 1.20
mainnet_e58758 - phase0 processEth1DataReset 1.0710 us/op 952.00 ns/op 1.13
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.1569 ms/op 2.1789 ms/op 0.99
mainnet_e58758 - phase0 processSlashingsReset 7.1740 us/op 6.5810 us/op 1.09
mainnet_e58758 - phase0 processRandaoMixesReset 8.5640 us/op 7.6350 us/op 1.12
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.4840 us/op 1.0740 us/op 1.38
mainnet_e58758 - phase0 processParticipationRecordUpdates 5.6790 us/op 6.5570 us/op 0.87
mainnet_e58758 - phase0 afterProcessEpoch 142.14 ms/op 162.71 ms/op 0.87
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.7685 ms/op 2.6840 ms/op 1.03
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.4136 ms/op 3.5223 ms/op 0.97
altair processInactivityUpdates - 250000 normalcase 47.456 ms/op 44.367 ms/op 1.07
altair processInactivityUpdates - 250000 worstcase 38.274 ms/op 54.059 ms/op 0.71
phase0 processRegistryUpdates - 250000 normalcase 10.520 us/op 10.129 us/op 1.04
phase0 processRegistryUpdates - 250000 badcase_full_deposits 480.08 us/op 440.28 us/op 1.09
phase0 processRegistryUpdates - 250000 worstcase 0.5 207.71 ms/op 227.31 ms/op 0.91
altair processRewardsAndPenalties - 250000 normalcase 78.602 ms/op 136.34 ms/op 0.58
altair processRewardsAndPenalties - 250000 worstcase 80.028 ms/op 90.286 ms/op 0.89
phase0 getAttestationDeltas - 250000 normalcase 11.628 ms/op 16.316 ms/op 0.71
phase0 getAttestationDeltas - 250000 worstcase 12.177 ms/op 16.513 ms/op 0.74
phase0 processSlashings - 250000 worstcase 5.3292 ms/op 5.6272 ms/op 0.95
altair processSyncCommitteeUpdates - 250000 269.76 ms/op 300.73 ms/op 0.90
BeaconState.hashTreeRoot - No change 499.00 ns/op 480.00 ns/op 1.04
BeaconState.hashTreeRoot - 1 full validator 57.373 us/op 63.186 us/op 0.91
BeaconState.hashTreeRoot - 32 full validator 572.06 us/op 784.36 us/op 0.73
BeaconState.hashTreeRoot - 512 full validator 7.2641 ms/op 6.6026 ms/op 1.10
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 69.855 us/op 80.821 us/op 0.86
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.1557 ms/op 1.2353 ms/op 0.94
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 14.635 ms/op 17.137 ms/op 0.85
BeaconState.hashTreeRoot - 1 balances 57.768 us/op 53.144 us/op 1.09
BeaconState.hashTreeRoot - 32 balances 570.57 us/op 580.55 us/op 0.98
BeaconState.hashTreeRoot - 512 balances 5.4472 ms/op 6.3355 ms/op 0.86
BeaconState.hashTreeRoot - 250000 balances 78.536 ms/op 96.927 ms/op 0.81
aggregationBits - 2048 els - zipIndexesInBitList 34.459 us/op 43.217 us/op 0.80
regular array get 100000 times 67.419 us/op 68.193 us/op 0.99
wrappedArray get 100000 times 67.401 us/op 68.112 us/op 0.99
arrayWithProxy get 100000 times 29.042 ms/op 29.500 ms/op 0.98
ssz.Root.equals 410.00 ns/op 505.00 ns/op 0.81
byteArrayEquals 408.00 ns/op 490.00 ns/op 0.83
shuffle list - 16384 els 11.215 ms/op 11.193 ms/op 1.00
shuffle list - 250000 els 165.09 ms/op 169.36 ms/op 0.97
processSlot - 1 slots 11.298 us/op 13.208 us/op 0.86
processSlot - 32 slots 1.6822 ms/op 1.8552 ms/op 0.91
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 396.45 us/op 562.84 us/op 0.70
getCommitteeAssignments - req 1 vs - 250000 vc 5.2731 ms/op 5.5192 ms/op 0.96
getCommitteeAssignments - req 100 vs - 250000 vc 7.3187 ms/op 7.5594 ms/op 0.97
getCommitteeAssignments - req 1000 vs - 250000 vc 7.7582 ms/op 7.9836 ms/op 0.97
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 9.0800 ns/op 10.230 ns/op 0.89
state getBlockRootAtSlot - 250000 vs - 7PWei 1.0130 us/op 1.4074 us/op 0.72
computeProposers - vc 250000 14.940 ms/op 18.051 ms/op 0.83
computeEpochShuffling - vc 250000 162.76 ms/op 173.49 ms/op 0.94
getNextSyncCommittee - vc 250000 269.72 ms/op 293.89 ms/op 0.92

by benchmarkbot/action

@wemeetagain
Copy link
Member

setting this as draft

@wemeetagain wemeetagain marked this pull request as draft October 30, 2022 18:02
@dapplion
Copy link
Contributor Author

Tests work fine now

@dapplion dapplion closed this May 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@dapplion @wemeetagain