Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use mix of local key manager and external signer for sim tests #4583

Merged
merged 11 commits into from Sep 23, 2022

Conversation

nazarhussain
Copy link
Contributor

Motivation

Use mix of both local key manager and external signer for sim tests.

Resolves #4276

@nazarhussain nazarhussain requested a review from a team as a code owner September 21, 2022 12:51
@nazarhussain nazarhussain self-assigned this Sep 21, 2022
@nazarhussain nazarhussain linked an issue Sep 21, 2022 that may be closed by this pull request
15 tasks
Base automatically changed from nh/4276-assertions-participation-per-slot to unstable September 23, 2022 07:40
@github-actions
Copy link
Contributor

github-actions bot commented Sep 23, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 4e0960a Previous: 8b90b2c Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.4884 ms/op 1.8226 ms/op 1.37
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 86.858 us/op 64.852 us/op 1.34
BLS verify - blst-native 2.7683 ms/op 2.1675 ms/op 1.28
BLS verifyMultipleSignatures 3 - blst-native 5.7647 ms/op 4.4757 ms/op 1.29
BLS verifyMultipleSignatures 8 - blst-native 12.192 ms/op 9.6761 ms/op 1.26
BLS verifyMultipleSignatures 32 - blst-native 45.537 ms/op 35.171 ms/op 1.29
BLS aggregatePubkeys 32 - blst-native 61.405 us/op 46.442 us/op 1.32
BLS aggregatePubkeys 128 - blst-native 238.62 us/op 181.88 us/op 1.31
getAttestationsForBlock 108.77 ms/op 78.659 ms/op 1.38
isKnown best case - 1 super set check 519.00 ns/op 497.00 ns/op 1.04
isKnown normal case - 2 super set checks 521.00 ns/op 486.00 ns/op 1.07
isKnown worse case - 16 super set checks 516.00 ns/op 489.00 ns/op 1.06
CheckpointStateCache - add get delete 11.141 us/op 8.6610 us/op 1.29
validate gossip signedAggregateAndProof - struct 6.3127 ms/op 5.0222 ms/op 1.26
validate gossip attestation - struct 2.9990 ms/op 2.3653 ms/op 1.27
pickEth1Vote - no votes 2.5526 ms/op 2.0988 ms/op 1.22
pickEth1Vote - max votes 22.264 ms/op 17.915 ms/op 1.24
pickEth1Vote - Eth1Data hashTreeRoot value x2048 14.681 ms/op 12.246 ms/op 1.20
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 24.299 ms/op 19.231 ms/op 1.26
pickEth1Vote - Eth1Data fastSerialize value x2048 1.8420 ms/op 1.3992 ms/op 1.32
pickEth1Vote - Eth1Data fastSerialize tree x2048 15.534 ms/op 11.914 ms/op 1.30
bytes32 toHexString 1.3300 us/op 939.00 ns/op 1.42
bytes32 Buffer.toString(hex) 845.00 ns/op 707.00 ns/op 1.20
bytes32 Buffer.toString(hex) from Uint8Array 1.1100 us/op 991.00 ns/op 1.12
bytes32 Buffer.toString(hex) + 0x 833.00 ns/op 696.00 ns/op 1.20
Object access 1 prop 0.45900 ns/op 0.33900 ns/op 1.35
Map access 1 prop 0.33100 ns/op 0.30500 ns/op 1.09
Object get x1000 17.197 ns/op 11.536 ns/op 1.49
Map get x1000 0.99700 ns/op 1.0390 ns/op 0.96
Object set x1000 126.64 ns/op 68.629 ns/op 1.85
Map set x1000 85.212 ns/op 46.281 ns/op 1.84
Return object 10000 times 0.43120 ns/op 0.44250 ns/op 0.97
Throw Error 10000 times 9.3186 us/op 6.0973 us/op 1.53
enrSubnets - fastDeserialize 64 bits 3.3570 us/op 2.5360 us/op 1.32
enrSubnets - ssz BitVector 64 bits 892.00 ns/op 812.00 ns/op 1.10
enrSubnets - fastDeserialize 4 bits 469.00 ns/op 370.00 ns/op 1.27
enrSubnets - ssz BitVector 4 bits 913.00 ns/op 783.00 ns/op 1.17
prioritizePeers score -10:0 att 32-0.1 sync 2-0 114.84 us/op 80.453 us/op 1.43
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 152.60 us/op 115.21 us/op 1.32
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 285.60 us/op 190.32 us/op 1.50
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 612.95 us/op 370.67 us/op 1.65
prioritizePeers score 0:0 att 64-1 sync 4-1 653.24 us/op 409.76 us/op 1.59
RateTracker 1000000 limit, 1 obj count per request 216.89 ns/op 178.68 ns/op 1.21
RateTracker 1000000 limit, 2 obj count per request 161.76 ns/op 130.19 ns/op 1.24
RateTracker 1000000 limit, 4 obj count per request 134.06 ns/op 105.61 ns/op 1.27
RateTracker 1000000 limit, 8 obj count per request 122.23 ns/op 92.657 ns/op 1.32
RateTracker with prune 5.6780 us/op 3.8630 us/op 1.47
array of 16000 items push then shift 5.3690 us/op 51.574 us/op 0.10
LinkedList of 16000 items push then shift 19.766 ns/op 12.076 ns/op 1.64
array of 16000 items push then pop 254.80 ns/op 211.29 ns/op 1.21
LinkedList of 16000 items push then pop 18.367 ns/op 12.091 ns/op 1.52
array of 24000 items push then shift 7.9043 us/op 77.357 us/op 0.10
LinkedList of 24000 items push then shift 19.668 ns/op 12.413 ns/op 1.58
array of 24000 items push then pop 223.76 ns/op 191.34 ns/op 1.17
LinkedList of 24000 items push then pop 18.432 ns/op 11.863 ns/op 1.55
intersect bitArray bitLen 8 12.657 ns/op 10.804 ns/op 1.17
intersect array and set length 8 206.91 ns/op 130.00 ns/op 1.59
intersect bitArray bitLen 128 72.786 ns/op 55.574 ns/op 1.31
intersect array and set length 128 2.5833 us/op 1.7266 us/op 1.50
Buffer.concat 32 items 2.3870 ns/op 1.9990 ns/op 1.19
pass gossip attestations to forkchoice per slot 6.0817 ms/op 2.9339 ms/op 2.07
computeDeltas 4.0522 ms/op 3.2759 ms/op 1.24
computeProposerBoostScoreFromBalances 910.60 us/op 803.99 us/op 1.13
altair processAttestation - 250000 vs - 7PWei normalcase 5.3981 ms/op 3.2542 ms/op 1.66
altair processAttestation - 250000 vs - 7PWei worstcase 8.0954 ms/op 4.9803 ms/op 1.63
altair processAttestation - setStatus - 1/6 committees join 262.96 us/op 175.39 us/op 1.50
altair processAttestation - setStatus - 1/3 committees join 485.26 us/op 349.39 us/op 1.39
altair processAttestation - setStatus - 1/2 committees join 701.06 us/op 502.80 us/op 1.39
altair processAttestation - setStatus - 2/3 committees join 948.82 us/op 656.07 us/op 1.45
altair processAttestation - setStatus - 4/5 committees join 1.2975 ms/op 922.38 us/op 1.41
altair processAttestation - setStatus - 100% committees join 1.5631 ms/op 1.1118 ms/op 1.41
altair processBlock - 250000 vs - 7PWei normalcase 30.910 ms/op 25.944 ms/op 1.19
altair processBlock - 250000 vs - 7PWei normalcase hashState 41.806 ms/op 35.906 ms/op 1.16
altair processBlock - 250000 vs - 7PWei worstcase 115.70 ms/op 87.368 ms/op 1.32
altair processBlock - 250000 vs - 7PWei worstcase hashState 125.86 ms/op 101.46 ms/op 1.24
phase0 processBlock - 250000 vs - 7PWei normalcase 4.4269 ms/op 3.2001 ms/op 1.38
phase0 processBlock - 250000 vs - 7PWei worstcase 64.831 ms/op 50.181 ms/op 1.29
altair processEth1Data - 250000 vs - 7PWei normalcase 1.1332 ms/op 736.71 us/op 1.54
Tree 40 250000 create 1.0319 s/op 681.66 ms/op 1.51
Tree 40 250000 get(125000) 327.74 ns/op 238.58 ns/op 1.37
Tree 40 250000 set(125000) 3.4226 us/op 2.1258 us/op 1.61
Tree 40 250000 toArray() 35.346 ms/op 27.421 ms/op 1.29
Tree 40 250000 iterate all - toArray() + loop 35.441 ms/op 26.961 ms/op 1.31
Tree 40 250000 iterate all - get(i) 132.43 ms/op 105.21 ms/op 1.26
MutableVector 250000 create 18.911 ms/op 13.054 ms/op 1.45
MutableVector 250000 get(125000) 14.701 ns/op 10.634 ns/op 1.38
MutableVector 250000 set(125000) 867.40 ns/op 499.60 ns/op 1.74
MutableVector 250000 toArray() 7.7095 ms/op 6.0058 ms/op 1.28
MutableVector 250000 iterate all - toArray() + loop 7.5979 ms/op 5.9924 ms/op 1.27
MutableVector 250000 iterate all - get(i) 3.6467 ms/op 2.7164 ms/op 1.34
Array 250000 create 7.3009 ms/op 5.9159 ms/op 1.23
Array 250000 clone - spread 4.4252 ms/op 3.2800 ms/op 1.35
Array 250000 get(125000) 1.8700 ns/op 1.4940 ns/op 1.25
Array 250000 set(125000) 1.8950 ns/op 1.5070 ns/op 1.26
Array 250000 iterate all - loop 151.74 us/op 152.75 us/op 0.99
effectiveBalanceIncrements clone Uint8Array 300000 316.44 us/op 91.483 us/op 3.46
effectiveBalanceIncrements clone MutableVector 300000 895.00 ns/op 675.00 ns/op 1.33
effectiveBalanceIncrements rw all Uint8Array 300000 309.65 us/op 247.33 us/op 1.25
effectiveBalanceIncrements rw all MutableVector 300000 225.36 ms/op 130.04 ms/op 1.73
phase0 afterProcessEpoch - 250000 vs - 7PWei 219.76 ms/op 195.82 ms/op 1.12
phase0 beforeProcessEpoch - 250000 vs - 7PWei 76.908 ms/op 56.956 ms/op 1.35
altair processEpoch - mainnet_e81889 676.32 ms/op 569.03 ms/op 1.19
mainnet_e81889 - altair beforeProcessEpoch 191.58 ms/op 119.80 ms/op 1.60
mainnet_e81889 - altair processJustificationAndFinalization 64.679 us/op 26.765 us/op 2.42
mainnet_e81889 - altair processInactivityUpdates 12.205 ms/op 8.8701 ms/op 1.38
mainnet_e81889 - altair processRewardsAndPenalties 101.52 ms/op 78.773 ms/op 1.29
mainnet_e81889 - altair processRegistryUpdates 13.944 us/op 3.6360 us/op 3.83
mainnet_e81889 - altair processSlashings 4.0100 us/op 765.00 ns/op 5.24
mainnet_e81889 - altair processEth1DataReset 3.8480 us/op 1.0120 us/op 3.80
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.5250 ms/op 2.0777 ms/op 1.22
mainnet_e81889 - altair processSlashingsReset 23.968 us/op 7.3520 us/op 3.26
mainnet_e81889 - altair processRandaoMixesReset 23.565 us/op 7.8440 us/op 3.00
mainnet_e81889 - altair processHistoricalRootsUpdate 3.8360 us/op 1.3540 us/op 2.83
mainnet_e81889 - altair processParticipationFlagUpdates 15.339 us/op 5.6490 us/op 2.72
mainnet_e81889 - altair processSyncCommitteeUpdates 3.4340 us/op 1.2370 us/op 2.78
mainnet_e81889 - altair afterProcessEpoch 216.94 ms/op 220.69 ms/op 0.98
phase0 processEpoch - mainnet_e58758 625.33 ms/op 621.72 ms/op 1.01
mainnet_e58758 - phase0 beforeProcessEpoch 278.79 ms/op 226.31 ms/op 1.23
mainnet_e58758 - phase0 processJustificationAndFinalization 56.131 us/op 22.986 us/op 2.44
mainnet_e58758 - phase0 processRewardsAndPenalties 154.76 ms/op 128.19 ms/op 1.21
mainnet_e58758 - phase0 processRegistryUpdates 31.280 us/op 14.182 us/op 2.21
mainnet_e58758 - phase0 processSlashings 2.9470 us/op 614.00 ns/op 4.80
mainnet_e58758 - phase0 processEth1DataReset 3.0780 us/op 892.00 ns/op 3.45
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.1197 ms/op 1.9367 ms/op 1.09
mainnet_e58758 - phase0 processSlashingsReset 14.853 us/op 6.0250 us/op 2.47
mainnet_e58758 - phase0 processRandaoMixesReset 23.261 us/op 7.8740 us/op 2.95
mainnet_e58758 - phase0 processHistoricalRootsUpdate 3.9650 us/op 1.0070 us/op 3.94
mainnet_e58758 - phase0 processParticipationRecordUpdates 21.706 us/op 5.8240 us/op 3.73
mainnet_e58758 - phase0 afterProcessEpoch 177.58 ms/op 164.33 ms/op 1.08
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.4863 ms/op 2.0070 ms/op 1.24
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.8203 ms/op 2.3574 ms/op 1.20
altair processInactivityUpdates - 250000 normalcase 51.066 ms/op 58.011 ms/op 0.88
altair processInactivityUpdates - 250000 worstcase 61.468 ms/op 55.723 ms/op 1.10
phase0 processRegistryUpdates - 250000 normalcase 25.664 us/op 6.9100 us/op 3.71
phase0 processRegistryUpdates - 250000 badcase_full_deposits 505.91 us/op 526.84 us/op 0.96
phase0 processRegistryUpdates - 250000 worstcase 0.5 239.28 ms/op 226.97 ms/op 1.05
altair processRewardsAndPenalties - 250000 normalcase 144.15 ms/op 134.59 ms/op 1.07
altair processRewardsAndPenalties - 250000 worstcase 96.140 ms/op 137.56 ms/op 0.70
phase0 getAttestationDeltas - 250000 normalcase 16.136 ms/op 11.596 ms/op 1.39
phase0 getAttestationDeltas - 250000 worstcase 13.611 ms/op 11.980 ms/op 1.14
phase0 processSlashings - 250000 worstcase 6.7453 ms/op 5.2813 ms/op 1.28
altair processSyncCommitteeUpdates - 250000 339.98 ms/op 284.59 ms/op 1.19
BeaconState.hashTreeRoot - No change 613.00 ns/op 564.00 ns/op 1.09
BeaconState.hashTreeRoot - 1 full validator 76.007 us/op 69.994 us/op 1.09
BeaconState.hashTreeRoot - 32 full validator 757.52 us/op 650.22 us/op 1.17
BeaconState.hashTreeRoot - 512 full validator 7.8242 ms/op 8.3159 ms/op 0.94
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 109.41 us/op 98.115 us/op 1.12
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.8761 ms/op 1.2313 ms/op 1.52
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 20.263 ms/op 16.355 ms/op 1.24
BeaconState.hashTreeRoot - 1 balances 80.876 us/op 80.564 us/op 1.00
BeaconState.hashTreeRoot - 32 balances 742.21 us/op 627.69 us/op 1.18
BeaconState.hashTreeRoot - 512 balances 7.3665 ms/op 6.7358 ms/op 1.09
BeaconState.hashTreeRoot - 250000 balances 112.46 ms/op 105.80 ms/op 1.06
aggregationBits - 2048 els - zipIndexesInBitList 41.815 us/op 23.956 us/op 1.75
regular array get 100000 times 59.109 us/op 60.734 us/op 0.97
wrappedArray get 100000 times 58.462 us/op 61.265 us/op 0.95
arrayWithProxy get 100000 times 35.807 ms/op 28.278 ms/op 1.27
ssz.Root.equals 607.00 ns/op 450.00 ns/op 1.35
byteArrayEquals 592.00 ns/op 444.00 ns/op 1.33
shuffle list - 16384 els 12.847 ms/op 11.524 ms/op 1.11
shuffle list - 250000 els 183.58 ms/op 169.23 ms/op 1.08
processSlot - 1 slots 18.176 us/op 12.889 us/op 1.41
processSlot - 32 slots 2.4842 ms/op 1.9276 ms/op 1.29
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 514.32 us/op 388.73 us/op 1.32
getCommitteeAssignments - req 1 vs - 250000 vc 5.6564 ms/op 5.3689 ms/op 1.05
getCommitteeAssignments - req 100 vs - 250000 vc 8.0173 ms/op 7.8165 ms/op 1.03
getCommitteeAssignments - req 1000 vs - 250000 vc 8.5526 ms/op 8.4409 ms/op 1.01
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 11.040 ns/op 8.4000 ns/op 1.31
state getBlockRootAtSlot - 250000 vs - 7PWei 1.3335 us/op 951.40 ns/op 1.40
computeProposers - vc 250000 20.483 ms/op 16.891 ms/op 1.21
computeEpochShuffling - vc 250000 185.83 ms/op 172.82 ms/op 1.08
getNextSyncCommittee - vc 250000 348.22 ms/op 279.13 ms/op 1.25

by benchmarkbot/action

@@ -58,6 +58,9 @@ export const LodestarValidatorProcess: ValidatorConstructor = class LodestarVali
});
this.secretKeys = validatorSecretKeys;

// Split half of the keys to external signer
this.externalSigner = new ExternalSignerServer(this.secretKeys.slice(0, this.secretKeys.length / 2));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we define and set the ranges to be divided locally and externally as validables (and the default split as half/half)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also this.rcConfig["externalSigner.url"] = this.externalSigner.url; isn't required for this using this external signer?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also this.rcConfig["externalSigner.url"] = this.externalSigner.url; isn't required for this using this external signer?

oh ok since we now start keymanager to use this external signer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@g11tech Done.

Copy link
Contributor

@g11tech g11tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@nazarhussain nazarhussain merged commit 8cac35f into unstable Sep 23, 2022
@nazarhussain nazarhussain deleted the nh/4276-split-keys branch September 23, 2022 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add assertions to sim tests
2 participants