Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make dev command handler transparent #4314

Merged
merged 2 commits into from Jul 18, 2022
Merged

Make dev command handler transparent #4314

merged 2 commits into from Jul 18, 2022

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Jul 17, 2022 •
edited

Motivation

dev command features duplicated initialization logic from beacon and validator. Hard to test some features like keymanager server for #4313

Description

Make dev command handler transparent

  • just call beacon cmd handler as is
  • then, just call validator cmd handler as is

lodestar/packages/cli/src/cmds/dev/handler.ts

Lines 65 to 77 in e6e2736

// Note: recycle entire beacon handler
await beaconHandler(args);
if (args.startValidators) {
// TODO: Map dev option to validator's option
args.interopIndexes = args.startValidators;
// Note: recycle entire validator handler:
// - keystore handling
// - metrics
// - keymanager server
await validatorHandler(args);
}

@dapplion dapplion requested a review from a team as a code owner July 17, 2022 13:57
@github-actions
Copy link
Contributor

github-actions bot commented Jul 17, 2022 •
edited

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite Current: d384f7f Previous: 57673fe Ratio
altair processAttestation - setStatus - 1/6 committees join 172.93 us/op 584.04 us/op 0.30
Full benchmark results
Benchmark suite Current: d384f7f Previous: 57673fe Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.7251 ms/op 2.5853 ms/op 0.67
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 64.529 us/op 79.186 us/op 0.81
BLS verify - blst-native 2.1627 ms/op 1.8551 ms/op 1.17
BLS verifyMultipleSignatures 3 - blst-native 4.4672 ms/op 3.8050 ms/op 1.17
BLS verifyMultipleSignatures 8 - blst-native 9.6660 ms/op 8.1838 ms/op 1.18
BLS verifyMultipleSignatures 32 - blst-native 35.167 ms/op 29.681 ms/op 1.18
BLS aggregatePubkeys 32 - blst-native 46.701 us/op 39.199 us/op 1.19
BLS aggregatePubkeys 128 - blst-native 182.73 us/op 152.62 us/op 1.20
getAttestationsForBlock 43.268 ms/op 51.804 ms/op 0.84
isKnown best case - 1 super set check 482.00 ns/op 427.00 ns/op 1.13
isKnown normal case - 2 super set checks 467.00 ns/op 414.00 ns/op 1.13
isKnown worse case - 16 super set checks 470.00 ns/op 411.00 ns/op 1.14
CheckpointStateCache - add get delete 8.8030 us/op 9.3870 us/op 0.94
validate gossip signedAggregateAndProof - struct 5.0100 ms/op 4.3724 ms/op 1.15
validate gossip attestation - struct 2.3663 ms/op 2.0271 ms/op 1.17
altair verifyImport mainnet_s3766816:31 12.584 s/op 12.741 s/op 0.99
pickEth1Vote - no votes 2.0795 ms/op 2.1312 ms/op 0.98
pickEth1Vote - max votes 20.274 ms/op 24.597 ms/op 0.82
pickEth1Vote - Eth1Data hashTreeRoot value x2048 12.581 ms/op 11.591 ms/op 1.09
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 20.620 ms/op 21.908 ms/op 0.94
pickEth1Vote - Eth1Data fastSerialize value x2048 1.4946 ms/op 1.6431 ms/op 0.91
pickEth1Vote - Eth1Data fastSerialize tree x2048 15.377 ms/op 19.185 ms/op 0.80
bytes32 toHexString 1.0950 us/op 1.2150 us/op 0.90
bytes32 Buffer.toString(hex) 789.00 ns/op 766.00 ns/op 1.03
bytes32 Buffer.toString(hex) from Uint8Array 1.0370 us/op 963.00 ns/op 1.08
bytes32 Buffer.toString(hex) + 0x 803.00 ns/op 754.00 ns/op 1.06
Object access 1 prop 0.36300 ns/op 0.41800 ns/op 0.87
Map access 1 prop 0.35100 ns/op 0.29200 ns/op 1.20
Object get x1000 10.840 ns/op 17.875 ns/op 0.61
Map get x1000 1.1930 ns/op 1.0510 ns/op 1.14
Object set x1000 74.003 ns/op 135.58 ns/op 0.55
Map set x1000 50.197 ns/op 77.840 ns/op 0.64
Return object 10000 times 0.44820 ns/op 0.37740 ns/op 1.19
Throw Error 10000 times 6.0230 us/op 6.0253 us/op 1.00
enrSubnets - fastDeserialize 64 bits 2.7850 us/op 3.0640 us/op 0.91
enrSubnets - ssz BitVector 64 bits 866.00 ns/op 808.00 ns/op 1.07
enrSubnets - fastDeserialize 4 bits 488.00 ns/op 462.00 ns/op 1.06
enrSubnets - ssz BitVector 4 bits 838.00 ns/op 832.00 ns/op 1.01
prioritizePeers score -10:0 att 32-0.1 sync 2-0 85.866 us/op 108.39 us/op 0.79
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 128.91 us/op 125.30 us/op 1.03
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 218.40 us/op 245.63 us/op 0.89
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 335.56 us/op 511.25 us/op 0.66
prioritizePeers score 0:0 att 64-1 sync 4-1 406.19 us/op 464.58 us/op 0.87
RateTracker 1000000 limit, 1 obj count per request 201.81 ns/op 207.99 ns/op 0.97
RateTracker 1000000 limit, 2 obj count per request 144.35 ns/op 159.57 ns/op 0.90
RateTracker 1000000 limit, 4 obj count per request 117.38 ns/op 136.15 ns/op 0.86
RateTracker 1000000 limit, 8 obj count per request 101.10 ns/op 121.80 ns/op 0.83
RateTracker with prune 4.1120 us/op 5.4770 us/op 0.75
array of 16000 items push then shift 51.590 us/op 3.2046 us/op 16.10
LinkedList of 16000 items push then shift 12.481 ns/op 30.141 ns/op 0.41
array of 16000 items push then pop 220.01 ns/op 258.47 ns/op 0.85
LinkedList of 16000 items push then pop 12.012 ns/op 23.620 ns/op 0.51
array of 24000 items push then shift 77.347 us/op 4.5582 us/op 16.97
LinkedList of 24000 items push then shift 13.105 ns/op 32.164 ns/op 0.41
array of 24000 items push then pop 190.93 ns/op 219.43 ns/op 0.87
LinkedList of 24000 items push then pop 12.080 ns/op 23.820 ns/op 0.51
intersect bitArray bitLen 8 10.766 ns/op 11.817 ns/op 0.91
intersect array and set length 8 137.76 ns/op 199.48 ns/op 0.69
intersect bitArray bitLen 128 58.120 ns/op 62.031 ns/op 0.94
intersect array and set length 128 1.8152 us/op 2.5561 us/op 0.71
pass gossip attestations to forkchoice per slot 2.8402 ms/op 6.6769 ms/op 0.43
computeDeltas 3.2325 ms/op 3.8081 ms/op 0.85
computeProposerBoostScoreFromBalances 807.87 us/op 907.71 us/op 0.89
altair processAttestation - 250000 vs - 7PWei normalcase 3.6627 ms/op 4.9866 ms/op 0.73
altair processAttestation - 250000 vs - 7PWei worstcase 5.3790 ms/op 6.8111 ms/op 0.79
altair processAttestation - setStatus - 1/6 committees join 172.93 us/op 584.04 us/op 0.30
altair processAttestation - setStatus - 1/3 committees join 340.22 us/op 414.25 us/op 0.82
altair processAttestation - setStatus - 1/2 committees join 482.59 us/op 578.02 us/op 0.83
altair processAttestation - setStatus - 2/3 committees join 630.41 us/op 738.06 us/op 0.85
altair processAttestation - setStatus - 4/5 committees join 885.91 us/op 1.0207 ms/op 0.87
altair processAttestation - setStatus - 100% committees join 1.0621 ms/op 1.2171 ms/op 0.87
altair processBlock - 250000 vs - 7PWei normalcase 24.761 ms/op 29.569 ms/op 0.84
altair processBlock - 250000 vs - 7PWei normalcase hashState 40.097 ms/op 45.670 ms/op 0.88
altair processBlock - 250000 vs - 7PWei worstcase 90.754 ms/op 84.609 ms/op 1.07
altair processBlock - 250000 vs - 7PWei worstcase hashState 101.74 ms/op 117.35 ms/op 0.87
phase0 processBlock - 250000 vs - 7PWei normalcase 4.0676 ms/op 4.5695 ms/op 0.89
phase0 processBlock - 250000 vs - 7PWei worstcase 52.814 ms/op 48.594 ms/op 1.09
altair processEth1Data - 250000 vs - 7PWei normalcase 763.20 us/op 939.20 us/op 0.81
Tree 40 250000 create 692.51 ms/op 996.10 ms/op 0.70
Tree 40 250000 get(125000) 231.11 ns/op 286.90 ns/op 0.81
Tree 40 250000 set(125000) 2.1499 us/op 2.9963 us/op 0.72
Tree 40 250000 toArray() 27.530 ms/op 33.923 ms/op 0.81
Tree 40 250000 iterate all - toArray() + loop 27.668 ms/op 34.884 ms/op 0.79
Tree 40 250000 iterate all - get(i) 111.59 ms/op 113.19 ms/op 0.99
MutableVector 250000 create 14.238 ms/op 18.058 ms/op 0.79
MutableVector 250000 get(125000) 10.625 ns/op 14.763 ns/op 0.72
MutableVector 250000 set(125000) 534.88 ns/op 779.30 ns/op 0.69
MutableVector 250000 toArray() 5.8253 ms/op 8.0287 ms/op 0.73
MutableVector 250000 iterate all - toArray() + loop 6.0649 ms/op 8.3950 ms/op 0.72
MutableVector 250000 iterate all - get(i) 2.8213 ms/op 3.4390 ms/op 0.82
Array 250000 create 5.7748 ms/op 7.7516 ms/op 0.74
Array 250000 clone - spread 3.4080 ms/op 4.1488 ms/op 0.82
Array 250000 get(125000) 1.5650 ns/op 1.8760 ns/op 0.83
Array 250000 set(125000) 1.5790 ns/op 1.7840 ns/op 0.89
Array 250000 iterate all - loop 151.16 us/op 167.93 us/op 0.90
effectiveBalanceIncrements clone Uint8Array 300000 73.771 us/op 116.31 us/op 0.63
effectiveBalanceIncrements clone MutableVector 300000 633.00 ns/op 1.4210 us/op 0.45
effectiveBalanceIncrements rw all Uint8Array 300000 247.40 us/op 252.54 us/op 0.98
effectiveBalanceIncrements rw all MutableVector 300000 129.05 ms/op 268.09 ms/op 0.48
phase0 afterProcessEpoch - 250000 vs - 7PWei 185.61 ms/op 184.32 ms/op 1.01
phase0 beforeProcessEpoch - 250000 vs - 7PWei 57.350 ms/op 106.30 ms/op 0.54
altair processEpoch - mainnet_e81889 545.64 ms/op 608.76 ms/op 0.90
mainnet_e81889 - altair beforeProcessEpoch 119.60 ms/op 167.74 ms/op 0.71
mainnet_e81889 - altair processJustificationAndFinalization 16.367 us/op 22.460 us/op 0.73
mainnet_e81889 - altair processInactivityUpdates 8.8273 ms/op 11.642 ms/op 0.76
mainnet_e81889 - altair processRewardsAndPenalties 81.184 ms/op 98.461 ms/op 0.82
mainnet_e81889 - altair processRegistryUpdates 3.0260 us/op 3.9400 us/op 0.77
mainnet_e81889 - altair processSlashings 618.00 ns/op 714.00 ns/op 0.87
mainnet_e81889 - altair processEth1DataReset 621.00 ns/op 850.00 ns/op 0.73
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.1758 ms/op 2.2725 ms/op 0.96
mainnet_e81889 - altair processSlashingsReset 5.0620 us/op 6.0230 us/op 0.84
mainnet_e81889 - altair processRandaoMixesReset 4.4390 us/op 5.7690 us/op 0.77
mainnet_e81889 - altair processHistoricalRootsUpdate 651.00 ns/op 922.00 ns/op 0.71
mainnet_e81889 - altair processParticipationFlagUpdates 2.3810 us/op 3.7640 us/op 0.63
mainnet_e81889 - altair processSyncCommitteeUpdates 599.00 ns/op 700.00 ns/op 0.86
mainnet_e81889 - altair afterProcessEpoch 196.11 ms/op 255.88 ms/op 0.77
phase0 processEpoch - mainnet_e58758 482.58 ms/op 606.37 ms/op 0.80
mainnet_e58758 - phase0 beforeProcessEpoch 195.43 ms/op 260.77 ms/op 0.75
mainnet_e58758 - phase0 processJustificationAndFinalization 15.595 us/op 18.745 us/op 0.83
mainnet_e58758 - phase0 processRewardsAndPenalties 68.929 ms/op 83.638 ms/op 0.82
mainnet_e58758 - phase0 processRegistryUpdates 7.9790 us/op 10.789 us/op 0.74
mainnet_e58758 - phase0 processSlashings 615.00 ns/op 1.0250 us/op 0.60
mainnet_e58758 - phase0 processEth1DataReset 609.00 ns/op 897.00 ns/op 0.68
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.6123 ms/op 1.8598 ms/op 0.87
mainnet_e58758 - phase0 processSlashingsReset 2.9510 us/op 3.7010 us/op 0.80
mainnet_e58758 - phase0 processRandaoMixesReset 4.8040 us/op 6.1100 us/op 0.79
mainnet_e58758 - phase0 processHistoricalRootsUpdate 694.00 ns/op 940.00 ns/op 0.74
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.0720 us/op 5.0710 us/op 0.80
mainnet_e58758 - phase0 afterProcessEpoch 163.05 ms/op 160.54 ms/op 1.02
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.9826 ms/op 2.5499 ms/op 0.78
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.2311 ms/op 2.9617 ms/op 0.75
altair processInactivityUpdates - 250000 normalcase 32.365 ms/op 43.606 ms/op 0.74
altair processInactivityUpdates - 250000 worstcase 32.794 ms/op 43.093 ms/op 0.76
phase0 processRegistryUpdates - 250000 normalcase 6.5730 us/op 9.9900 us/op 0.66
phase0 processRegistryUpdates - 250000 badcase_full_deposits 382.16 us/op 421.59 us/op 0.91
phase0 processRegistryUpdates - 250000 worstcase 0.5 174.61 ms/op 234.64 ms/op 0.74
altair processRewardsAndPenalties - 250000 normalcase 78.513 ms/op 121.23 ms/op 0.65
altair processRewardsAndPenalties - 250000 worstcase 106.78 ms/op 150.58 ms/op 0.71
phase0 getAttestationDeltas - 250000 normalcase 11.874 ms/op 12.773 ms/op 0.93
phase0 getAttestationDeltas - 250000 worstcase 11.802 ms/op 12.723 ms/op 0.93
phase0 processSlashings - 250000 worstcase 5.0235 ms/op 5.4393 ms/op 0.92
altair processSyncCommitteeUpdates - 250000 293.59 ms/op 282.97 ms/op 1.04
BeaconState.hashTreeRoot - No change 533.00 ns/op 477.00 ns/op 1.12
BeaconState.hashTreeRoot - 1 full validator 74.414 us/op 63.254 us/op 1.18
BeaconState.hashTreeRoot - 32 full validator 719.71 us/op 640.24 us/op 1.12
BeaconState.hashTreeRoot - 512 full validator 7.0431 ms/op 5.9574 ms/op 1.18
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 89.504 us/op 81.114 us/op 1.10
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.3145 ms/op 1.2084 ms/op 1.09
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 17.010 ms/op 16.339 ms/op 1.04
BeaconState.hashTreeRoot - 1 balances 73.134 us/op 63.321 us/op 1.15
BeaconState.hashTreeRoot - 32 balances 646.81 us/op 758.58 us/op 0.85
BeaconState.hashTreeRoot - 512 balances 7.2550 ms/op 5.6295 ms/op 1.29
BeaconState.hashTreeRoot - 250000 balances 101.84 ms/op 86.953 ms/op 1.17
aggregationBits - 2048 els - zipIndexesInBitList 26.822 us/op 35.100 us/op 0.76
regular array get 100000 times 61.731 us/op 67.426 us/op 0.92
wrappedArray get 100000 times 61.944 us/op 67.449 us/op 0.92
arrayWithProxy get 100000 times 30.394 ms/op 29.256 ms/op 1.04
ssz.Root.equals 476.00 ns/op 535.00 ns/op 0.89
byteArrayEquals 477.00 ns/op 523.00 ns/op 0.91
shuffle list - 16384 els 11.415 ms/op 11.059 ms/op 1.03
shuffle list - 250000 els 166.06 ms/op 163.29 ms/op 1.02
processSlot - 1 slots 13.346 us/op 13.579 us/op 0.98
processSlot - 32 slots 1.9851 ms/op 1.8778 ms/op 1.06
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 411.54 us/op 374.64 us/op 1.10
getCommitteeAssignments - req 1 vs - 250000 vc 5.4720 ms/op 5.3415 ms/op 1.02
getCommitteeAssignments - req 100 vs - 250000 vc 7.9798 ms/op 7.3346 ms/op 1.09
getCommitteeAssignments - req 1000 vs - 250000 vc 8.5937 ms/op 7.8077 ms/op 1.10
computeProposers - vc 250000 17.382 ms/op 19.029 ms/op 0.91
computeEpochShuffling - vc 250000 185.55 ms/op 166.26 ms/op 1.12
getNextSyncCommittee - vc 250000 290.00 ms/op 275.04 ms/op 1.05

by benchmarkbot/action

@wemeetagain
Copy link
Member

Test failing https://github.com/ChainSafe/lodestar/runs/7390042459?check_suite_focus=true#step:18:2566

@wemeetagain wemeetagain merged commit fdd4c6b into unstable Jul 18, 2022
@wemeetagain wemeetagain deleted the dapplion/dev-cmd-transparent branch July 18, 2022 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nazarhussain nazarhussain nazarhussain approved these changes

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@dapplion @wemeetagain @nazarhussain