Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: reusable thread pool to decrypt keystores #5623

Merged
merged 1 commit into from
Jun 9, 2023
Merged

refactor: reusable thread pool to decrypt keystores #5623

merged 1 commit into from
Jun 9, 2023

Conversation

nflaig
Copy link
Member

@nflaig nflaig commented Jun 8, 2023

Motivation

Decrypting keystores is required in multiple places

  • when importing from keystore definition files (#5357)
  • when importing keystores through the keymanager API (#5438)

This PR makes the thread pool implementation reusable, improves pool lifecycle management and error handling.

Description

Reusable thread pool to decrypt keystores, addresses the following issues

  • terminates workers if process receives exit signal
  • ensures consistent configuration of thread pool
  • enables worker to decrypt from file or keystore string
  • more graceful and deterministic error handling
  • improved error handling and formatting 👇

Current error handling

Current error thrown if decryption of keystores fails is not that helpful if there are multiple. There is no indication on which keystore file is affected and only reports one error.

✖ Error: Invalid password
    at Object.<anonymous> (/home/devops/goerli/lodestar/node_modules/@chainsafe/bls-keystore/lib/functional.js:86:19)
    at Generator.next (<anonymous>)
    at fulfilled (/home/devops/goerli/lodestar/node_modules/@chainsafe/bls-keystore/lib/functional.js:5:58)

Updated error handling

Updated error handling provides more details on which keystore import failed and why it failed.

Single keystore import error

 ✖ Error importing keystore

keystore-m_12381_3600_2_0_0-1673696854.json: Invalid password

Multiple keystore import errors

 ✖ Multiple errors importing keystores

keystore-m_12381_3600_2_0_0-1673696854.json: Invalid password
keystore-m_12381_3600_8_0_0-1673696857.json: Invalid password
keystore-m_12381_3600_5_0_0-1673696855.json: Invalid password

Single error, multiple aborted due to error

 ✖ Error importing keystore

keystore-malformed.json: Unexpected token s in JSON at position 0

Aborted 100 pending keystore imports

Multiple errors, multiple aborted due to errors

 ✖ Multiple errors importing keystores

keystore-m_12381_3600_1028_0_0-1680958115.json: Invalid password
keystore-m_12381_3600_1031_0_0-1680958117.json: Invalid password
keystore-m_12381_3600_1032_0_0-1680958118.json: Invalid password
keystore-m_12381_3600_1045_0_0-1680958125.json: Invalid password
keystore-m_12381_3600_1049_0_0-1680958127.json: Invalid password
keystore-m_12381_3600_1059_0_0-1680958133.json: Invalid password
keystore-m_12381_3600_1018_0_0-1680958110.json: Invalid password
keystore-m_12381_3600_1034_0_0-1680958119.json: Invalid password
keystore-m_12381_3600_1087_0_0-1680958149.json: Invalid password

Aborted 91 pending keystore imports

nflaig
nflaig commented Jun 8, 2023
View reviewed changes
packages/cli/src/cmds/validator/handler.ts
// Since it is perfectly fine to have listeners > 10
setMaxListeners(Infinity, abortController.signal);

onGracefulShutdownCbs.push(async () => abortController.abort());
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Abort has to be pushed to graceful shutdown callbacks before invoking getSignersFromArgs.

packages/cli/src/cmds/validator/keymanager/decryptKeystores/threadPool.ts
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting DEBUG=threads:pool:* is pretty helpful to analyse thread and task lifecycle

packages/cli/src/cmds/validator/keymanager/decryptKeystores/threadPool.ts
{
// Adjust worker pool size based on keystore count
size: Math.min(keystoreCount, maxPoolSize),
// Decrypt keystores in sequence, increasing concurrency does not improve performance
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did several test runs with different concurrency but it seems to just overload the threads and throttle the CPU

packages/cli/src/cmds/validator/signers/importExternalKeystores.ts
@@ -37,9 +37,6 @@ export async function readPassphraseOrPrompt(args: {importKeystoresPassword?: st
},
]);

// eslint-disable-next-line no-console
console.log("Password is correct");
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This console.log is incorrect and not helpful as password verification is done later in the process

@github-actions
Copy link
Contributor

github-actions bot commented Jun 8, 2023 •
edited

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: c8833ef Previous: 864c70f Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 790.65 us/op 1.1406 ms/op 0.69
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 46.814 us/op 63.289 us/op 0.74
BLS verify - blst-native 1.2461 ms/op 1.3376 ms/op 0.93
BLS verifyMultipleSignatures 3 - blst-native 2.5251 ms/op 2.8080 ms/op 0.90
BLS verifyMultipleSignatures 8 - blst-native 5.4526 ms/op 5.5852 ms/op 0.98
BLS verifyMultipleSignatures 32 - blst-native 19.856 ms/op 20.912 ms/op 0.95
BLS aggregatePubkeys 32 - blst-native 26.402 us/op 27.763 us/op 0.95
BLS aggregatePubkeys 128 - blst-native 102.56 us/op 119.21 us/op 0.86
getAttestationsForBlock 52.075 ms/op 75.028 ms/op 0.69
isKnown best case - 1 super set check 260.00 ns/op 301.00 ns/op 0.86
isKnown normal case - 2 super set checks 255.00 ns/op 291.00 ns/op 0.88
isKnown worse case - 16 super set checks 257.00 ns/op 298.00 ns/op 0.86
CheckpointStateCache - add get delete 5.1160 us/op 6.2000 us/op 0.83
validate gossip signedAggregateAndProof - struct 2.7196 ms/op 2.9179 ms/op 0.93
validate gossip attestation - struct 1.2762 ms/op 1.4486 ms/op 0.88
pickEth1Vote - no votes 1.2088 ms/op 1.4613 ms/op 0.83
pickEth1Vote - max votes 11.060 ms/op 13.557 ms/op 0.82
pickEth1Vote - Eth1Data hashTreeRoot value x2048 8.5706 ms/op 10.639 ms/op 0.81
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 13.236 ms/op 18.214 ms/op 0.73
pickEth1Vote - Eth1Data fastSerialize value x2048 659.27 us/op 825.52 us/op 0.80
pickEth1Vote - Eth1Data fastSerialize tree x2048 7.7287 ms/op 6.5532 ms/op 1.18
bytes32 toHexString 483.00 ns/op 772.00 ns/op 0.63
bytes32 Buffer.toString(hex) 353.00 ns/op 460.00 ns/op 0.77
bytes32 Buffer.toString(hex) from Uint8Array 535.00 ns/op 684.00 ns/op 0.78
bytes32 Buffer.toString(hex) + 0x 338.00 ns/op 436.00 ns/op 0.78
Object access 1 prop 0.16600 ns/op 0.21800 ns/op 0.76
Map access 1 prop 0.16600 ns/op 0.19300 ns/op 0.86
Object get x1000 6.8950 ns/op 7.5780 ns/op 0.91
Map get x1000 0.60400 ns/op 0.61800 ns/op 0.98
Object set x1000 51.173 ns/op 76.897 ns/op 0.67
Map set x1000 44.735 ns/op 61.111 ns/op 0.73
Return object 10000 times 0.23020 ns/op 0.25300 ns/op 0.91
Throw Error 10000 times 4.0788 us/op 4.4220 us/op 0.92
fastMsgIdFn sha256 / 200 bytes 3.3600 us/op 3.5650 us/op 0.94
fastMsgIdFn h32 xxhash / 200 bytes 272.00 ns/op 328.00 ns/op 0.83
fastMsgIdFn h64 xxhash / 200 bytes 391.00 ns/op 497.00 ns/op 0.79
fastMsgIdFn sha256 / 1000 bytes 11.355 us/op 12.140 us/op 0.94
fastMsgIdFn h32 xxhash / 1000 bytes 415.00 ns/op 469.00 ns/op 0.88
fastMsgIdFn h64 xxhash / 1000 bytes 468.00 ns/op 572.00 ns/op 0.82
fastMsgIdFn sha256 / 10000 bytes 101.14 us/op 109.30 us/op 0.93
fastMsgIdFn h32 xxhash / 10000 bytes 1.8880 us/op 2.0380 us/op 0.93
fastMsgIdFn h64 xxhash / 10000 bytes 1.3480 us/op 1.5010 us/op 0.90
enrSubnets - fastDeserialize 64 bits 1.2430 us/op 1.6920 us/op 0.73
enrSubnets - ssz BitVector 64 bits 477.00 ns/op 639.00 ns/op 0.75
enrSubnets - fastDeserialize 4 bits 166.00 ns/op 214.00 ns/op 0.78
enrSubnets - ssz BitVector 4 bits 469.00 ns/op 651.00 ns/op 0.72
prioritizePeers score -10:0 att 32-0.1 sync 2-0 103.53 us/op 116.49 us/op 0.89
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 133.42 us/op 172.04 us/op 0.78
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 167.53 us/op 209.20 us/op 0.80
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 300.35 us/op 390.22 us/op 0.77
prioritizePeers score 0:0 att 64-1 sync 4-1 361.82 us/op 499.52 us/op 0.72
array of 16000 items push then shift 1.5924 us/op 1.8216 us/op 0.87
LinkedList of 16000 items push then shift 8.8160 ns/op 11.262 ns/op 0.78
array of 16000 items push then pop 78.822 ns/op 141.81 ns/op 0.56
LinkedList of 16000 items push then pop 8.6610 ns/op 10.417 ns/op 0.83
array of 24000 items push then shift 2.3118 us/op 2.5589 us/op 0.90
LinkedList of 24000 items push then shift 8.5920 ns/op 11.882 ns/op 0.72
array of 24000 items push then pop 75.813 ns/op 101.15 ns/op 0.75
LinkedList of 24000 items push then pop 8.3390 ns/op 9.5450 ns/op 0.87
intersect bitArray bitLen 8 13.078 ns/op 14.092 ns/op 0.93
intersect array and set length 8 76.377 ns/op 107.88 ns/op 0.71
intersect bitArray bitLen 128 43.114 ns/op 46.042 ns/op 0.94
intersect array and set length 128 1.0258 us/op 1.3761 us/op 0.75
Buffer.concat 32 items 2.8420 us/op 3.4360 us/op 0.83
Uint8Array.set 32 items 2.7730 us/op 2.5730 us/op 1.08
transfer serialized Status (84 B) 2.0680 us/op 2.3670 us/op 0.87
copy serialized Status (84 B) 1.6630 us/op 2.1040 us/op 0.79
transfer serialized SignedVoluntaryExit (112 B) 2.2310 us/op 2.4110 us/op 0.93
copy serialized SignedVoluntaryExit (112 B) 1.7670 us/op 2.1710 us/op 0.81
transfer serialized ProposerSlashing (416 B) 2.8680 us/op 3.3190 us/op 0.86
copy serialized ProposerSlashing (416 B) 2.6320 us/op 3.5490 us/op 0.74
transfer serialized Attestation (485 B) 2.6090 us/op 2.7950 us/op 0.93
copy serialized Attestation (485 B) 2.3990 us/op 2.5970 us/op 0.92
transfer serialized AttesterSlashing (33232 B) 2.7580 us/op 2.8090 us/op 0.98
copy serialized AttesterSlashing (33232 B) 5.7050 us/op 8.5790 us/op 0.66
transfer serialized Small SignedBeaconBlock (128000 B) 3.2720 us/op 3.4900 us/op 0.94
copy serialized Small SignedBeaconBlock (128000 B) 13.315 us/op 20.125 us/op 0.66
transfer serialized Avg SignedBeaconBlock (200000 B) 3.5470 us/op 3.8580 us/op 0.92
copy serialized Avg SignedBeaconBlock (200000 B) 18.894 us/op 74.534 us/op 0.25
transfer serialized BlobsSidecar (524380 B) 3.4370 us/op 4.2290 us/op 0.81
copy serialized BlobsSidecar (524380 B) 176.53 us/op 205.37 us/op 0.86
transfer serialized Big SignedBeaconBlock (1000000 B) 3.2640 us/op 4.0930 us/op 0.80
copy serialized Big SignedBeaconBlock (1000000 B) 282.36 us/op 353.19 us/op 0.80
pass gossip attestations to forkchoice per slot 2.5810 ms/op 2.8909 ms/op 0.89
forkChoice updateHead vc 100000 bc 64 eq 0 2.0244 ms/op 2.2848 ms/op 0.89
forkChoice updateHead vc 600000 bc 64 eq 0 11.136 ms/op 12.909 ms/op 0.86
forkChoice updateHead vc 1000000 bc 64 eq 0 21.171 ms/op 25.472 ms/op 0.83
forkChoice updateHead vc 600000 bc 320 eq 0 16.020 ms/op 17.995 ms/op 0.89
forkChoice updateHead vc 600000 bc 1200 eq 0 77.568 ms/op 91.599 ms/op 0.85
forkChoice updateHead vc 600000 bc 64 eq 1000 20.367 ms/op 22.564 ms/op 0.90
forkChoice updateHead vc 600000 bc 64 eq 10000 22.000 ms/op 24.454 ms/op 0.90
forkChoice updateHead vc 600000 bc 64 eq 300000 30.187 ms/op 71.056 ms/op 0.42
computeDeltas 3.8278 ms/op 3.4712 ms/op 1.10
computeProposerBoostScoreFromBalances 1.7611 ms/op 1.9166 ms/op 0.92
altair processAttestation - 250000 vs - 7PWei normalcase 2.1369 ms/op 2.7568 ms/op 0.78
altair processAttestation - 250000 vs - 7PWei worstcase 3.8217 ms/op 4.4284 ms/op 0.86
altair processAttestation - setStatus - 1/6 committees join 139.50 us/op 158.43 us/op 0.88
altair processAttestation - setStatus - 1/3 committees join 274.54 us/op 283.39 us/op 0.97
altair processAttestation - setStatus - 1/2 committees join 362.93 us/op 390.52 us/op 0.93
altair processAttestation - setStatus - 2/3 committees join 451.09 us/op 472.03 us/op 0.96
altair processAttestation - setStatus - 4/5 committees join 631.99 us/op 687.75 us/op 0.92
altair processAttestation - setStatus - 100% committees join 756.68 us/op 784.50 us/op 0.96
altair processBlock - 250000 vs - 7PWei normalcase 17.435 ms/op 19.116 ms/op 0.91
altair processBlock - 250000 vs - 7PWei normalcase hashState 26.582 ms/op 29.143 ms/op 0.91
altair processBlock - 250000 vs - 7PWei worstcase 55.026 ms/op 50.594 ms/op 1.09
altair processBlock - 250000 vs - 7PWei worstcase hashState 68.529 ms/op 70.301 ms/op 0.97
phase0 processBlock - 250000 vs - 7PWei normalcase 1.9893 ms/op 2.3736 ms/op 0.84
phase0 processBlock - 250000 vs - 7PWei worstcase 27.804 ms/op 30.909 ms/op 0.90
altair processEth1Data - 250000 vs - 7PWei normalcase 471.28 us/op 494.55 us/op 0.95
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 8.4320 us/op 12.065 us/op 0.70
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 23.877 us/op 26.881 us/op 0.89
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 12.965 us/op 11.700 us/op 1.11
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 8.2460 us/op 8.1420 us/op 1.01
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 95.385 us/op 106.19 us/op 0.90
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 649.35 us/op 678.75 us/op 0.96
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 929.14 us/op 945.13 us/op 0.98
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 880.92 us/op 946.20 us/op 0.93
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.3487 ms/op 2.8034 ms/op 0.84
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.5500 ms/op 1.7659 ms/op 0.88
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.9457 ms/op 4.3176 ms/op 0.91
Tree 40 250000 create 322.78 ms/op 345.32 ms/op 0.93
Tree 40 250000 get(125000) 197.71 ns/op 191.74 ns/op 1.03
Tree 40 250000 set(125000) 937.64 ns/op 1.0192 us/op 0.92
Tree 40 250000 toArray() 21.473 ms/op 24.163 ms/op 0.89
Tree 40 250000 iterate all - toArray() + loop 22.564 ms/op 23.801 ms/op 0.95
Tree 40 250000 iterate all - get(i) 76.533 ms/op 76.546 ms/op 1.00
MutableVector 250000 create 11.744 ms/op 11.673 ms/op 1.01
MutableVector 250000 get(125000) 6.6280 ns/op 6.4800 ns/op 1.02
MutableVector 250000 set(125000) 266.84 ns/op 270.36 ns/op 0.99
MutableVector 250000 toArray() 3.5340 ms/op 3.0486 ms/op 1.16
MutableVector 250000 iterate all - toArray() + loop 3.5553 ms/op 3.1821 ms/op 1.12
MutableVector 250000 iterate all - get(i) 1.5153 ms/op 1.5389 ms/op 0.98
Array 250000 create 2.9501 ms/op 3.0824 ms/op 0.96
Array 250000 clone - spread 1.1768 ms/op 1.1443 ms/op 1.03
Array 250000 get(125000) 0.59200 ns/op 0.59400 ns/op 1.00
Array 250000 set(125000) 0.66700 ns/op 0.67400 ns/op 0.99
Array 250000 iterate all - loop 99.985 us/op 83.515 us/op 1.20
effectiveBalanceIncrements clone Uint8Array 300000 28.459 us/op 39.676 us/op 0.72
effectiveBalanceIncrements clone MutableVector 300000 366.00 ns/op 370.00 ns/op 0.99
effectiveBalanceIncrements rw all Uint8Array 300000 166.35 us/op 169.51 us/op 0.98
effectiveBalanceIncrements rw all MutableVector 300000 82.397 ms/op 89.952 ms/op 0.92
phase0 afterProcessEpoch - 250000 vs - 7PWei 114.09 ms/op 116.85 ms/op 0.98
phase0 beforeProcessEpoch - 250000 vs - 7PWei 33.300 ms/op 51.767 ms/op 0.64
altair processEpoch - mainnet_e81889 326.05 ms/op 365.72 ms/op 0.89
mainnet_e81889 - altair beforeProcessEpoch 69.474 ms/op 71.260 ms/op 0.97
mainnet_e81889 - altair processJustificationAndFinalization 17.101 us/op 18.379 us/op 0.93
mainnet_e81889 - altair processInactivityUpdates 5.2892 ms/op 7.9689 ms/op 0.66
mainnet_e81889 - altair processRewardsAndPenalties 51.550 ms/op 51.909 ms/op 0.99
mainnet_e81889 - altair processRegistryUpdates 2.4810 us/op 3.9220 us/op 0.63
mainnet_e81889 - altair processSlashings 452.00 ns/op 534.00 ns/op 0.85
mainnet_e81889 - altair processEth1DataReset 528.00 ns/op 661.00 ns/op 0.80
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2295 ms/op 1.2945 ms/op 0.95
mainnet_e81889 - altair processSlashingsReset 5.1890 us/op 5.4990 us/op 0.94
mainnet_e81889 - altair processRandaoMixesReset 4.9520 us/op 8.6840 us/op 0.57
mainnet_e81889 - altair processHistoricalRootsUpdate 1.1240 us/op 1.4370 us/op 0.78
mainnet_e81889 - altair processParticipationFlagUpdates 3.7020 us/op 4.7010 us/op 0.79
mainnet_e81889 - altair processSyncCommitteeUpdates 1.1780 us/op 889.00 ns/op 1.33
mainnet_e81889 - altair afterProcessEpoch 129.36 ms/op 140.19 ms/op 0.92
phase0 processEpoch - mainnet_e58758 348.11 ms/op 405.05 ms/op 0.86
mainnet_e58758 - phase0 beforeProcessEpoch 136.48 ms/op 165.73 ms/op 0.82
mainnet_e58758 - phase0 processJustificationAndFinalization 18.275 us/op 21.628 us/op 0.84
mainnet_e58758 - phase0 processRewardsAndPenalties 63.613 ms/op 69.358 ms/op 0.92
mainnet_e58758 - phase0 processRegistryUpdates 9.8700 us/op 13.811 us/op 0.71
mainnet_e58758 - phase0 processSlashings 513.00 ns/op 1.0630 us/op 0.48
mainnet_e58758 - phase0 processEth1DataReset 523.00 ns/op 1.5030 us/op 0.35
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.1558 ms/op 1.2477 ms/op 0.93
mainnet_e58758 - phase0 processSlashingsReset 4.4820 us/op 6.8320 us/op 0.66
mainnet_e58758 - phase0 processRandaoMixesReset 4.6870 us/op 10.442 us/op 0.45
mainnet_e58758 - phase0 processHistoricalRootsUpdate 830.00 ns/op 1.3640 us/op 0.61
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.7970 us/op 6.7860 us/op 0.71
mainnet_e58758 - phase0 afterProcessEpoch 101.06 ms/op 107.65 ms/op 0.94
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2877 ms/op 1.6312 ms/op 0.79
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.5422 ms/op 1.8514 ms/op 0.83
altair processInactivityUpdates - 250000 normalcase 23.919 ms/op 28.092 ms/op 0.85
altair processInactivityUpdates - 250000 worstcase 22.465 ms/op 30.471 ms/op 0.74
phase0 processRegistryUpdates - 250000 normalcase 7.2800 us/op 14.283 us/op 0.51
phase0 processRegistryUpdates - 250000 badcase_full_deposits 267.83 us/op 350.51 us/op 0.76
phase0 processRegistryUpdates - 250000 worstcase 0.5 127.66 ms/op 170.68 ms/op 0.75
altair processRewardsAndPenalties - 250000 normalcase 66.697 ms/op 76.595 ms/op 0.87
altair processRewardsAndPenalties - 250000 worstcase 69.159 ms/op 76.988 ms/op 0.90
phase0 getAttestationDeltas - 250000 normalcase 6.8702 ms/op 10.542 ms/op 0.65
phase0 getAttestationDeltas - 250000 worstcase 6.8340 ms/op 9.5509 ms/op 0.72
phase0 processSlashings - 250000 worstcase 3.6483 ms/op 4.4354 ms/op 0.82
altair processSyncCommitteeUpdates - 250000 179.48 ms/op 232.75 ms/op 0.77
BeaconState.hashTreeRoot - No change 271.00 ns/op 368.00 ns/op 0.74
BeaconState.hashTreeRoot - 1 full validator 53.346 us/op 60.909 us/op 0.88
BeaconState.hashTreeRoot - 32 full validator 539.72 us/op 671.08 us/op 0.80
BeaconState.hashTreeRoot - 512 full validator 5.8430 ms/op 6.8291 ms/op 0.86
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 63.105 us/op 77.653 us/op 0.81
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 933.67 us/op 1.1255 ms/op 0.83
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 12.430 ms/op 14.878 ms/op 0.84
BeaconState.hashTreeRoot - 1 balances 49.782 us/op 57.281 us/op 0.87
BeaconState.hashTreeRoot - 32 balances 449.45 us/op 550.21 us/op 0.82
BeaconState.hashTreeRoot - 512 balances 4.3956 ms/op 5.8863 ms/op 0.75
BeaconState.hashTreeRoot - 250000 balances 74.688 ms/op 81.478 ms/op 0.92
aggregationBits - 2048 els - zipIndexesInBitList 16.378 us/op 24.093 us/op 0.68
regular array get 100000 times 43.255 us/op 43.966 us/op 0.98
wrappedArray get 100000 times 44.030 us/op 36.972 us/op 1.19
arrayWithProxy get 100000 times 15.729 ms/op 17.867 ms/op 0.88
ssz.Root.equals 548.00 ns/op 670.00 ns/op 0.82
byteArrayEquals 549.00 ns/op 630.00 ns/op 0.87
shuffle list - 16384 els 6.8902 ms/op 7.9008 ms/op 0.87
shuffle list - 250000 els 100.90 ms/op 110.40 ms/op 0.91
processSlot - 1 slots 8.9650 us/op 10.026 us/op 0.89
processSlot - 32 slots 1.3341 ms/op 1.5374 ms/op 0.87
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 35.029 ms/op 42.787 ms/op 0.82
getCommitteeAssignments - req 1 vs - 250000 vc 2.9025 ms/op 3.3585 ms/op 0.86
getCommitteeAssignments - req 100 vs - 250000 vc 4.1271 ms/op 4.9030 ms/op 0.84
getCommitteeAssignments - req 1000 vs - 250000 vc 4.5078 ms/op 5.0630 ms/op 0.89
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.1500 ns/op 5.2500 ns/op 0.98
state getBlockRootAtSlot - 250000 vs - 7PWei 916.00 ns/op 958.60 ns/op 0.96
computeProposers - vc 250000 11.951 ms/op 12.242 ms/op 0.98
computeEpochShuffling - vc 250000 106.21 ms/op 113.11 ms/op 0.94
getNextSyncCommittee - vc 250000 185.97 ms/op 225.07 ms/op 0.83
computeSigningRoot for AttestationData 14.153 us/op 16.090 us/op 0.88
hash AttestationData serialized data then Buffer.toString(base64) 2.4337 us/op 2.6696 us/op 0.91
toHexString serialized data 1.3228 us/op 1.7523 us/op 0.75
Buffer.toString(base64) 403.66 ns/op 422.28 ns/op 0.96

by benchmarkbot/action

@nflaig nflaig mentioned this pull request Jun 8, 2023
Merged
@nflaig nflaig marked this pull request as ready for review June 8, 2023 11:50
@nflaig nflaig requested a review from a team as a code owner June 8, 2023 11:50
@nflaig nflaig force-pushed the nflaig/reuseable-thread-pool branch from d608c94 to 0ca782a Compare June 8, 2023 13:15
wemeetagain
wemeetagain previously approved these changes Jun 8, 2023
View reviewed changes
Copy link
Member

@wemeetagain wemeetagain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job on the error handling.
Also having a cancelable decryptKeystoreDefinitions is really nice.

@nflaig
Copy link
Member Author

nflaig commented Jun 8, 2023

Also having a cancelable decryptKeystoreDefinitions is really nice.

Yeah, that one is quite important because if you force close the process then the .lock files are not removed and need to be manually cleaned up.

@nflaig nflaig force-pushed the nflaig/reuseable-thread-pool branch from 0ca782a to 6eb8210 Compare June 8, 2023 16:11
@nflaig nflaig requested a review from wemeetagain June 8, 2023 17:16
@nflaig nflaig force-pushed the nflaig/reuseable-thread-pool branch from 6eb8210 to a529ddf Compare June 9, 2023 09:56
@wemeetagain wemeetagain merged commit 8463445 into unstable Jun 9, 2023
11 checks passed
@wemeetagain wemeetagain deleted the nflaig/reuseable-thread-pool branch June 9, 2023 15:19
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.9.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@nflaig @wemeetagain