Add dumpDb debug routes

[dapplion]

Motivation

Useful tools (not enabled by default) to debug db state and retained keys.

• Allowed me to confirm this issue StateArchiver use epoch as key instead of slot #4417

Description

Add routes:

• dumpDbBucketKeys: Dump level-db entry keys for a given Bucket declared in code, or for all buckets.
• dumpDbStateIndex: Return all entries in the StateArchive index with bucket index_stateArchiveRootIndex

[github-actions]

Performance Report

✔️ no performance regression detected

Full benchmark results

Benchmark suite	Current: 3cb8e39	Previous: db9450b	Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc	1.5330 ms/op	1.8763 ms/op	0.82
getPubkeys - validatorsArr - req 1000 vs - 250000 vc	63.704 us/op	64.459 us/op	0.99
BLS verify - blst-native	2.1637 ms/op	2.1650 ms/op	1.00
BLS verifyMultipleSignatures 3 - blst-native	4.4745 ms/op	4.4676 ms/op	1.00
BLS verifyMultipleSignatures 8 - blst-native	9.6728 ms/op	9.6656 ms/op	1.00
BLS verifyMultipleSignatures 32 - blst-native	35.161 ms/op	35.132 ms/op	1.00
BLS aggregatePubkeys 32 - blst-native	46.483 us/op	46.657 us/op	1.00
BLS aggregatePubkeys 128 - blst-native	182.37 us/op	182.50 us/op	1.00
getAttestationsForBlock	142.45 ms/op	150.16 ms/op	0.95
isKnown best case - 1 super set check	493.00 ns/op	488.00 ns/op	1.01
isKnown normal case - 2 super set checks	476.00 ns/op	473.00 ns/op	1.01
isKnown worse case - 16 super set checks	475.00 ns/op	475.00 ns/op	1.00
CheckpointStateCache - add get delete	8.7180 us/op	9.0660 us/op	0.96
validate gossip signedAggregateAndProof - struct	5.0242 ms/op	5.0207 ms/op	1.00
validate gossip attestation - struct	2.3619 ms/op	2.3601 ms/op	1.00
altair verifyImport mainnet_s3766816:31	4.7843 s/op	4.7097 s/op	1.02
pickEth1Vote - no votes	2.0843 ms/op	2.0655 ms/op	1.01
pickEth1Vote - max votes	19.659 ms/op	20.521 ms/op	0.96
pickEth1Vote - Eth1Data hashTreeRoot value x2048	12.061 ms/op	14.537 ms/op	0.83
pickEth1Vote - Eth1Data hashTreeRoot tree x2048	22.092 ms/op	19.627 ms/op	1.13
pickEth1Vote - Eth1Data fastSerialize value x2048	1.3498 ms/op	1.4965 ms/op	0.90
pickEth1Vote - Eth1Data fastSerialize tree x2048	11.459 ms/op	12.745 ms/op	0.90
bytes32 toHexString	868.00 ns/op	932.00 ns/op	0.93
bytes32 Buffer.toString(hex)	752.00 ns/op	747.00 ns/op	1.01
bytes32 Buffer.toString(hex) from Uint8Array	971.00 ns/op	948.00 ns/op	1.02
bytes32 Buffer.toString(hex) + 0x	750.00 ns/op	770.00 ns/op	0.97
Object access 1 prop	0.32400 ns/op	0.35700 ns/op	0.91
Map access 1 prop	0.29900 ns/op	0.34000 ns/op	0.88
Object get x1000	10.806 ns/op	11.325 ns/op	0.95
Map get x1000	0.92500 ns/op	0.93900 ns/op	0.99
Object set x1000	64.451 ns/op	68.296 ns/op	0.94
Map set x1000	44.327 ns/op	45.937 ns/op	0.96
Return object 10000 times	0.43280 ns/op	0.43460 ns/op	1.00
Throw Error 10000 times	5.9303 us/op	5.9406 us/op	1.00
enrSubnets - fastDeserialize 64 bits	2.5570 us/op	2.6050 us/op	0.98
enrSubnets - ssz BitVector 64 bits	771.00 ns/op	829.00 ns/op	0.93
enrSubnets - fastDeserialize 4 bits	383.00 ns/op	378.00 ns/op	1.01
enrSubnets - ssz BitVector 4 bits	784.00 ns/op	818.00 ns/op	0.96
prioritizePeers score -10:0 att 32-0.1 sync 2-0	79.589 us/op	82.694 us/op	0.96
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25	114.93 us/op	124.67 us/op	0.92
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5	188.84 us/op	201.26 us/op	0.94
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75	366.06 us/op	336.84 us/op	1.09
prioritizePeers score 0:0 att 64-1 sync 4-1	406.29 us/op	405.63 us/op	1.00
RateTracker 1000000 limit, 1 obj count per request	177.69 ns/op	182.62 ns/op	0.97
RateTracker 1000000 limit, 2 obj count per request	130.13 ns/op	133.46 ns/op	0.98
RateTracker 1000000 limit, 4 obj count per request	107.34 ns/op	109.56 ns/op	0.98
RateTracker 1000000 limit, 8 obj count per request	94.488 ns/op	95.347 ns/op	0.99
RateTracker with prune	3.7060 us/op	3.6480 us/op	1.02
array of 16000 items push then shift	51.581 us/op	51.574 us/op	1.00
LinkedList of 16000 items push then shift	14.916 ns/op	16.709 ns/op	0.89
array of 16000 items push then pop	204.51 ns/op	198.89 ns/op	1.03
LinkedList of 16000 items push then pop	13.805 ns/op	14.404 ns/op	0.96
array of 24000 items push then shift	77.341 us/op	77.587 us/op	1.00
LinkedList of 24000 items push then shift	17.503 ns/op	20.431 ns/op	0.86
array of 24000 items push then pop	190.15 ns/op	198.59 ns/op	0.96
LinkedList of 24000 items push then pop	14.682 ns/op	15.486 ns/op	0.95
intersect bitArray bitLen 8	11.069 ns/op	10.666 ns/op	1.04
intersect array and set length 8	127.50 ns/op	130.35 ns/op	0.98
intersect bitArray bitLen 128	55.497 ns/op	57.682 ns/op	0.96
intersect array and set length 128	1.6777 us/op	1.9544 us/op	0.86
Buffer.concat 32 items	1.7400 ns/op	1.8510 ns/op	0.94
pass gossip attestations to forkchoice per slot	2.8714 ms/op	3.2350 ms/op	0.89
computeDeltas	4.6799 ms/op	3.6273 ms/op	1.29
computeProposerBoostScoreFromBalances	806.44 us/op	804.02 us/op	1.00
altair processAttestation - 250000 vs - 7PWei normalcase	3.4437 ms/op	3.8228 ms/op	0.90
altair processAttestation - 250000 vs - 7PWei worstcase	5.1789 ms/op	5.5919 ms/op	0.93
altair processAttestation - setStatus - 1/6 committees join	176.12 us/op	175.34 us/op	1.00
altair processAttestation - setStatus - 1/3 committees join	342.45 us/op	343.79 us/op	1.00
altair processAttestation - setStatus - 1/2 committees join	483.61 us/op	491.20 us/op	0.98
altair processAttestation - setStatus - 2/3 committees join	641.17 us/op	645.52 us/op	0.99
altair processAttestation - setStatus - 4/5 committees join	891.53 us/op	903.24 us/op	0.99
altair processAttestation - setStatus - 100% committees join	1.0664 ms/op	1.0880 ms/op	0.98
altair processBlock - 250000 vs - 7PWei normalcase	25.731 ms/op	25.142 ms/op	1.02
altair processBlock - 250000 vs - 7PWei normalcase hashState	38.678 ms/op	39.817 ms/op	0.97
altair processBlock - 250000 vs - 7PWei worstcase	92.786 ms/op	83.524 ms/op	1.11
altair processBlock - 250000 vs - 7PWei worstcase hashState	96.907 ms/op	98.284 ms/op	0.99
phase0 processBlock - 250000 vs - 7PWei normalcase	3.1517 ms/op	3.9004 ms/op	0.81
phase0 processBlock - 250000 vs - 7PWei worstcase	50.623 ms/op	52.992 ms/op	0.96
altair processEth1Data - 250000 vs - 7PWei normalcase	1.0011 ms/op	722.93 us/op	1.38
Tree 40 250000 create	688.90 ms/op	741.34 ms/op	0.93
Tree 40 250000 get(125000)	262.79 ns/op	255.31 ns/op	1.03
Tree 40 250000 set(125000)	1.8191 us/op	2.2361 us/op	0.81
Tree 40 250000 toArray()	26.296 ms/op	27.856 ms/op	0.94
Tree 40 250000 iterate all - toArray() + loop	26.632 ms/op	28.198 ms/op	0.94
Tree 40 250000 iterate all - get(i)	108.10 ms/op	109.70 ms/op	0.99
MutableVector 250000 create	20.140 ms/op	13.322 ms/op	1.51
MutableVector 250000 get(125000)	11.215 ns/op	10.690 ns/op	1.05
MutableVector 250000 set(125000)	426.39 ns/op	556.03 ns/op	0.77
MutableVector 250000 toArray()	5.4524 ms/op	5.9009 ms/op	0.92
MutableVector 250000 iterate all - toArray() + loop	5.6184 ms/op	5.9496 ms/op	0.94
MutableVector 250000 iterate all - get(i)	2.6849 ms/op	2.6822 ms/op	1.00
Array 250000 create	5.3169 ms/op	5.8296 ms/op	0.91
Array 250000 clone - spread	2.4027 ms/op	3.5519 ms/op	0.68
Array 250000 get(125000)	1.1820 ns/op	1.6280 ns/op	0.73
Array 250000 set(125000)	1.1720 ns/op	1.6650 ns/op	0.70
Array 250000 iterate all - loop	158.38 us/op	150.98 us/op	1.05
effectiveBalanceIncrements clone Uint8Array 300000	64.934 us/op	78.458 us/op	0.83
effectiveBalanceIncrements clone MutableVector 300000	651.00 ns/op	721.00 ns/op	0.90
effectiveBalanceIncrements rw all Uint8Array 300000	247.29 us/op	247.53 us/op	1.00
effectiveBalanceIncrements rw all MutableVector 300000	130.20 ms/op	135.40 ms/op	0.96
phase0 afterProcessEpoch - 250000 vs - 7PWei	187.24 ms/op	201.32 ms/op	0.93
phase0 beforeProcessEpoch - 250000 vs - 7PWei	115.77 ms/op	65.889 ms/op	1.76
altair processEpoch - mainnet_e81889	646.84 ms/op	578.48 ms/op	1.12
mainnet_e81889 - altair beforeProcessEpoch	167.81 ms/op	136.20 ms/op	1.23
mainnet_e81889 - altair processJustificationAndFinalization	27.442 us/op	17.437 us/op	1.57
mainnet_e81889 - altair processInactivityUpdates	8.7676 ms/op	9.6275 ms/op	0.91
mainnet_e81889 - altair processRewardsAndPenalties	161.72 ms/op	89.043 ms/op	1.82
mainnet_e81889 - altair processRegistryUpdates	5.3740 us/op	2.8610 us/op	1.88
mainnet_e81889 - altair processSlashings	1.1020 us/op	632.00 ns/op	1.74
mainnet_e81889 - altair processEth1DataReset	1.1350 us/op	657.00 ns/op	1.73
mainnet_e81889 - altair processEffectiveBalanceUpdates	2.5123 ms/op	1.9765 ms/op	1.27
mainnet_e81889 - altair processSlashingsReset	6.7810 us/op	4.2460 us/op	1.60
mainnet_e81889 - altair processRandaoMixesReset	6.7130 us/op	4.2370 us/op	1.58
mainnet_e81889 - altair processHistoricalRootsUpdate	889.00 ns/op	614.00 ns/op	1.45
mainnet_e81889 - altair processParticipationFlagUpdates	2.7170 us/op	2.2990 us/op	1.18
mainnet_e81889 - altair processSyncCommitteeUpdates	615.00 ns/op	620.00 ns/op	0.99
mainnet_e81889 - altair afterProcessEpoch	197.62 ms/op	203.16 ms/op	0.97
phase0 processEpoch - mainnet_e58758	615.21 ms/op	519.70 ms/op	1.18
mainnet_e58758 - phase0 beforeProcessEpoch	229.29 ms/op	201.67 ms/op	1.14
mainnet_e58758 - phase0 processJustificationAndFinalization	16.660 us/op	16.193 us/op	1.03
mainnet_e58758 - phase0 processRewardsAndPenalties	128.70 ms/op	121.92 ms/op	1.06
mainnet_e58758 - phase0 processRegistryUpdates	7.8540 us/op	7.9890 us/op	0.98
mainnet_e58758 - phase0 processSlashings	622.00 ns/op	613.00 ns/op	1.01
mainnet_e58758 - phase0 processEth1DataReset	599.00 ns/op	602.00 ns/op	1.00
mainnet_e58758 - phase0 processEffectiveBalanceUpdates	2.0253 ms/op	2.5015 ms/op	0.81
mainnet_e58758 - phase0 processSlashingsReset	5.9690 us/op	3.5410 us/op	1.69
mainnet_e58758 - phase0 processRandaoMixesReset	7.8860 us/op	5.2930 us/op	1.49
mainnet_e58758 - phase0 processHistoricalRootsUpdate	1.2100 us/op	655.00 ns/op	1.85
mainnet_e58758 - phase0 processParticipationRecordUpdates	4.4230 us/op	3.8990 us/op	1.13
mainnet_e58758 - phase0 afterProcessEpoch	163.19 ms/op	164.70 ms/op	0.99
phase0 processEffectiveBalanceUpdates - 250000 normalcase	2.0089 ms/op	1.9475 ms/op	1.03
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5	2.2615 ms/op	2.2997 ms/op	0.98
altair processInactivityUpdates - 250000 normalcase	55.366 ms/op	41.575 ms/op	1.33
altair processInactivityUpdates - 250000 worstcase	53.369 ms/op	34.011 ms/op	1.57
phase0 processRegistryUpdates - 250000 normalcase	6.1380 us/op	6.5220 us/op	0.94
phase0 processRegistryUpdates - 250000 badcase_full_deposits	384.13 us/op	381.42 us/op	1.01
phase0 processRegistryUpdates - 250000 worstcase 0.5	224.30 ms/op	205.82 ms/op	1.09
altair processRewardsAndPenalties - 250000 normalcase	138.87 ms/op	104.20 ms/op	1.33
altair processRewardsAndPenalties - 250000 worstcase	138.29 ms/op	75.524 ms/op	1.83
phase0 getAttestationDeltas - 250000 normalcase	11.780 ms/op	12.209 ms/op	0.96
phase0 getAttestationDeltas - 250000 worstcase	11.825 ms/op	12.628 ms/op	0.94
phase0 processSlashings - 250000 worstcase	5.1403 ms/op	5.0471 ms/op	1.02
altair processSyncCommitteeUpdates - 250000	295.16 ms/op	300.61 ms/op	0.98
BeaconState.hashTreeRoot - No change	536.00 ns/op	544.00 ns/op	0.99
BeaconState.hashTreeRoot - 1 full validator	72.691 us/op	75.610 us/op	0.96
BeaconState.hashTreeRoot - 32 full validator	735.22 us/op	730.41 us/op	1.01
BeaconState.hashTreeRoot - 512 full validator	9.2220 ms/op	6.8999 ms/op	1.34
BeaconState.hashTreeRoot - 1 validator.effectiveBalance	90.158 us/op	100.83 us/op	0.89
BeaconState.hashTreeRoot - 32 validator.effectiveBalance	1.3859 ms/op	1.3023 ms/op	1.06
BeaconState.hashTreeRoot - 512 validator.effectiveBalance	16.915 ms/op	16.695 ms/op	1.01
BeaconState.hashTreeRoot - 1 balances	67.015 us/op	71.114 us/op	0.94
BeaconState.hashTreeRoot - 32 balances	699.98 us/op	653.86 us/op	1.07
BeaconState.hashTreeRoot - 512 balances	7.0829 ms/op	6.3981 ms/op	1.11
BeaconState.hashTreeRoot - 250000 balances	117.21 ms/op	101.79 ms/op	1.15
aggregationBits - 2048 els - zipIndexesInBitList	25.937 us/op	29.726 us/op	0.87
regular array get 100000 times	61.770 us/op	64.693 us/op	0.95
wrappedArray get 100000 times	60.580 us/op	62.745 us/op	0.97
arrayWithProxy get 100000 times	33.897 ms/op	30.152 ms/op	1.12
ssz.Root.equals	538.00 ns/op	588.00 ns/op	0.91
byteArrayEquals	529.00 ns/op	567.00 ns/op	0.93
shuffle list - 16384 els	11.518 ms/op	11.577 ms/op	0.99
shuffle list - 250000 els	167.50 ms/op	169.33 ms/op	0.99
processSlot - 1 slots	13.127 us/op	14.504 us/op	0.91
processSlot - 32 slots	1.9322 ms/op	2.0591 ms/op	0.94
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei	386.56 us/op	387.72 us/op	1.00
getCommitteeAssignments - req 1 vs - 250000 vc	5.4847 ms/op	5.6009 ms/op	0.98
getCommitteeAssignments - req 100 vs - 250000 vc	7.9761 ms/op	8.1977 ms/op	0.97
getCommitteeAssignments - req 1000 vs - 250000 vc	8.6173 ms/op	8.6539 ms/op	1.00
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei	7.6700 ns/op	8.3500 ns/op	0.92
state getBlockRootAtSlot - 250000 vs - 7PWei	994.54 ns/op	1.0721 us/op	0.93
computeProposers - vc 250000	17.580 ms/op	17.887 ms/op	0.98
computeEpochShuffling - vc 250000	171.98 ms/op	171.36 ms/op	1.00
getNextSyncCommittee - vc 250000	294.88 ms/op	303.17 ms/op	0.97

by benchmarkbot/action