Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: move block getter from API impl to chain #5570

Merged
merged 4 commits into from
May 31, 2023

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented May 29, 2023

Motivation

Our strategy to store blocks in not trivial and should be managed by the chain class not the API impl. The API impl should be concerned only with resolving the request payload and then forwarding to the chain.

Description

Chain exposes two methods for it

  /**
   * Since we can have multiple parallel chains,
   * this methods returns blocks in current chain head according to
   * forkchoice. Works for finalized slots as well
   */
  getCanonicalBlockAtSlot(
    slot: Slot
  ): Promise<{block: allForks.SignedBeaconBlock; executionOptimistic: boolean} | null>;
  /**
   * Get local block by root, does not fetch from the network
   */
  getBlockByRoot(root: RootHex): Promise<{block: allForks.SignedBeaconBlock; executionOptimistic: boolean} | null>;

An edge case important to note is what happens if an archive operation happens during a getBlock request. In the future we may want to have a lock mechanism on during archiving, which will be handled by the chain class

Move block getter from API impl to chain

@dapplion dapplion requested a review from a team as a code owner May 29, 2023 15:41
@dapplion dapplion changed the title Move block getter from API impl to chain refactor: move block getter from API impl to chain May 29, 2023
@github-actions
Copy link
Contributor

github-actions bot commented May 29, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: edcf9b0 Previous: 3fe9afd Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 710.86 us/op 597.99 us/op 1.19
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 68.305 us/op 47.462 us/op 1.44
BLS verify - blst-native 1.4374 ms/op 1.2127 ms/op 1.19
BLS verifyMultipleSignatures 3 - blst-native 2.7114 ms/op 2.4645 ms/op 1.10
BLS verifyMultipleSignatures 8 - blst-native 6.0793 ms/op 5.3099 ms/op 1.14
BLS verifyMultipleSignatures 32 - blst-native 23.440 ms/op 19.063 ms/op 1.23
BLS aggregatePubkeys 32 - blst-native 28.795 us/op 25.734 us/op 1.12
BLS aggregatePubkeys 128 - blst-native 109.07 us/op 99.868 us/op 1.09
getAttestationsForBlock 112.60 ms/op 62.678 ms/op 1.80
isKnown best case - 1 super set check 508.00 ns/op 268.00 ns/op 1.90
isKnown normal case - 2 super set checks 359.00 ns/op 255.00 ns/op 1.41
isKnown worse case - 16 super set checks 290.00 ns/op 256.00 ns/op 1.13
CheckpointStateCache - add get delete 6.5240 us/op 5.1540 us/op 1.27
validate gossip signedAggregateAndProof - struct 2.9835 ms/op 2.7685 ms/op 1.08
validate gossip attestation - struct 1.4129 ms/op 1.3209 ms/op 1.07
pickEth1Vote - no votes 1.6074 ms/op 1.2958 ms/op 1.24
pickEth1Vote - max votes 15.156 ms/op 10.701 ms/op 1.42
pickEth1Vote - Eth1Data hashTreeRoot value x2048 10.822 ms/op 9.7034 ms/op 1.12
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 17.639 ms/op 17.497 ms/op 1.01
pickEth1Vote - Eth1Data fastSerialize value x2048 815.99 us/op 774.56 us/op 1.05
pickEth1Vote - Eth1Data fastSerialize tree x2048 8.3813 ms/op 5.6485 ms/op 1.48
bytes32 toHexString 801.00 ns/op 609.00 ns/op 1.32
bytes32 Buffer.toString(hex) 543.00 ns/op 415.00 ns/op 1.31
bytes32 Buffer.toString(hex) from Uint8Array 886.00 ns/op 611.00 ns/op 1.45
bytes32 Buffer.toString(hex) + 0x 483.00 ns/op 412.00 ns/op 1.17
Object access 1 prop 0.23600 ns/op 0.20900 ns/op 1.13
Map access 1 prop 0.17600 ns/op 0.17300 ns/op 1.02
Object get x1000 8.8530 ns/op 6.8100 ns/op 1.30
Map get x1000 0.76600 ns/op 0.60900 ns/op 1.26
Object set x1000 108.28 ns/op 62.603 ns/op 1.73
Map set x1000 73.916 ns/op 46.544 ns/op 1.59
Return object 10000 times 0.42000 ns/op 0.25000 ns/op 1.68
Throw Error 10000 times 4.9882 us/op 4.2328 us/op 1.18
fastMsgIdFn sha256 / 200 bytes 4.3920 us/op 3.5410 us/op 1.24
fastMsgIdFn h32 xxhash / 200 bytes 418.00 ns/op 313.00 ns/op 1.34
fastMsgIdFn h64 xxhash / 200 bytes 720.00 ns/op 460.00 ns/op 1.57
fastMsgIdFn sha256 / 1000 bytes 12.714 us/op 11.734 us/op 1.08
fastMsgIdFn h32 xxhash / 1000 bytes 531.00 ns/op 439.00 ns/op 1.21
fastMsgIdFn h64 xxhash / 1000 bytes 643.00 ns/op 536.00 ns/op 1.20
fastMsgIdFn sha256 / 10000 bytes 108.14 us/op 104.01 us/op 1.04
fastMsgIdFn h32 xxhash / 10000 bytes 2.1250 us/op 1.9990 us/op 1.06
fastMsgIdFn h64 xxhash / 10000 bytes 1.5020 us/op 1.4260 us/op 1.05
enrSubnets - fastDeserialize 64 bits 2.1230 us/op 1.3750 us/op 1.54
enrSubnets - ssz BitVector 64 bits 716.00 ns/op 522.00 ns/op 1.37
enrSubnets - fastDeserialize 4 bits 249.00 ns/op 184.00 ns/op 1.35
enrSubnets - ssz BitVector 4 bits 729.00 ns/op 524.00 ns/op 1.39
prioritizePeers score -10:0 att 32-0.1 sync 2-0 135.54 us/op 121.94 us/op 1.11
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 197.70 us/op 157.65 us/op 1.25
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 233.93 us/op 194.54 us/op 1.20
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 497.91 us/op 350.12 us/op 1.42
prioritizePeers score 0:0 att 64-1 sync 4-1 526.75 us/op 446.46 us/op 1.18
array of 16000 items push then shift 1.8321 us/op 1.6404 us/op 1.12
LinkedList of 16000 items push then shift 9.5850 ns/op 8.8810 ns/op 1.08
array of 16000 items push then pop 105.06 ns/op 106.30 ns/op 0.99
LinkedList of 16000 items push then pop 8.8990 ns/op 8.8180 ns/op 1.01
array of 24000 items push then shift 2.4751 us/op 2.3728 us/op 1.04
LinkedList of 24000 items push then shift 9.1960 ns/op 9.7250 ns/op 0.95
array of 24000 items push then pop 91.280 ns/op 87.980 ns/op 1.04
LinkedList of 24000 items push then pop 9.3530 ns/op 8.9140 ns/op 1.05
intersect bitArray bitLen 8 13.991 ns/op 13.393 ns/op 1.04
intersect array and set length 8 88.763 ns/op 82.443 ns/op 1.08
intersect bitArray bitLen 128 46.615 ns/op 44.815 ns/op 1.04
intersect array and set length 128 1.3744 us/op 1.1815 us/op 1.16
Buffer.concat 32 items 3.1980 us/op 2.6840 us/op 1.19
Uint8Array.set 32 items 3.0610 us/op 2.6870 us/op 1.14
transfer serialized Status (84 B) 2.0250 us/op 2.1700 us/op 0.93
copy serialized Status (84 B) 1.6420 us/op 1.7610 us/op 0.93
transfer serialized SignedVoluntaryExit (112 B) 2.0950 us/op 2.2200 us/op 0.94
copy serialized SignedVoluntaryExit (112 B) 1.7060 us/op 1.9000 us/op 0.90
transfer serialized ProposerSlashing (416 B) 2.2360 us/op 3.0210 us/op 0.74
copy serialized ProposerSlashing (416 B) 2.6070 us/op 2.8350 us/op 0.92
transfer serialized Attestation (485 B) 2.5570 us/op 2.7480 us/op 0.93
copy serialized Attestation (485 B) 2.4390 us/op 2.6720 us/op 0.91
transfer serialized AttesterSlashing (33232 B) 2.4180 us/op 2.7300 us/op 0.89
copy serialized AttesterSlashing (33232 B) 5.9510 us/op 7.3210 us/op 0.81
transfer serialized Small SignedBeaconBlock (128000 B) 2.9090 us/op 2.9580 us/op 0.98
copy serialized Small SignedBeaconBlock (128000 B) 18.115 us/op 48.292 us/op 0.38
transfer serialized Avg SignedBeaconBlock (200000 B) 2.9730 us/op 3.2400 us/op 0.92
copy serialized Avg SignedBeaconBlock (200000 B) 24.319 us/op 24.154 us/op 1.01
transfer serialized BlobsSidecar (524380 B) 3.6460 us/op 3.4340 us/op 1.06
copy serialized BlobsSidecar (524380 B) 80.711 us/op 182.41 us/op 0.44
transfer serialized Big SignedBeaconBlock (1000000 B) 3.8570 us/op 3.5030 us/op 1.10
copy serialized Big SignedBeaconBlock (1000000 B) 360.02 us/op 385.26 us/op 0.93
pass gossip attestations to forkchoice per slot 2.3159 ms/op 2.3524 ms/op 0.98
computeDeltas 3.3852 ms/op 3.3581 ms/op 1.01
computeProposerBoostScoreFromBalances 1.8599 ms/op 1.7891 ms/op 1.04
altair processAttestation - 250000 vs - 7PWei normalcase 2.9691 ms/op 2.5162 ms/op 1.18
altair processAttestation - 250000 vs - 7PWei worstcase 3.5852 ms/op 4.4035 ms/op 0.81
altair processAttestation - setStatus - 1/6 committees join 145.56 us/op 150.13 us/op 0.97
altair processAttestation - setStatus - 1/3 committees join 302.43 us/op 285.56 us/op 1.06
altair processAttestation - setStatus - 1/2 committees join 379.10 us/op 374.19 us/op 1.01
altair processAttestation - setStatus - 2/3 committees join 535.56 us/op 468.76 us/op 1.14
altair processAttestation - setStatus - 4/5 committees join 690.98 us/op 656.96 us/op 1.05
altair processAttestation - setStatus - 100% committees join 839.57 us/op 773.46 us/op 1.09
altair processBlock - 250000 vs - 7PWei normalcase 22.687 ms/op 19.864 ms/op 1.14
altair processBlock - 250000 vs - 7PWei normalcase hashState 29.453 ms/op 27.914 ms/op 1.06
altair processBlock - 250000 vs - 7PWei worstcase 58.731 ms/op 55.521 ms/op 1.06
altair processBlock - 250000 vs - 7PWei worstcase hashState 86.900 ms/op 78.789 ms/op 1.10
phase0 processBlock - 250000 vs - 7PWei normalcase 2.2820 ms/op 2.2043 ms/op 1.04
phase0 processBlock - 250000 vs - 7PWei worstcase 33.093 ms/op 32.327 ms/op 1.02
altair processEth1Data - 250000 vs - 7PWei normalcase 539.59 us/op 571.08 us/op 0.94
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 14.384 us/op 9.6750 us/op 1.49
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 38.435 us/op 31.665 us/op 1.21
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 18.364 us/op 11.979 us/op 1.53
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 10.029 us/op 8.2360 us/op 1.22
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 109.27 us/op 104.54 us/op 1.05
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 706.02 us/op 682.50 us/op 1.03
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 951.48 us/op 1.1425 ms/op 0.83
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 933.64 us/op 1.0429 ms/op 0.90
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.7445 ms/op 2.8809 ms/op 1.30
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.8745 ms/op 2.1281 ms/op 0.88
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 6.0796 ms/op 4.1208 ms/op 1.48
Tree 40 250000 create 509.01 ms/op 827.76 ms/op 0.61
Tree 40 250000 get(125000) 200.10 ns/op 239.72 ns/op 0.83
Tree 40 250000 set(125000) 1.2120 us/op 1.9099 us/op 0.63
Tree 40 250000 toArray() 23.053 ms/op 23.976 ms/op 0.96
Tree 40 250000 iterate all - toArray() + loop 24.639 ms/op 24.220 ms/op 1.02
Tree 40 250000 iterate all - get(i) 82.806 ms/op 84.000 ms/op 0.99
MutableVector 250000 create 11.408 ms/op 12.769 ms/op 0.89
MutableVector 250000 get(125000) 6.8200 ns/op 6.6160 ns/op 1.03
MutableVector 250000 set(125000) 362.64 ns/op 261.36 ns/op 1.39
MutableVector 250000 toArray() 6.2620 ms/op 3.9734 ms/op 1.58
MutableVector 250000 iterate all - toArray() + loop 4.8006 ms/op 4.0761 ms/op 1.18
MutableVector 250000 iterate all - get(i) 1.5952 ms/op 1.5571 ms/op 1.02
Array 250000 create 4.6381 ms/op 3.4339 ms/op 1.35
Array 250000 clone - spread 1.8748 ms/op 1.1633 ms/op 1.61
Array 250000 get(125000) 0.80000 ns/op 0.59000 ns/op 1.36
Array 250000 set(125000) 1.3680 ns/op 0.64600 ns/op 2.12
Array 250000 iterate all - loop 119.59 us/op 95.317 us/op 1.25
effectiveBalanceIncrements clone Uint8Array 300000 59.559 us/op 38.194 us/op 1.56
effectiveBalanceIncrements clone MutableVector 300000 368.00 ns/op 351.00 ns/op 1.05
effectiveBalanceIncrements rw all Uint8Array 300000 186.19 us/op 174.09 us/op 1.07
effectiveBalanceIncrements rw all MutableVector 300000 103.58 ms/op 93.955 ms/op 1.10
phase0 afterProcessEpoch - 250000 vs - 7PWei 123.01 ms/op 117.06 ms/op 1.05
phase0 beforeProcessEpoch - 250000 vs - 7PWei 47.906 ms/op 49.717 ms/op 0.96
altair processEpoch - mainnet_e81889 360.93 ms/op 447.88 ms/op 0.81
mainnet_e81889 - altair beforeProcessEpoch 87.015 ms/op 89.741 ms/op 0.97
mainnet_e81889 - altair processJustificationAndFinalization 24.581 us/op 37.715 us/op 0.65
mainnet_e81889 - altair processInactivityUpdates 6.7472 ms/op 10.618 ms/op 0.64
mainnet_e81889 - altair processRewardsAndPenalties 63.177 ms/op 72.953 ms/op 0.87
mainnet_e81889 - altair processRegistryUpdates 3.7640 us/op 7.4790 us/op 0.50
mainnet_e81889 - altair processSlashings 926.00 ns/op 1.6030 us/op 0.58
mainnet_e81889 - altair processEth1DataReset 798.00 ns/op 1.9020 us/op 0.42
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2720 ms/op 2.2827 ms/op 0.56
mainnet_e81889 - altair processSlashingsReset 5.0060 us/op 8.8520 us/op 0.57
mainnet_e81889 - altair processRandaoMixesReset 12.116 us/op 11.049 us/op 1.10
mainnet_e81889 - altair processHistoricalRootsUpdate 1.2500 us/op 2.3440 us/op 0.53
mainnet_e81889 - altair processParticipationFlagUpdates 3.0110 us/op 6.7540 us/op 0.45
mainnet_e81889 - altair processSyncCommitteeUpdates 643.00 ns/op 1.2930 us/op 0.50
mainnet_e81889 - altair afterProcessEpoch 131.01 ms/op 148.74 ms/op 0.88
phase0 processEpoch - mainnet_e58758 374.95 ms/op 451.89 ms/op 0.83
mainnet_e58758 - phase0 beforeProcessEpoch 140.00 ms/op 189.87 ms/op 0.74
mainnet_e58758 - phase0 processJustificationAndFinalization 17.719 us/op 29.562 us/op 0.60
mainnet_e58758 - phase0 processRewardsAndPenalties 63.068 ms/op 70.365 ms/op 0.90
mainnet_e58758 - phase0 processRegistryUpdates 10.005 us/op 17.254 us/op 0.58
mainnet_e58758 - phase0 processSlashings 662.00 ns/op 1.4250 us/op 0.46
mainnet_e58758 - phase0 processEth1DataReset 686.00 ns/op 1.2930 us/op 0.53
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0645 ms/op 1.5619 ms/op 0.68
mainnet_e58758 - phase0 processSlashingsReset 4.0200 us/op 10.769 us/op 0.37
mainnet_e58758 - phase0 processRandaoMixesReset 4.9150 us/op 10.050 us/op 0.49
mainnet_e58758 - phase0 processHistoricalRootsUpdate 648.00 ns/op 2.1400 us/op 0.30
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.5700 us/op 8.4120 us/op 0.54
mainnet_e58758 - phase0 afterProcessEpoch 101.63 ms/op 104.47 ms/op 0.97
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2890 ms/op 1.7328 ms/op 0.74
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.6082 ms/op 2.9707 ms/op 0.54
altair processInactivityUpdates - 250000 normalcase 24.523 ms/op 35.154 ms/op 0.70
altair processInactivityUpdates - 250000 worstcase 25.885 ms/op 32.367 ms/op 0.80
phase0 processRegistryUpdates - 250000 normalcase 8.1580 us/op 14.001 us/op 0.58
phase0 processRegistryUpdates - 250000 badcase_full_deposits 414.10 us/op 414.05 us/op 1.00
phase0 processRegistryUpdates - 250000 worstcase 0.5 134.41 ms/op 167.55 ms/op 0.80
altair processRewardsAndPenalties - 250000 normalcase 70.780 ms/op 85.829 ms/op 0.82
altair processRewardsAndPenalties - 250000 worstcase 69.475 ms/op 80.829 ms/op 0.86
phase0 getAttestationDeltas - 250000 normalcase 6.7989 ms/op 10.057 ms/op 0.68
phase0 getAttestationDeltas - 250000 worstcase 7.4389 ms/op 11.746 ms/op 0.63
phase0 processSlashings - 250000 worstcase 3.6957 ms/op 5.1376 ms/op 0.72
altair processSyncCommitteeUpdates - 250000 173.35 ms/op 219.80 ms/op 0.79
BeaconState.hashTreeRoot - No change 401.00 ns/op 407.00 ns/op 0.99
BeaconState.hashTreeRoot - 1 full validator 51.195 us/op 54.628 us/op 0.94
BeaconState.hashTreeRoot - 32 full validator 539.85 us/op 613.52 us/op 0.88
BeaconState.hashTreeRoot - 512 full validator 5.0588 ms/op 6.9084 ms/op 0.73
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 65.883 us/op 72.687 us/op 0.91
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 874.14 us/op 1.2608 ms/op 0.69
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 12.170 ms/op 15.789 ms/op 0.77
BeaconState.hashTreeRoot - 1 balances 51.565 us/op 60.237 us/op 0.86
BeaconState.hashTreeRoot - 32 balances 459.74 us/op 550.65 us/op 0.83
BeaconState.hashTreeRoot - 512 balances 4.6792 ms/op 5.9105 ms/op 0.79
BeaconState.hashTreeRoot - 250000 balances 76.436 ms/op 87.432 ms/op 0.87
aggregationBits - 2048 els - zipIndexesInBitList 16.848 us/op 24.246 us/op 0.69
regular array get 100000 times 43.368 us/op 36.303 us/op 1.19
wrappedArray get 100000 times 33.772 us/op 36.520 us/op 0.92
arrayWithProxy get 100000 times 16.372 ms/op 18.943 ms/op 0.86
ssz.Root.equals 596.00 ns/op 698.00 ns/op 0.85
byteArrayEquals 585.00 ns/op 705.00 ns/op 0.83
shuffle list - 16384 els 7.0018 ms/op 7.7340 ms/op 0.91
shuffle list - 250000 els 101.17 ms/op 116.23 ms/op 0.87
processSlot - 1 slots 8.7100 us/op 12.612 us/op 0.69
processSlot - 32 slots 1.3913 ms/op 1.7520 ms/op 0.79
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 34.401 ms/op 45.723 ms/op 0.75
getCommitteeAssignments - req 1 vs - 250000 vc 2.9304 ms/op 3.1955 ms/op 0.92
getCommitteeAssignments - req 100 vs - 250000 vc 4.1417 ms/op 4.6902 ms/op 0.88
getCommitteeAssignments - req 1000 vs - 250000 vc 4.5015 ms/op 4.9316 ms/op 0.91
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.5600 ns/op 5.6600 ns/op 0.81
state getBlockRootAtSlot - 250000 vs - 7PWei 839.31 ns/op 884.48 ns/op 0.95
computeProposers - vc 250000 10.095 ms/op 11.639 ms/op 0.87
computeEpochShuffling - vc 250000 101.14 ms/op 114.13 ms/op 0.89
getNextSyncCommittee - vc 250000 162.57 ms/op 221.91 ms/op 0.73
computeSigningRoot for AttestationData 13.162 us/op 16.682 us/op 0.79
hash AttestationData serialized data then Buffer.toString(base64) 2.4122 us/op 2.7233 us/op 0.89
toHexString serialized data 1.0543 us/op 1.8926 us/op 0.56
Buffer.toString(base64) 327.61 ns/op 429.29 ns/op 0.76

by benchmarkbot/action

@dapplion dapplion force-pushed the dapplion/block-api-to-chain branch from 2fdc823 to 60c776f Compare May 31, 2023 09:49
@wemeetagain wemeetagain merged commit cef8562 into unstable May 31, 2023
10 of 11 checks passed
@wemeetagain wemeetagain deleted the dapplion/block-api-to-chain branch May 31, 2023 17:41
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.9.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants