Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove ChainEvent.forkChoiceHead event #4553

Merged
merged 2 commits into from Sep 17, 2022
Merged

Remove ChainEvent.forkChoiceHead event #4553

merged 2 commits into from Sep 17, 2022

Conversation

dapplion
Copy link
Contributor

Motivation

It's unnecessary to have this intermediary event between what the eventstream requires and where it's emitted. Splitting sensitive code like this has been a source of inconsistencies and bugs in the past. In the case for example it hide away this really ugly code

      const state = chain.stateCache.get(head.stateRoot);
      if (!state) {
        throw Error("cannot get state for head " + head.stateRoot);
      }

      const currentEpoch = state.epochCtx.epoch;
      const [previousDutyDependentRoot, currentDutyDependentRoot] = [currentEpoch - 1, currentEpoch].map((epoch) =>
        toHexString(getBlockRootAtSlot(state, Math.max(computeStartSlotAtEpoch(epoch) - 1, 0)))
      );

This code forces dangerous assumptions:

  • When is the forkchoice:head event emitted? Is it emitted BEFORE or AFTER storing the state in the cache?
  • Is there always a state available for the head? If not, under what circumstances? If it's not available should the event be delayed?

Description

  • Remove intermediary head event
  • Move event data logic into sequential importBlock flow

@dapplion dapplion added the meta-technical-debt Issues introducing or resolving technical debts. label Sep 16, 2022
@dapplion dapplion requested a review from a team as a code owner September 16, 2022 09:47
@github-actions
Copy link
Contributor

github-actions bot commented Sep 16, 2022 •
edited

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 887eedd Previous: aa1af04 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.9931 ms/op 2.3679 ms/op 0.84
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 71.217 us/op 90.530 us/op 0.79
BLS verify - blst-native 1.8771 ms/op 2.3229 ms/op 0.81
BLS verifyMultipleSignatures 3 - blst-native 3.8019 ms/op 4.7552 ms/op 0.80
BLS verifyMultipleSignatures 8 - blst-native 8.1837 ms/op 10.245 ms/op 0.80
BLS verifyMultipleSignatures 32 - blst-native 29.666 ms/op 37.176 ms/op 0.80
BLS aggregatePubkeys 32 - blst-native 39.122 us/op 49.516 us/op 0.79
BLS aggregatePubkeys 128 - blst-native 153.07 us/op 191.28 us/op 0.80
getAttestationsForBlock 83.900 ms/op 105.20 ms/op 0.80
isKnown best case - 1 super set check 435.00 ns/op 539.00 ns/op 0.81
isKnown normal case - 2 super set checks 419.00 ns/op 519.00 ns/op 0.81
isKnown worse case - 16 super set checks 417.00 ns/op 522.00 ns/op 0.80
CheckpointStateCache - add get delete 9.0310 us/op 11.286 us/op 0.80
validate gossip signedAggregateAndProof - struct 4.2659 ms/op 5.3265 ms/op 0.80
validate gossip attestation - struct 2.0327 ms/op 2.5115 ms/op 0.81
pickEth1Vote - no votes 2.1970 ms/op 2.7329 ms/op 0.80
pickEth1Vote - max votes 18.637 ms/op 25.427 ms/op 0.73
pickEth1Vote - Eth1Data hashTreeRoot value x2048 10.698 ms/op 13.685 ms/op 0.78
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 19.214 ms/op 24.345 ms/op 0.79
pickEth1Vote - Eth1Data fastSerialize value x2048 1.4808 ms/op 1.9103 ms/op 0.78
pickEth1Vote - Eth1Data fastSerialize tree x2048 12.048 ms/op 16.470 ms/op 0.73
bytes32 toHexString 1.0070 us/op 1.3270 us/op 0.76
bytes32 Buffer.toString(hex) 752.00 ns/op 903.00 ns/op 0.83
bytes32 Buffer.toString(hex) from Uint8Array 929.00 ns/op 1.2190 us/op 0.76
bytes32 Buffer.toString(hex) + 0x 715.00 ns/op 879.00 ns/op 0.81
Object access 1 prop 0.34200 ns/op 0.44600 ns/op 0.77
Map access 1 prop 0.28600 ns/op 0.37000 ns/op 0.77
Object get x1000 17.785 ns/op 22.495 ns/op 0.79
Map get x1000 1.0760 ns/op 1.2550 ns/op 0.86
Object set x1000 112.88 ns/op 144.91 ns/op 0.78
Map set x1000 66.175 ns/op 87.521 ns/op 0.76
Return object 10000 times 0.37560 ns/op 0.46820 ns/op 0.80
Throw Error 10000 times 5.8624 us/op 7.1984 us/op 0.81
enrSubnets - fastDeserialize 64 bits 2.5190 us/op 3.2490 us/op 0.78
enrSubnets - ssz BitVector 64 bits 745.00 ns/op 941.00 ns/op 0.79
enrSubnets - fastDeserialize 4 bits 368.00 ns/op 479.00 ns/op 0.77
enrSubnets - ssz BitVector 4 bits 767.00 ns/op 951.00 ns/op 0.81
prioritizePeers score -10:0 att 32-0.1 sync 2-0 89.371 us/op 127.69 us/op 0.70
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 123.04 us/op 150.85 us/op 0.82
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 202.12 us/op 262.08 us/op 0.77
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 441.64 us/op 561.65 us/op 0.79
prioritizePeers score 0:0 att 64-1 sync 4-1 455.70 us/op 566.35 us/op 0.80
RateTracker 1000000 limit, 1 obj count per request 190.66 ns/op 253.12 ns/op 0.75
RateTracker 1000000 limit, 2 obj count per request 146.39 ns/op 195.76 ns/op 0.75
RateTracker 1000000 limit, 4 obj count per request 124.65 ns/op 164.43 ns/op 0.76
RateTracker 1000000 limit, 8 obj count per request 114.41 ns/op 151.76 ns/op 0.75
RateTracker with prune 4.0420 us/op 5.4640 us/op 0.74
array of 16000 items push then shift 3.0727 us/op 3.8483 us/op 0.80
LinkedList of 16000 items push then shift 16.890 ns/op 21.422 ns/op 0.79
array of 16000 items push then pop 236.67 ns/op 276.08 ns/op 0.86
LinkedList of 16000 items push then pop 16.427 ns/op 20.106 ns/op 0.82
array of 24000 items push then shift 4.5512 us/op 5.6888 us/op 0.80
LinkedList of 24000 items push then shift 19.720 ns/op 24.405 ns/op 0.81
array of 24000 items push then pop 210.16 ns/op 256.80 ns/op 0.82
LinkedList of 24000 items push then pop 17.723 ns/op 21.275 ns/op 0.83
intersect bitArray bitLen 8 11.560 ns/op 14.372 ns/op 0.80
intersect array and set length 8 160.17 ns/op 213.31 ns/op 0.75
intersect bitArray bitLen 128 71.964 ns/op 90.046 ns/op 0.80
intersect array and set length 128 2.1597 us/op 2.8166 us/op 0.77
Buffer.concat 32 items 1.8600 ns/op 2.3840 ns/op 0.78
pass gossip attestations to forkchoice per slot 3.1625 ms/op 6.5181 ms/op 0.49
computeDeltas 3.6648 ms/op 3.7689 ms/op 0.97
computeProposerBoostScoreFromBalances 921.29 us/op 1.1303 ms/op 0.82
altair processAttestation - 250000 vs - 7PWei normalcase 3.5065 ms/op 4.6712 ms/op 0.75
altair processAttestation - 250000 vs - 7PWei worstcase 5.4162 ms/op 7.7541 ms/op 0.70
altair processAttestation - setStatus - 1/6 committees join 206.70 us/op 268.46 us/op 0.77
altair processAttestation - setStatus - 1/3 committees join 396.99 us/op 509.74 us/op 0.78
altair processAttestation - setStatus - 1/2 committees join 552.89 us/op 716.98 us/op 0.77
altair processAttestation - setStatus - 2/3 committees join 716.28 us/op 919.62 us/op 0.78
altair processAttestation - setStatus - 4/5 committees join 985.31 us/op 1.2817 ms/op 0.77
altair processAttestation - setStatus - 100% committees join 1.1778 ms/op 1.4902 ms/op 0.79
altair processBlock - 250000 vs - 7PWei normalcase 27.102 ms/op 27.172 ms/op 1.00
altair processBlock - 250000 vs - 7PWei normalcase hashState 42.409 ms/op 46.304 ms/op 0.92
altair processBlock - 250000 vs - 7PWei worstcase 83.491 ms/op 95.678 ms/op 0.87
altair processBlock - 250000 vs - 7PWei worstcase hashState 107.70 ms/op 139.44 ms/op 0.77
phase0 processBlock - 250000 vs - 7PWei normalcase 3.6871 ms/op 4.3120 ms/op 0.86
phase0 processBlock - 250000 vs - 7PWei worstcase 45.570 ms/op 57.919 ms/op 0.79
altair processEth1Data - 250000 vs - 7PWei normalcase 789.85 us/op 1.0265 ms/op 0.77
Tree 40 250000 create 802.39 ms/op 979.01 ms/op 0.82
Tree 40 250000 get(125000) 290.38 ns/op 356.01 ns/op 0.82
Tree 40 250000 set(125000) 2.9514 us/op 2.8932 us/op 1.02
Tree 40 250000 toArray() 33.911 ms/op 38.330 ms/op 0.88
Tree 40 250000 iterate all - toArray() + loop 33.773 ms/op 38.555 ms/op 0.88
Tree 40 250000 iterate all - get(i) 115.05 ms/op 138.81 ms/op 0.83
MutableVector 250000 create 20.241 ms/op 22.141 ms/op 0.91
MutableVector 250000 get(125000) 15.260 ns/op 16.693 ns/op 0.91
MutableVector 250000 set(125000) 773.39 ns/op 725.54 ns/op 1.07
MutableVector 250000 toArray() 8.0323 ms/op 8.6315 ms/op 0.93
MutableVector 250000 iterate all - toArray() + loop 8.3415 ms/op 9.1098 ms/op 0.92
MutableVector 250000 iterate all - get(i) 3.4431 ms/op 4.3018 ms/op 0.80
Array 250000 create 7.3655 ms/op 8.1175 ms/op 0.91
Array 250000 clone - spread 4.1322 ms/op 4.1450 ms/op 1.00
Array 250000 get(125000) 1.5940 ns/op 1.7940 ns/op 0.89
Array 250000 set(125000) 1.5080 ns/op 1.8390 ns/op 0.82
Array 250000 iterate all - loop 167.76 us/op 209.85 us/op 0.80
effectiveBalanceIncrements clone Uint8Array 300000 79.741 us/op 104.65 us/op 0.76
effectiveBalanceIncrements clone MutableVector 300000 1.1740 us/op 1.3530 us/op 0.87
effectiveBalanceIncrements rw all Uint8Array 300000 252.55 us/op 318.55 us/op 0.79
effectiveBalanceIncrements rw all MutableVector 300000 201.93 ms/op 247.45 ms/op 0.82
phase0 afterProcessEpoch - 250000 vs - 7PWei 185.15 ms/op 228.17 ms/op 0.81
phase0 beforeProcessEpoch - 250000 vs - 7PWei 93.135 ms/op 119.87 ms/op 0.78
altair processEpoch - mainnet_e81889 500.30 ms/op 710.33 ms/op 0.70
mainnet_e81889 - altair beforeProcessEpoch 137.90 ms/op 167.02 ms/op 0.83
mainnet_e81889 - altair processJustificationAndFinalization 29.448 us/op 40.810 us/op 0.72
mainnet_e81889 - altair processInactivityUpdates 11.546 ms/op 13.511 ms/op 0.85
mainnet_e81889 - altair processRewardsAndPenalties 91.542 ms/op 111.39 ms/op 0.82
mainnet_e81889 - altair processRegistryUpdates 5.4350 us/op 7.9320 us/op 0.69
mainnet_e81889 - altair processSlashings 1.0710 us/op 2.2040 us/op 0.49
mainnet_e81889 - altair processEth1DataReset 1.6210 us/op 2.1740 us/op 0.75
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.4152 ms/op 3.0310 ms/op 0.80
mainnet_e81889 - altair processSlashingsReset 8.2980 us/op 12.485 us/op 0.66
mainnet_e81889 - altair processRandaoMixesReset 8.2770 us/op 13.942 us/op 0.59
mainnet_e81889 - altair processHistoricalRootsUpdate 1.3230 us/op 2.7410 us/op 0.48
mainnet_e81889 - altair processParticipationFlagUpdates 4.4900 us/op 10.343 us/op 0.43
mainnet_e81889 - altair processSyncCommitteeUpdates 1.5230 us/op 2.1070 us/op 0.72
mainnet_e81889 - altair afterProcessEpoch 194.30 ms/op 239.95 ms/op 0.81
phase0 processEpoch - mainnet_e58758 524.79 ms/op 634.32 ms/op 0.83
mainnet_e58758 - phase0 beforeProcessEpoch 237.89 ms/op 265.72 ms/op 0.90
mainnet_e58758 - phase0 processJustificationAndFinalization 28.304 us/op 30.016 us/op 0.94
mainnet_e58758 - phase0 processRewardsAndPenalties 133.78 ms/op 173.02 ms/op 0.77
mainnet_e58758 - phase0 processRegistryUpdates 13.469 us/op 19.073 us/op 0.71
mainnet_e58758 - phase0 processSlashings 1.3190 us/op 1.8900 us/op 0.70
mainnet_e58758 - phase0 processEth1DataReset 1.3060 us/op 2.2560 us/op 0.58
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.2877 ms/op 2.7305 ms/op 0.84
mainnet_e58758 - phase0 processSlashingsReset 7.4160 us/op 11.026 us/op 0.67
mainnet_e58758 - phase0 processRandaoMixesReset 8.8880 us/op 14.396 us/op 0.62
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.6030 us/op 2.5110 us/op 0.64
mainnet_e58758 - phase0 processParticipationRecordUpdates 9.4150 us/op 11.280 us/op 0.83
mainnet_e58758 - phase0 afterProcessEpoch 158.82 ms/op 197.87 ms/op 0.80
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.5396 ms/op 3.4780 ms/op 0.73
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.5683 ms/op 3.7273 ms/op 0.96
altair processInactivityUpdates - 250000 normalcase 40.458 ms/op 47.149 ms/op 0.86
altair processInactivityUpdates - 250000 worstcase 49.006 ms/op 56.724 ms/op 0.86
phase0 processRegistryUpdates - 250000 normalcase 13.035 us/op 17.101 us/op 0.76
phase0 processRegistryUpdates - 250000 badcase_full_deposits 438.51 us/op 548.69 us/op 0.80
phase0 processRegistryUpdates - 250000 worstcase 0.5 214.62 ms/op 256.44 ms/op 0.84
altair processRewardsAndPenalties - 250000 normalcase 114.85 ms/op 160.42 ms/op 0.72
altair processRewardsAndPenalties - 250000 worstcase 113.10 ms/op 102.50 ms/op 1.10
phase0 getAttestationDeltas - 250000 normalcase 13.281 ms/op 15.591 ms/op 0.85
phase0 getAttestationDeltas - 250000 worstcase 13.467 ms/op 14.998 ms/op 0.90
phase0 processSlashings - 250000 worstcase 5.6170 ms/op 6.6428 ms/op 0.85
altair processSyncCommitteeUpdates - 250000 286.34 ms/op 345.42 ms/op 0.83
BeaconState.hashTreeRoot - No change 497.00 ns/op 601.00 ns/op 0.83
BeaconState.hashTreeRoot - 1 full validator 60.911 us/op 68.221 us/op 0.89
BeaconState.hashTreeRoot - 32 full validator 628.77 us/op 787.19 us/op 0.80
BeaconState.hashTreeRoot - 512 full validator 6.5053 ms/op 9.0885 ms/op 0.72
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 74.349 us/op 99.200 us/op 0.75
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2289 ms/op 1.4357 ms/op 0.86
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.243 ms/op 19.101 ms/op 0.85
BeaconState.hashTreeRoot - 1 balances 71.789 us/op 77.837 us/op 0.92
BeaconState.hashTreeRoot - 32 balances 527.50 us/op 732.25 us/op 0.72
BeaconState.hashTreeRoot - 512 balances 5.5289 ms/op 7.1845 ms/op 0.77
BeaconState.hashTreeRoot - 250000 balances 85.064 ms/op 119.26 ms/op 0.71
aggregationBits - 2048 els - zipIndexesInBitList 43.768 us/op 36.509 us/op 1.20
regular array get 100000 times 67.437 us/op 84.328 us/op 0.80
wrappedArray get 100000 times 67.453 us/op 84.283 us/op 0.80
arrayWithProxy get 100000 times 28.929 ms/op 36.571 ms/op 0.79
ssz.Root.equals 521.00 ns/op 593.00 ns/op 0.88
byteArrayEquals 512.00 ns/op 587.00 ns/op 0.87
shuffle list - 16384 els 11.171 ms/op 14.011 ms/op 0.80
shuffle list - 250000 els 164.51 ms/op 202.38 ms/op 0.81
processSlot - 1 slots 14.387 us/op 16.144 us/op 0.89
processSlot - 32 slots 1.7452 ms/op 2.1973 ms/op 0.79
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 385.07 us/op 432.78 us/op 0.89
getCommitteeAssignments - req 1 vs - 250000 vc 5.2712 ms/op 6.6798 ms/op 0.79
getCommitteeAssignments - req 100 vs - 250000 vc 7.3322 ms/op 9.1345 ms/op 0.80
getCommitteeAssignments - req 1000 vs - 250000 vc 7.8170 ms/op 9.8044 ms/op 0.80
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 10.110 ns/op 12.330 ns/op 0.82
state getBlockRootAtSlot - 250000 vs - 7PWei 1.0757 us/op 1.5862 us/op 0.68
computeProposers - vc 250000 16.312 ms/op 21.015 ms/op 0.78
computeEpochShuffling - vc 250000 166.11 ms/op 211.20 ms/op 0.79
getNextSyncCommittee - vc 250000 280.74 ms/op 349.84 ms/op 0.80

by benchmarkbot/action

@dapplion dapplion mentioned this pull request Sep 16, 2022
Closed
@dapplion dapplion marked this pull request as draft September 16, 2022 10:59
@dapplion
Copy link
Contributor Author

and use

      previousDutyDependentRoot: this.forkChoice.getDependantRoot(newHead, EpochDifference.previous),
      currentDutyDependentRoot: this.forkChoice.getDependantRoot(newHead, EpochDifference.current),

@dapplion dapplion marked this pull request as ready for review September 17, 2022 11:59
@wemeetagain wemeetagain changed the title Remove intermediary head event Remove ChainEvent.forkChoiceHead event Sep 17, 2022
@wemeetagain wemeetagain merged commit dd29d86 into unstable Sep 17, 2022
@wemeetagain wemeetagain deleted the dapplion/remove-head-event branch September 17, 2022 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
meta-technical-debt Issues introducing or resolving technical debts.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@dapplion @wemeetagain