Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor!: move network logic into NetworkCore #5351

Closed
wants to merge 9 commits into from

Conversation

wemeetagain
Copy link
Member

@wemeetagain wemeetagain commented Apr 7, 2023

Motivation

In anticipation of moving the network to a separate thread, move all network functionality to a separate module with an interface that may be implemented as a worker thread.

Description

In anticipation of moving the network to a separate thread, move all network functionality to a separate module with an interface that may be implemented as a worker thread.

Pulled into separate PRs

Important points

  • Move the core network logic into NetworkCore which implements INetworkCore
  • Track connected peers in the Network implementation to allow for synchronous getConnectedPeers and getConnectedPeerCount.
  • Track SimpleAttnetsService info to allow for synchronous shouldProcessAttestation during gossip and api attestation processing
  • A few renamings networkEventBus->network.events, prepareBeaconCommitteeSubnet->prepareBeaconCommitteeSubnets

@github-actions
Copy link
Contributor

github-actions bot commented Apr 7, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 3300b4b Previous: 4981f5e Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 902.19 us/op 482.97 us/op 1.87
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 44.463 us/op 45.482 us/op 0.98
BLS verify - blst-native 1.1794 ms/op 1.2176 ms/op 0.97
BLS verifyMultipleSignatures 3 - blst-native 2.3952 ms/op 2.4848 ms/op 0.96
BLS verifyMultipleSignatures 8 - blst-native 5.1399 ms/op 5.3299 ms/op 0.96
BLS verifyMultipleSignatures 32 - blst-native 18.643 ms/op 19.544 ms/op 0.95
BLS aggregatePubkeys 32 - blst-native 24.829 us/op 25.841 us/op 0.96
BLS aggregatePubkeys 128 - blst-native 97.337 us/op 100.35 us/op 0.97
getAttestationsForBlock 54.379 ms/op 51.454 ms/op 1.06
isKnown best case - 1 super set check 247.00 ns/op 254.00 ns/op 0.97
isKnown normal case - 2 super set checks 246.00 ns/op 252.00 ns/op 0.98
isKnown worse case - 16 super set checks 244.00 ns/op 248.00 ns/op 0.98
CheckpointStateCache - add get delete 4.9290 us/op 4.9030 us/op 1.01
validate gossip signedAggregateAndProof - struct 2.7462 ms/op 2.7690 ms/op 0.99
validate gossip attestation - struct 1.3226 ms/op 1.3384 ms/op 0.99
pickEth1Vote - no votes 1.2179 ms/op 1.2616 ms/op 0.97
pickEth1Vote - max votes 10.154 ms/op 9.5681 ms/op 1.06
pickEth1Vote - Eth1Data hashTreeRoot value x2048 8.5774 ms/op 8.5132 ms/op 1.01
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 14.698 ms/op 14.313 ms/op 1.03
pickEth1Vote - Eth1Data fastSerialize value x2048 690.93 us/op 648.33 us/op 1.07
pickEth1Vote - Eth1Data fastSerialize tree x2048 6.2972 ms/op 4.5720 ms/op 1.38
bytes32 toHexString 485.00 ns/op 478.00 ns/op 1.01
bytes32 Buffer.toString(hex) 337.00 ns/op 359.00 ns/op 0.94
bytes32 Buffer.toString(hex) from Uint8Array 543.00 ns/op 569.00 ns/op 0.95
bytes32 Buffer.toString(hex) + 0x 337.00 ns/op 359.00 ns/op 0.94
Object access 1 prop 0.16100 ns/op 0.16600 ns/op 0.97
Map access 1 prop 0.15800 ns/op 0.16800 ns/op 0.94
Object get x1000 6.6570 ns/op 6.5410 ns/op 1.02
Map get x1000 0.60400 ns/op 0.62900 ns/op 0.96
Object set x1000 51.183 ns/op 52.659 ns/op 0.97
Map set x1000 42.745 ns/op 43.412 ns/op 0.98
Return object 10000 times 0.23390 ns/op 0.23810 ns/op 0.98
Throw Error 10000 times 4.1459 us/op 4.1699 us/op 0.99
fastMsgIdFn sha256 / 200 bytes 3.4470 us/op 3.4610 us/op 1.00
fastMsgIdFn h32 xxhash / 200 bytes 278.00 ns/op 275.00 ns/op 1.01
fastMsgIdFn h64 xxhash / 200 bytes 395.00 ns/op 396.00 ns/op 1.00
fastMsgIdFn sha256 / 1000 bytes 11.655 us/op 11.672 us/op 1.00
fastMsgIdFn h32 xxhash / 1000 bytes 400.00 ns/op 440.00 ns/op 0.91
fastMsgIdFn h64 xxhash / 1000 bytes 469.00 ns/op 534.00 ns/op 0.88
fastMsgIdFn sha256 / 10000 bytes 103.62 us/op 105.33 us/op 0.98
fastMsgIdFn h32 xxhash / 10000 bytes 1.8890 us/op 2.0200 us/op 0.94
fastMsgIdFn h64 xxhash / 10000 bytes 1.3280 us/op 1.4650 us/op 0.91
enrSubnets - fastDeserialize 64 bits 1.2700 us/op 1.3770 us/op 0.92
enrSubnets - ssz BitVector 64 bits 498.00 ns/op 544.00 ns/op 0.92
enrSubnets - fastDeserialize 4 bits 172.00 ns/op 182.00 ns/op 0.95
enrSubnets - ssz BitVector 4 bits 499.00 ns/op 508.00 ns/op 0.98
prioritizePeers score -10:0 att 32-0.1 sync 2-0 100.98 us/op 111.94 us/op 0.90
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 130.60 us/op 149.36 us/op 0.87
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 169.97 us/op 188.87 us/op 0.90
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 298.23 us/op 317.94 us/op 0.94
prioritizePeers score 0:0 att 64-1 sync 4-1 360.68 us/op 422.65 us/op 0.85
array of 16000 items push then shift 1.6058 us/op 1.6532 us/op 0.97
LinkedList of 16000 items push then shift 8.8270 ns/op 8.9280 ns/op 0.99
array of 16000 items push then pop 76.715 ns/op 106.55 ns/op 0.72
LinkedList of 16000 items push then pop 8.4770 ns/op 8.6990 ns/op 0.97
array of 24000 items push then shift 2.3326 us/op 2.3781 us/op 0.98
LinkedList of 24000 items push then shift 8.8820 ns/op 8.8590 ns/op 1.00
array of 24000 items push then pop 73.251 ns/op 79.228 ns/op 0.92
LinkedList of 24000 items push then pop 8.4270 ns/op 8.6510 ns/op 0.97
intersect bitArray bitLen 8 15.139 ns/op 13.050 ns/op 1.16
intersect array and set length 8 77.869 ns/op 79.769 ns/op 0.98
intersect bitArray bitLen 128 44.022 ns/op 43.749 ns/op 1.01
intersect array and set length 128 1.0423 us/op 1.0792 us/op 0.97
Buffer.concat 32 items 2.9090 us/op 2.9040 us/op 1.00
Uint8Array.set 32 items 2.5420 us/op 2.4190 us/op 1.05
pass gossip attestations to forkchoice per slot 2.7428 ms/op 2.9034 ms/op 0.94
computeDeltas 3.3506 ms/op 3.0124 ms/op 1.11
computeProposerBoostScoreFromBalances 1.7631 ms/op 1.7943 ms/op 0.98
altair processAttestation - 250000 vs - 7PWei normalcase 2.0916 ms/op 2.1402 ms/op 0.98
altair processAttestation - 250000 vs - 7PWei worstcase 3.2511 ms/op 3.1787 ms/op 1.02
altair processAttestation - setStatus - 1/6 committees join 136.69 us/op 135.54 us/op 1.01
altair processAttestation - setStatus - 1/3 committees join 273.55 us/op 277.31 us/op 0.99
altair processAttestation - setStatus - 1/2 committees join 361.46 us/op 371.17 us/op 0.97
altair processAttestation - setStatus - 2/3 committees join 461.30 us/op 465.17 us/op 0.99
altair processAttestation - setStatus - 4/5 committees join 649.43 us/op 662.06 us/op 0.98
altair processAttestation - setStatus - 100% committees join 756.63 us/op 756.11 us/op 1.00
altair processBlock - 250000 vs - 7PWei normalcase 16.682 ms/op 15.003 ms/op 1.11
altair processBlock - 250000 vs - 7PWei normalcase hashState 26.431 ms/op 25.604 ms/op 1.03
altair processBlock - 250000 vs - 7PWei worstcase 48.576 ms/op 49.147 ms/op 0.99
altair processBlock - 250000 vs - 7PWei worstcase hashState 68.372 ms/op 64.380 ms/op 1.06
phase0 processBlock - 250000 vs - 7PWei normalcase 2.0137 ms/op 1.9982 ms/op 1.01
phase0 processBlock - 250000 vs - 7PWei worstcase 28.745 ms/op 27.870 ms/op 1.03
altair processEth1Data - 250000 vs - 7PWei normalcase 466.85 us/op 483.00 us/op 0.97
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.6720 us/op 7.1310 us/op 0.94
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 21.716 us/op 19.424 us/op 1.12
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 10.056 us/op 8.5120 us/op 1.18
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 7.4460 us/op 6.6320 us/op 1.12
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 91.191 us/op 75.023 us/op 1.22
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 666.70 us/op 603.64 us/op 1.10
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 923.97 us/op 882.96 us/op 1.05
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 905.96 us/op 909.72 us/op 1.00
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.3639 ms/op 2.3261 ms/op 1.02
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.5619 ms/op 1.4965 ms/op 1.04
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 3.8938 ms/op 3.7600 ms/op 1.04
Tree 40 250000 create 301.55 ms/op 306.37 ms/op 0.98
Tree 40 250000 get(125000) 187.99 ns/op 173.79 ns/op 1.08
Tree 40 250000 set(125000) 957.72 ns/op 903.24 ns/op 1.06
Tree 40 250000 toArray() 22.750 ms/op 16.536 ms/op 1.38
Tree 40 250000 iterate all - toArray() + loop 23.710 ms/op 16.625 ms/op 1.43
Tree 40 250000 iterate all - get(i) 74.400 ms/op 64.502 ms/op 1.15
MutableVector 250000 create 10.470 ms/op 9.4896 ms/op 1.10
MutableVector 250000 get(125000) 6.4490 ns/op 6.0780 ns/op 1.06
MutableVector 250000 set(125000) 271.91 ns/op 242.83 ns/op 1.12
MutableVector 250000 toArray() 2.9239 ms/op 2.6414 ms/op 1.11
MutableVector 250000 iterate all - toArray() + loop 3.1021 ms/op 2.7396 ms/op 1.13
MutableVector 250000 iterate all - get(i) 1.5243 ms/op 1.5150 ms/op 1.01
Array 250000 create 2.6749 ms/op 2.8313 ms/op 0.94
Array 250000 clone - spread 1.3520 ms/op 1.0629 ms/op 1.27
Array 250000 get(125000) 0.64200 ns/op 0.54300 ns/op 1.18
Array 250000 set(125000) 0.71700 ns/op 0.60300 ns/op 1.19
Array 250000 iterate all - loop 82.838 us/op 84.285 us/op 0.98
effectiveBalanceIncrements clone Uint8Array 300000 30.913 us/op 26.742 us/op 1.16
effectiveBalanceIncrements clone MutableVector 300000 322.00 ns/op 317.00 ns/op 1.02
effectiveBalanceIncrements rw all Uint8Array 300000 172.50 us/op 168.56 us/op 1.02
effectiveBalanceIncrements rw all MutableVector 300000 78.724 ms/op 76.382 ms/op 1.03
phase0 afterProcessEpoch - 250000 vs - 7PWei 115.46 ms/op 111.15 ms/op 1.04
phase0 beforeProcessEpoch - 250000 vs - 7PWei 35.888 ms/op 41.915 ms/op 0.86
altair processEpoch - mainnet_e81889 308.09 ms/op 296.26 ms/op 1.04
mainnet_e81889 - altair beforeProcessEpoch 50.416 ms/op 62.464 ms/op 0.81
mainnet_e81889 - altair processJustificationAndFinalization 17.929 us/op 16.006 us/op 1.12
mainnet_e81889 - altair processInactivityUpdates 5.5859 ms/op 5.0042 ms/op 1.12
mainnet_e81889 - altair processRewardsAndPenalties 54.540 ms/op 67.920 ms/op 0.80
mainnet_e81889 - altair processRegistryUpdates 2.6400 us/op 2.5090 us/op 1.05
mainnet_e81889 - altair processSlashings 502.00 ns/op 591.00 ns/op 0.85
mainnet_e81889 - altair processEth1DataReset 609.00 ns/op 583.00 ns/op 1.04
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2584 ms/op 1.2439 ms/op 1.01
mainnet_e81889 - altair processSlashingsReset 4.6170 us/op 4.5620 us/op 1.01
mainnet_e81889 - altair processRandaoMixesReset 4.3110 us/op 4.4900 us/op 0.96
mainnet_e81889 - altair processHistoricalRootsUpdate 599.00 ns/op 655.00 ns/op 0.91
mainnet_e81889 - altair processParticipationFlagUpdates 2.7590 us/op 2.4270 us/op 1.14
mainnet_e81889 - altair processSyncCommitteeUpdates 740.00 ns/op 625.00 ns/op 1.18
mainnet_e81889 - altair afterProcessEpoch 130.39 ms/op 114.98 ms/op 1.13
phase0 processEpoch - mainnet_e58758 374.28 ms/op 312.32 ms/op 1.20
mainnet_e58758 - phase0 beforeProcessEpoch 145.79 ms/op 118.51 ms/op 1.23
mainnet_e58758 - phase0 processJustificationAndFinalization 23.486 us/op 14.827 us/op 1.58
mainnet_e58758 - phase0 processRewardsAndPenalties 60.230 ms/op 44.577 ms/op 1.35
mainnet_e58758 - phase0 processRegistryUpdates 10.080 us/op 8.1790 us/op 1.23
mainnet_e58758 - phase0 processSlashings 551.00 ns/op 581.00 ns/op 0.95
mainnet_e58758 - phase0 processEth1DataReset 803.00 ns/op 612.00 ns/op 1.31
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0806 ms/op 955.40 us/op 1.13
mainnet_e58758 - phase0 processSlashingsReset 4.2110 us/op 3.2840 us/op 1.28
mainnet_e58758 - phase0 processRandaoMixesReset 6.9070 us/op 4.4920 us/op 1.54
mainnet_e58758 - phase0 processHistoricalRootsUpdate 792.00 ns/op 697.00 ns/op 1.14
mainnet_e58758 - phase0 processParticipationRecordUpdates 5.1020 us/op 3.9850 us/op 1.28
mainnet_e58758 - phase0 afterProcessEpoch 102.98 ms/op 96.280 ms/op 1.07
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.3113 ms/op 1.2193 ms/op 1.08
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.5569 ms/op 1.4169 ms/op 1.10
altair processInactivityUpdates - 250000 normalcase 26.005 ms/op 22.854 ms/op 1.14
altair processInactivityUpdates - 250000 worstcase 27.050 ms/op 20.531 ms/op 1.32
phase0 processRegistryUpdates - 250000 normalcase 7.7410 us/op 6.5650 us/op 1.18
phase0 processRegistryUpdates - 250000 badcase_full_deposits 268.84 us/op 240.77 us/op 1.12
phase0 processRegistryUpdates - 250000 worstcase 0.5 131.50 ms/op 113.94 ms/op 1.15
altair processRewardsAndPenalties - 250000 normalcase 68.670 ms/op 49.497 ms/op 1.39
altair processRewardsAndPenalties - 250000 worstcase 74.137 ms/op 56.698 ms/op 1.31
phase0 getAttestationDeltas - 250000 normalcase 7.0027 ms/op 6.6432 ms/op 1.05
phase0 getAttestationDeltas - 250000 worstcase 7.1124 ms/op 6.5752 ms/op 1.08
phase0 processSlashings - 250000 worstcase 3.6739 ms/op 3.5372 ms/op 1.04
altair processSyncCommitteeUpdates - 250000 189.89 ms/op 174.96 ms/op 1.09
BeaconState.hashTreeRoot - No change 274.00 ns/op 264.00 ns/op 1.04
BeaconState.hashTreeRoot - 1 full validator 55.517 us/op 51.294 us/op 1.08
BeaconState.hashTreeRoot - 32 full validator 563.32 us/op 522.95 us/op 1.08
BeaconState.hashTreeRoot - 512 full validator 5.1300 ms/op 5.3381 ms/op 0.96
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 63.463 us/op 63.173 us/op 1.00
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 914.76 us/op 846.42 us/op 1.08
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 12.428 ms/op 10.808 ms/op 1.15
BeaconState.hashTreeRoot - 1 balances 50.173 us/op 46.163 us/op 1.09
BeaconState.hashTreeRoot - 32 balances 495.36 us/op 450.75 us/op 1.10
BeaconState.hashTreeRoot - 512 balances 4.6533 ms/op 4.3717 ms/op 1.06
BeaconState.hashTreeRoot - 250000 balances 77.102 ms/op 72.364 ms/op 1.07
aggregationBits - 2048 els - zipIndexesInBitList 17.007 us/op 17.541 us/op 0.97
regular array get 100000 times 43.852 us/op 32.557 us/op 1.35
wrappedArray get 100000 times 34.307 us/op 32.944 us/op 1.04
arrayWithProxy get 100000 times 17.041 ms/op 16.582 ms/op 1.03
ssz.Root.equals 573.00 ns/op 551.00 ns/op 1.04
byteArrayEquals 570.00 ns/op 536.00 ns/op 1.06
shuffle list - 16384 els 7.2735 ms/op 6.8668 ms/op 1.06
shuffle list - 250000 els 103.51 ms/op 100.64 ms/op 1.03
processSlot - 1 slots 8.7670 us/op 8.7380 us/op 1.00
processSlot - 32 slots 1.4316 ms/op 1.3636 ms/op 1.05
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 36.692 ms/op 32.872 ms/op 1.12
getCommitteeAssignments - req 1 vs - 250000 vc 3.0297 ms/op 2.9349 ms/op 1.03
getCommitteeAssignments - req 100 vs - 250000 vc 4.1555 ms/op 4.1426 ms/op 1.00
getCommitteeAssignments - req 1000 vs - 250000 vc 4.4832 ms/op 4.4875 ms/op 1.00
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.7400 ns/op 4.2600 ns/op 1.11
state getBlockRootAtSlot - 250000 vs - 7PWei 568.35 ns/op 765.76 ns/op 0.74
computeProposers - vc 250000 10.974 ms/op 10.812 ms/op 1.02
computeEpochShuffling - vc 250000 107.61 ms/op 103.41 ms/op 1.04
getNextSyncCommittee - vc 250000 179.38 ms/op 177.00 ms/op 1.01
computeSigningRoot for AttestationData 14.150 us/op 14.057 us/op 1.01
hash AttestationData serialized data then Buffer.toString(base64) 2.4904 us/op 2.4827 us/op 1.00
toHexString serialized data 1.1669 us/op 1.1167 us/op 1.04
Buffer.toString(base64) 350.15 ns/op 328.40 ns/op 1.07

by benchmarkbot/action

@wemeetagain wemeetagain marked this pull request as ready for review April 10, 2023 20:50
@wemeetagain wemeetagain requested a review from a team as a code owner April 10, 2023 20:50
Copy link
Contributor

@dapplion dapplion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good direction, nice spin off from the main thread PR to lay some groundwork

/**
* Some attnet subscription functionality is needed to be mirrored on the chain side of the thread boundary to determine if attestations should be processed
*/
export class SimpleAttnetsService {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this necessary? This data will be send from the worker to the main thread? IMO:

  • Main thread receives validator subscription requests via rest API
  • Main thread forward sends those network worker subscription updates
  • Worker tracks internal state on what to process and what to subscribe to
  • Worker emits gossip object to main thread along with a flag "should_process" or "process_before_slot"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also is used when submitting attestations via API, where a process_before_slot is not available

@wemeetagain wemeetagain changed the title Refactor network Move network logic into NetworkCore Apr 18, 2023
@dapplion dapplion changed the title Move network logic into NetworkCore refactor!: move network logic into NetworkCore May 1, 2023
@wemeetagain wemeetagain marked this pull request as draft May 1, 2023 19:14
@wemeetagain
Copy link
Member Author

Converting back to draft
will pull out all major features into smaller PRs before reopening for review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants