Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

msgIdFn: use Buffer.subarray() #4342

Merged
merged 2 commits into from Aug 1, 2022
Merged

msgIdFn: use Buffer.subarray() #4342

merged 2 commits into from Aug 1, 2022

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Jul 24, 2022 •
edited

Motivation

Buffer.slice() only create a view of the Buffer instead of creating a copy so we should use it instead of Uint8Array.slice()

Description

  • Use Buffer.slice() in msgIdFn function like other places

@twoeths twoeths requested a review from a team as a code owner July 24, 2022 08:07
@github-actions
Copy link
Contributor

github-actions bot commented Jul 24, 2022 •
edited

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 8df4a5c Previous: 384e393 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.0881 ms/op 2.8828 ms/op 0.72
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 73.287 us/op 81.568 us/op 0.90
BLS verify - blst-native 1.8545 ms/op 2.2293 ms/op 0.83
BLS verifyMultipleSignatures 3 - blst-native 3.8008 ms/op 4.4521 ms/op 0.85
BLS verifyMultipleSignatures 8 - blst-native 8.1810 ms/op 9.8764 ms/op 0.83
BLS verifyMultipleSignatures 32 - blst-native 29.677 ms/op 35.315 ms/op 0.84
BLS aggregatePubkeys 32 - blst-native 39.705 us/op 49.462 us/op 0.80
BLS aggregatePubkeys 128 - blst-native 152.47 us/op 192.07 us/op 0.79
getAttestationsForBlock 45.044 ms/op 55.350 ms/op 0.81
isKnown best case - 1 super set check 438.00 ns/op 523.00 ns/op 0.84
isKnown normal case - 2 super set checks 425.00 ns/op 479.00 ns/op 0.89
isKnown worse case - 16 super set checks 421.00 ns/op 490.00 ns/op 0.86
CheckpointStateCache - add get delete 8.7000 us/op 11.116 us/op 0.78
validate gossip signedAggregateAndProof - struct 4.2524 ms/op 5.2620 ms/op 0.81
validate gossip attestation - struct 2.0305 ms/op 2.4885 ms/op 0.82
altair verifyImport mainnet_s3766816:31 8.5642 s/op 10.238 s/op 0.84
pickEth1Vote - no votes 2.0481 ms/op 2.9210 ms/op 0.70
pickEth1Vote - max votes 21.659 ms/op 29.721 ms/op 0.73
pickEth1Vote - Eth1Data hashTreeRoot value x2048 10.999 ms/op 16.320 ms/op 0.67
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 19.771 ms/op 27.585 ms/op 0.72
pickEth1Vote - Eth1Data fastSerialize value x2048 1.5627 ms/op 1.7245 ms/op 0.91
pickEth1Vote - Eth1Data fastSerialize tree x2048 15.159 ms/op 15.024 ms/op 1.01
bytes32 toHexString 1.0360 us/op 1.1020 us/op 0.94
bytes32 Buffer.toString(hex) 691.00 ns/op 733.00 ns/op 0.94
bytes32 Buffer.toString(hex) from Uint8Array 917.00 ns/op 1.0670 us/op 0.86
bytes32 Buffer.toString(hex) + 0x 706.00 ns/op 782.00 ns/op 0.90
Object access 1 prop 0.36200 ns/op 0.38400 ns/op 0.94
Map access 1 prop 0.29700 ns/op 0.34000 ns/op 0.87
Object get x1000 17.634 ns/op 17.175 ns/op 1.03
Map get x1000 1.0330 ns/op 0.99200 ns/op 1.04
Object set x1000 116.44 ns/op 113.91 ns/op 1.02
Map set x1000 69.098 ns/op 82.725 ns/op 0.84
Return object 10000 times 0.37650 ns/op 0.41100 ns/op 0.92
Throw Error 10000 times 5.9554 us/op 7.2631 us/op 0.82
enrSubnets - fastDeserialize 64 bits 2.7070 us/op 3.1140 us/op 0.87
enrSubnets - ssz BitVector 64 bits 729.00 ns/op 834.00 ns/op 0.87
enrSubnets - fastDeserialize 4 bits 393.00 ns/op 435.00 ns/op 0.90
enrSubnets - ssz BitVector 4 bits 730.00 ns/op 864.00 ns/op 0.84
prioritizePeers score -10:0 att 32-0.1 sync 2-0 92.818 us/op 110.35 us/op 0.84
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 131.17 us/op 165.86 us/op 0.79
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 242.03 us/op 280.50 us/op 0.86
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 462.10 us/op 521.65 us/op 0.89
prioritizePeers score 0:0 att 64-1 sync 4-1 463.98 us/op 559.15 us/op 0.83
RateTracker 1000000 limit, 1 obj count per request 200.19 ns/op 199.88 ns/op 1.00
RateTracker 1000000 limit, 2 obj count per request 148.07 ns/op 151.34 ns/op 0.98
RateTracker 1000000 limit, 4 obj count per request 124.99 ns/op 119.00 ns/op 1.05
RateTracker 1000000 limit, 8 obj count per request 111.84 ns/op 104.73 ns/op 1.07
RateTracker with prune 4.5730 us/op 4.8840 us/op 0.94
array of 16000 items push then shift 3.1966 us/op 5.2348 us/op 0.61
LinkedList of 16000 items push then shift 26.763 ns/op 20.044 ns/op 1.34
array of 16000 items push then pop 259.87 ns/op 266.99 ns/op 0.97
LinkedList of 16000 items push then pop 22.245 ns/op 18.615 ns/op 1.20
array of 24000 items push then shift 4.5678 us/op 8.2401 us/op 0.55
LinkedList of 24000 items push then shift 29.646 ns/op 20.388 ns/op 1.45
array of 24000 items push then pop 199.35 ns/op 243.90 ns/op 0.82
LinkedList of 24000 items push then pop 22.812 ns/op 19.105 ns/op 1.19
intersect bitArray bitLen 8 11.831 ns/op 12.283 ns/op 0.96
intersect array and set length 8 166.91 ns/op 190.87 ns/op 0.87
intersect bitArray bitLen 128 72.025 ns/op 72.722 ns/op 0.99
intersect array and set length 128 2.3548 us/op 2.4721 us/op 0.95
pass gossip attestations to forkchoice per slot 5.4006 ms/op 6.6087 ms/op 0.82
computeDeltas 4.1404 ms/op 4.0003 ms/op 1.04
computeProposerBoostScoreFromBalances 921.00 us/op 894.73 us/op 1.03
altair processAttestation - 250000 vs - 7PWei normalcase 3.8714 ms/op 5.6909 ms/op 0.68
altair processAttestation - 250000 vs - 7PWei worstcase 5.8369 ms/op 7.6782 ms/op 0.76
altair processAttestation - setStatus - 1/6 committees join 212.94 us/op 262.74 us/op 0.81
altair processAttestation - setStatus - 1/3 committees join 401.88 us/op 487.38 us/op 0.82
altair processAttestation - setStatus - 1/2 committees join 562.78 us/op 671.95 us/op 0.84
altair processAttestation - setStatus - 2/3 committees join 722.73 us/op 913.98 us/op 0.79
altair processAttestation - setStatus - 4/5 committees join 999.47 us/op 1.2729 ms/op 0.79
altair processAttestation - setStatus - 100% committees join 1.1853 ms/op 1.5896 ms/op 0.75
altair processBlock - 250000 vs - 7PWei normalcase 28.133 ms/op 31.893 ms/op 0.88
altair processBlock - 250000 vs - 7PWei normalcase hashState 42.990 ms/op 42.038 ms/op 1.02
altair processBlock - 250000 vs - 7PWei worstcase 81.900 ms/op 105.11 ms/op 0.78
altair processBlock - 250000 vs - 7PWei worstcase hashState 98.223 ms/op 117.15 ms/op 0.84
phase0 processBlock - 250000 vs - 7PWei normalcase 4.5424 ms/op 4.5615 ms/op 1.00
phase0 processBlock - 250000 vs - 7PWei worstcase 46.540 ms/op 52.987 ms/op 0.88
altair processEth1Data - 250000 vs - 7PWei normalcase 776.89 us/op 1.0712 ms/op 0.73
Tree 40 250000 create 836.77 ms/op 968.34 ms/op 0.86
Tree 40 250000 get(125000) 282.48 ns/op 336.56 ns/op 0.84
Tree 40 250000 set(125000) 2.3923 us/op 3.0896 us/op 0.77
Tree 40 250000 toArray() 32.681 ms/op 37.292 ms/op 0.88
Tree 40 250000 iterate all - toArray() + loop 32.755 ms/op 37.712 ms/op 0.87
Tree 40 250000 iterate all - get(i) 110.99 ms/op 130.13 ms/op 0.85
MutableVector 250000 create 14.592 ms/op 18.240 ms/op 0.80
MutableVector 250000 get(125000) 13.288 ns/op 14.156 ns/op 0.94
MutableVector 250000 set(125000) 606.96 ns/op 897.93 ns/op 0.68
MutableVector 250000 toArray() 7.1680 ms/op 7.7273 ms/op 0.93
MutableVector 250000 iterate all - toArray() + loop 7.3828 ms/op 8.0970 ms/op 0.91
MutableVector 250000 iterate all - get(i) 3.3024 ms/op 3.2048 ms/op 1.03
Array 250000 create 6.4064 ms/op 7.9757 ms/op 0.80
Array 250000 clone - spread 3.4620 ms/op 2.8895 ms/op 1.20
Array 250000 get(125000) 1.4570 ns/op 1.2580 ns/op 1.16
Array 250000 set(125000) 1.4500 ns/op 1.2430 ns/op 1.17
Array 250000 iterate all - loop 169.50 us/op 139.38 us/op 1.22
effectiveBalanceIncrements clone Uint8Array 300000 82.301 us/op 216.45 us/op 0.38
effectiveBalanceIncrements clone MutableVector 300000 1.0860 us/op 721.00 ns/op 1.51
effectiveBalanceIncrements rw all Uint8Array 300000 252.47 us/op 293.97 us/op 0.86
effectiveBalanceIncrements rw all MutableVector 300000 200.02 ms/op 204.79 ms/op 0.98
phase0 afterProcessEpoch - 250000 vs - 7PWei 205.78 ms/op 211.86 ms/op 0.97
phase0 beforeProcessEpoch - 250000 vs - 7PWei 76.259 ms/op 82.238 ms/op 0.93
altair processEpoch - mainnet_e81889 586.73 ms/op 630.95 ms/op 0.93
mainnet_e81889 - altair beforeProcessEpoch 154.21 ms/op 158.28 ms/op 0.97
mainnet_e81889 - altair processJustificationAndFinalization 32.029 us/op 70.670 us/op 0.45
mainnet_e81889 - altair processInactivityUpdates 11.545 ms/op 10.521 ms/op 1.10
mainnet_e81889 - altair processRewardsAndPenalties 94.734 ms/op 104.71 ms/op 0.90
mainnet_e81889 - altair processRegistryUpdates 5.8680 us/op 15.191 us/op 0.39
mainnet_e81889 - altair processSlashings 1.2650 us/op 4.3670 us/op 0.29
mainnet_e81889 - altair processEth1DataReset 1.2980 us/op 4.3470 us/op 0.30
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.5184 ms/op 2.3716 ms/op 1.06
mainnet_e81889 - altair processSlashingsReset 7.7520 us/op 22.734 us/op 0.34
mainnet_e81889 - altair processRandaoMixesReset 9.2120 us/op 25.690 us/op 0.36
mainnet_e81889 - altair processHistoricalRootsUpdate 1.0520 us/op 4.3290 us/op 0.24
mainnet_e81889 - altair processParticipationFlagUpdates 3.4900 us/op 15.913 us/op 0.22
mainnet_e81889 - altair processSyncCommitteeUpdates 921.00 ns/op 3.6310 us/op 0.25
mainnet_e81889 - altair afterProcessEpoch 196.51 ms/op 201.44 ms/op 0.98
phase0 processEpoch - mainnet_e58758 534.42 ms/op 624.05 ms/op 0.86
mainnet_e58758 - phase0 beforeProcessEpoch 240.51 ms/op 265.70 ms/op 0.91
mainnet_e58758 - phase0 processJustificationAndFinalization 35.645 us/op 66.935 us/op 0.53
mainnet_e58758 - phase0 processRewardsAndPenalties 118.21 ms/op 155.07 ms/op 0.76
mainnet_e58758 - phase0 processRegistryUpdates 11.415 us/op 35.146 us/op 0.32
mainnet_e58758 - phase0 processSlashings 758.00 ns/op 3.5030 us/op 0.22
mainnet_e58758 - phase0 processEth1DataReset 1.3140 us/op 3.7080 us/op 0.35
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.4222 ms/op 2.1679 ms/op 1.12
mainnet_e58758 - phase0 processSlashingsReset 7.5260 us/op 21.282 us/op 0.35
mainnet_e58758 - phase0 processRandaoMixesReset 8.5020 us/op 24.594 us/op 0.35
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.3740 us/op 4.5950 us/op 0.30
mainnet_e58758 - phase0 processParticipationRecordUpdates 7.3590 us/op 23.404 us/op 0.31
mainnet_e58758 - phase0 afterProcessEpoch 160.71 ms/op 155.98 ms/op 1.03
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.6233 ms/op 2.3112 ms/op 1.14
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.1351 ms/op 2.5242 ms/op 1.24
altair processInactivityUpdates - 250000 normalcase 40.308 ms/op 51.310 ms/op 0.79
altair processInactivityUpdates - 250000 worstcase 40.969 ms/op 56.507 ms/op 0.73
phase0 processRegistryUpdates - 250000 normalcase 11.915 us/op 28.899 us/op 0.41
phase0 processRegistryUpdates - 250000 badcase_full_deposits 485.33 us/op 496.46 us/op 0.98
phase0 processRegistryUpdates - 250000 worstcase 0.5 212.62 ms/op 240.27 ms/op 0.88
altair processRewardsAndPenalties - 250000 normalcase 90.550 ms/op 101.91 ms/op 0.89
altair processRewardsAndPenalties - 250000 worstcase 143.01 ms/op 145.73 ms/op 0.98
phase0 getAttestationDeltas - 250000 normalcase 12.535 ms/op 13.573 ms/op 0.92
phase0 getAttestationDeltas - 250000 worstcase 12.619 ms/op 13.475 ms/op 0.94
phase0 processSlashings - 250000 worstcase 5.4819 ms/op 5.8971 ms/op 0.93
altair processSyncCommitteeUpdates - 250000 274.51 ms/op 344.78 ms/op 0.80
BeaconState.hashTreeRoot - No change 481.00 ns/op 596.00 ns/op 0.81
BeaconState.hashTreeRoot - 1 full validator 61.218 us/op 78.491 us/op 0.78
BeaconState.hashTreeRoot - 32 full validator 631.04 us/op 776.71 us/op 0.81
BeaconState.hashTreeRoot - 512 full validator 5.9761 ms/op 7.0313 ms/op 0.85
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 78.605 us/op 102.21 us/op 0.77
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.1711 ms/op 1.4812 ms/op 0.79
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 15.575 ms/op 17.853 ms/op 0.87
BeaconState.hashTreeRoot - 1 balances 60.120 us/op 69.413 us/op 0.87
BeaconState.hashTreeRoot - 32 balances 559.84 us/op 677.82 us/op 0.83
BeaconState.hashTreeRoot - 512 balances 6.5935 ms/op 6.2203 ms/op 1.06
BeaconState.hashTreeRoot - 250000 balances 89.355 ms/op 114.57 ms/op 0.78
aggregationBits - 2048 els - zipIndexesInBitList 32.051 us/op 27.898 us/op 1.15
regular array get 100000 times 67.448 us/op 55.634 us/op 1.21
wrappedArray get 100000 times 67.401 us/op 54.808 us/op 1.23
arrayWithProxy get 100000 times 29.550 ms/op 37.091 ms/op 0.80
ssz.Root.equals 466.00 ns/op 568.00 ns/op 0.82
byteArrayEquals 466.00 ns/op 560.00 ns/op 0.83
shuffle list - 16384 els 11.322 ms/op 11.826 ms/op 0.96
shuffle list - 250000 els 166.12 ms/op 170.69 ms/op 0.97
processSlot - 1 slots 12.549 us/op 17.823 us/op 0.70
processSlot - 32 slots 1.7427 ms/op 2.3033 ms/op 0.76
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 399.19 us/op 672.58 us/op 0.59
getCommitteeAssignments - req 1 vs - 250000 vc 5.3302 ms/op 5.1302 ms/op 1.04
getCommitteeAssignments - req 100 vs - 250000 vc 7.3234 ms/op 7.7979 ms/op 0.94
getCommitteeAssignments - req 1000 vs - 250000 vc 7.7645 ms/op 8.2319 ms/op 0.94
computeProposers - vc 250000 18.336 ms/op 21.567 ms/op 0.85
computeEpochShuffling - vc 250000 170.12 ms/op 168.20 ms/op 1.01
getNextSyncCommittee - vc 250000 269.53 ms/op 301.88 ms/op 0.89

by benchmarkbot/action

twoeths
twoeths commented Jul 27, 2022
View reviewed changes
packages/beacon-node/src/network/gossip/encoding.ts Outdated
@@ -53,7 +53,7 @@ export function msgIdFn(gossipTopicCache: GossipTopicCache, msg: GossipsubMessag
}
}

return digest(Buffer.concat(vec)).slice(0, 20);
return Buffer.from(digest(Buffer.concat(vec))).slice(0, 20);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Buffer.slice() is deprecated in Node.js 18, should we migrate similar pattern to Uint8Array.subarray()?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

@wemeetagain wemeetagain changed the title msgIdFn: use Buffer.slice() msgIdFn: use Buffer.subarray() Aug 1, 2022
@wemeetagain wemeetagain merged commit 9979836 into unstable Aug 1, 2022
@wemeetagain wemeetagain deleted the tuyen/Buffer-slice branch August 1, 2022 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion left review comments

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@twoeths @wemeetagain @dapplion