Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle skipped blobs side car for empty blobs in blobsSidecarsByRange #4936

Merged
merged 7 commits into from
Dec 27, 2022
Merged

Handle skipped blobs side car for empty blobs in blobsSidecarsByRange #4936

merged 7 commits into from
Dec 27, 2022

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Dec 23, 2022

Handle skipped blobs side car for empty blobs in blobsSidecarsByRange
see:

@g11tech g11tech requested a review from a team as a code owner December 23, 2022 12:14
@github-actions
Copy link
Contributor

github-actions bot commented Dec 23, 2022 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 6648901 Previous: c274d22 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.0114 ms/op 1.2985 ms/op 0.78
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 63.140 us/op 80.480 us/op 0.78
BLS verify - blst-native 2.1792 ms/op 1.8674 ms/op 1.17
BLS verifyMultipleSignatures 3 - blst-native 4.4904 ms/op 3.8210 ms/op 1.18
BLS verifyMultipleSignatures 8 - blst-native 9.7023 ms/op 8.2053 ms/op 1.18
BLS verifyMultipleSignatures 32 - blst-native 35.257 ms/op 29.707 ms/op 1.19
BLS aggregatePubkeys 32 - blst-native 46.348 us/op 39.343 us/op 1.18
BLS aggregatePubkeys 128 - blst-native 182.39 us/op 153.76 us/op 1.19
getAttestationsForBlock 78.801 ms/op 96.110 ms/op 0.82
isKnown best case - 1 super set check 455.00 ns/op 430.00 ns/op 1.06
isKnown normal case - 2 super set checks 448.00 ns/op 424.00 ns/op 1.06
isKnown worse case - 16 super set checks 451.00 ns/op 420.00 ns/op 1.07
CheckpointStateCache - add get delete 8.7390 us/op 9.0110 us/op 0.97
validate gossip signedAggregateAndProof - struct 5.0216 ms/op 4.2793 ms/op 1.17
validate gossip attestation - struct 2.3678 ms/op 2.0294 ms/op 1.17
pickEth1Vote - no votes 2.1839 ms/op 2.1408 ms/op 1.02
pickEth1Vote - max votes 16.220 ms/op 18.183 ms/op 0.89
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.217 ms/op 12.541 ms/op 1.05
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 20.958 ms/op 22.471 ms/op 0.93
pickEth1Vote - Eth1Data fastSerialize value x2048 1.0939 ms/op 1.2473 ms/op 0.88
pickEth1Vote - Eth1Data fastSerialize tree x2048 10.386 ms/op 10.826 ms/op 0.96
bytes32 toHexString 940.00 ns/op 1.1700 us/op 0.80
bytes32 Buffer.toString(hex) 593.00 ns/op 605.00 ns/op 0.98
bytes32 Buffer.toString(hex) from Uint8Array 985.00 ns/op 910.00 ns/op 1.08
bytes32 Buffer.toString(hex) + 0x 579.00 ns/op 606.00 ns/op 0.96
Object access 1 prop 0.28000 ns/op 0.28300 ns/op 0.99
Map access 1 prop 0.25600 ns/op 0.25000 ns/op 1.02
Object get x1000 11.572 ns/op 16.592 ns/op 0.70
Map get x1000 0.93000 ns/op 0.99200 ns/op 0.94
Object set x1000 74.093 ns/op 120.45 ns/op 0.62
Map set x1000 44.826 ns/op 73.961 ns/op 0.61
Return object 10000 times 0.43490 ns/op 0.36460 ns/op 1.19
Throw Error 10000 times 6.5895 us/op 6.7466 us/op 0.98
fastMsgIdFn sha256 / 200 bytes 4.8530 us/op 4.2510 us/op 1.14
fastMsgIdFn h32 xxhash / 200 bytes 458.00 ns/op 468.00 ns/op 0.98
fastMsgIdFn h64 xxhash / 200 bytes 646.00 ns/op 639.00 ns/op 1.01
fastMsgIdFn sha256 / 1000 bytes 15.756 us/op 13.565 us/op 1.16
fastMsgIdFn h32 xxhash / 1000 bytes 627.00 ns/op 617.00 ns/op 1.02
fastMsgIdFn h64 xxhash / 1000 bytes 725.00 ns/op 739.00 ns/op 0.98
fastMsgIdFn sha256 / 10000 bytes 137.36 us/op 117.57 us/op 1.17
fastMsgIdFn h32 xxhash / 10000 bytes 2.5460 us/op 2.3290 us/op 1.09
fastMsgIdFn h64 xxhash / 10000 bytes 1.6840 us/op 1.7000 us/op 0.99
enrSubnets - fastDeserialize 64 bits 2.2570 us/op 2.6730 us/op 0.84
enrSubnets - ssz BitVector 64 bits 827.00 ns/op 865.00 ns/op 0.96
enrSubnets - fastDeserialize 4 bits 290.00 ns/op 298.00 ns/op 0.97
enrSubnets - ssz BitVector 4 bits 924.00 ns/op 864.00 ns/op 1.07
prioritizePeers score -10:0 att 32-0.1 sync 2-0 150.42 us/op 151.49 us/op 0.99
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 163.86 us/op 218.27 us/op 0.75
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 257.94 us/op 326.21 us/op 0.79
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 442.31 us/op 556.72 us/op 0.79
prioritizePeers score 0:0 att 64-1 sync 4-1 501.98 us/op 629.79 us/op 0.80
array of 16000 items push then shift 51.673 us/op 2.8135 us/op 18.37
LinkedList of 16000 items push then shift 12.392 ns/op 19.472 ns/op 0.64
array of 16000 items push then pop 189.32 ns/op 224.73 ns/op 0.84
LinkedList of 16000 items push then pop 12.117 ns/op 17.962 ns/op 0.67
array of 24000 items push then shift 77.403 us/op 3.9694 us/op 19.50
LinkedList of 24000 items push then shift 13.068 ns/op 23.997 ns/op 0.54
array of 24000 items push then pop 197.11 ns/op 216.21 ns/op 0.91
LinkedList of 24000 items push then pop 12.166 ns/op 19.835 ns/op 0.61
intersect bitArray bitLen 8 21.673 ns/op 29.418 ns/op 0.74
intersect array and set length 8 127.06 ns/op 180.48 ns/op 0.70
intersect bitArray bitLen 128 71.299 ns/op 108.35 ns/op 0.66
intersect array and set length 128 1.7478 us/op 2.3818 us/op 0.73
Buffer.concat 32 items 4.4460 us/op 5.2220 us/op 0.85
Uint8Array.set 32 items 3.4700 us/op 4.1180 us/op 0.84
pass gossip attestations to forkchoice per slot 3.7520 ms/op 4.3641 ms/op 0.86
computeDeltas 4.2581 ms/op 6.1605 ms/op 0.69
computeProposerBoostScoreFromBalances 2.4633 ms/op 2.8484 ms/op 0.86
altair processAttestation - 250000 vs - 7PWei normalcase 3.5793 ms/op 4.8001 ms/op 0.75
altair processAttestation - 250000 vs - 7PWei worstcase 6.1681 ms/op 8.2241 ms/op 0.75
altair processAttestation - setStatus - 1/6 committees join 178.15 us/op 211.17 us/op 0.84
altair processAttestation - setStatus - 1/3 committees join 358.40 us/op 400.57 us/op 0.89
altair processAttestation - setStatus - 1/2 committees join 522.86 us/op 565.35 us/op 0.92
altair processAttestation - setStatus - 2/3 committees join 673.56 us/op 727.91 us/op 0.93
altair processAttestation - setStatus - 4/5 committees join 943.44 us/op 1.0033 ms/op 0.94
altair processAttestation - setStatus - 100% committees join 1.1260 ms/op 1.1887 ms/op 0.95
altair processBlock - 250000 vs - 7PWei normalcase 25.487 ms/op 27.880 ms/op 0.91
altair processBlock - 250000 vs - 7PWei normalcase hashState 34.853 ms/op 39.250 ms/op 0.89
altair processBlock - 250000 vs - 7PWei worstcase 74.626 ms/op 79.452 ms/op 0.94
altair processBlock - 250000 vs - 7PWei worstcase hashState 112.58 ms/op 97.533 ms/op 1.15
phase0 processBlock - 250000 vs - 7PWei normalcase 3.2681 ms/op 3.7387 ms/op 0.87
phase0 processBlock - 250000 vs - 7PWei worstcase 51.754 ms/op 47.371 ms/op 1.09
altair processEth1Data - 250000 vs - 7PWei normalcase 697.22 us/op 897.33 us/op 0.78
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 8.0230 us/op 9.0390 us/op 0.89
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 23.283 us/op 28.951 us/op 0.80
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 11.381 us/op 12.161 us/op 0.94
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 7.7660 us/op 8.9860 us/op 0.86
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 91.284 us/op 100.24 us/op 0.91
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 897.07 us/op 1.0169 ms/op 0.88
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 1.1954 ms/op 1.3690 ms/op 0.87
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 1.1529 ms/op 1.3260 ms/op 0.87
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 4.0761 ms/op 4.5219 ms/op 0.90
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 2.6033 ms/op 2.8656 ms/op 0.91
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 6.8063 ms/op 7.5943 ms/op 0.90
Tree 40 250000 create 656.61 ms/op 899.97 ms/op 0.73
Tree 40 250000 get(125000) 228.40 ns/op 285.21 ns/op 0.80
Tree 40 250000 set(125000) 2.1046 us/op 2.4731 us/op 0.85
Tree 40 250000 toArray() 27.220 ms/op 32.734 ms/op 0.83
Tree 40 250000 iterate all - toArray() + loop 27.525 ms/op 32.811 ms/op 0.84
Tree 40 250000 iterate all - get(i) 112.76 ms/op 107.01 ms/op 1.05
MutableVector 250000 create 14.729 ms/op 15.604 ms/op 0.94
MutableVector 250000 get(125000) 10.591 ns/op 13.168 ns/op 0.80
MutableVector 250000 set(125000) 527.99 ns/op 672.44 ns/op 0.79
MutableVector 250000 toArray() 5.9906 ms/op 6.7847 ms/op 0.88
MutableVector 250000 iterate all - toArray() + loop 6.1214 ms/op 6.9557 ms/op 0.88
MutableVector 250000 iterate all - get(i) 2.8341 ms/op 3.4778 ms/op 0.81
Array 250000 create 5.9366 ms/op 6.6489 ms/op 0.89
Array 250000 clone - spread 2.5959 ms/op 2.6796 ms/op 0.97
Array 250000 get(125000) 1.2320 ns/op 1.1460 ns/op 1.08
Array 250000 set(125000) 1.2290 ns/op 1.1200 ns/op 1.10
Array 250000 iterate all - loop 150.92 us/op 167.75 us/op 0.90
effectiveBalanceIncrements clone Uint8Array 300000 48.869 us/op 84.185 us/op 0.58
effectiveBalanceIncrements clone MutableVector 300000 800.00 ns/op 754.00 ns/op 1.06
effectiveBalanceIncrements rw all Uint8Array 300000 243.94 us/op 252.73 us/op 0.97
effectiveBalanceIncrements rw all MutableVector 300000 164.06 ms/op 215.92 ms/op 0.76
phase0 afterProcessEpoch - 250000 vs - 7PWei 199.54 ms/op 187.62 ms/op 1.06
phase0 beforeProcessEpoch - 250000 vs - 7PWei 57.370 ms/op 112.16 ms/op 0.51
altair processEpoch - mainnet_e81889 533.75 ms/op 592.08 ms/op 0.90
mainnet_e81889 - altair beforeProcessEpoch 110.71 ms/op 153.78 ms/op 0.72
mainnet_e81889 - altair processJustificationAndFinalization 20.627 us/op 20.871 us/op 0.99
mainnet_e81889 - altair processInactivityUpdates 9.0956 ms/op 11.773 ms/op 0.77
mainnet_e81889 - altair processRewardsAndPenalties 77.404 ms/op 95.096 ms/op 0.81
mainnet_e81889 - altair processRegistryUpdates 2.9390 us/op 2.8440 us/op 1.03
mainnet_e81889 - altair processSlashings 512.00 ns/op 701.00 ns/op 0.73
mainnet_e81889 - altair processEth1DataReset 805.00 ns/op 680.00 ns/op 1.18
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.9805 ms/op 2.4104 ms/op 0.82
mainnet_e81889 - altair processSlashingsReset 4.6040 us/op 4.7110 us/op 0.98
mainnet_e81889 - altair processRandaoMixesReset 6.0760 us/op 5.5180 us/op 1.10
mainnet_e81889 - altair processHistoricalRootsUpdate 643.00 ns/op 845.00 ns/op 0.76
mainnet_e81889 - altair processParticipationFlagUpdates 3.2550 us/op 2.6220 us/op 1.24
mainnet_e81889 - altair processSyncCommitteeUpdates 513.00 ns/op 714.00 ns/op 0.72
mainnet_e81889 - altair afterProcessEpoch 196.98 ms/op 195.16 ms/op 1.01
phase0 processEpoch - mainnet_e58758 480.28 ms/op 547.39 ms/op 0.88
mainnet_e58758 - phase0 beforeProcessEpoch 179.65 ms/op 254.30 ms/op 0.71
mainnet_e58758 - phase0 processJustificationAndFinalization 17.721 us/op 22.529 us/op 0.79
mainnet_e58758 - phase0 processRewardsAndPenalties 99.925 ms/op 142.03 ms/op 0.70
mainnet_e58758 - phase0 processRegistryUpdates 7.6690 us/op 9.3950 us/op 0.82
mainnet_e58758 - phase0 processSlashings 868.00 ns/op 634.00 ns/op 1.37
mainnet_e58758 - phase0 processEth1DataReset 783.00 ns/op 618.00 ns/op 1.27
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.6553 ms/op 1.9688 ms/op 0.84
mainnet_e58758 - phase0 processSlashingsReset 5.1630 us/op 4.6310 us/op 1.11
mainnet_e58758 - phase0 processRandaoMixesReset 5.2330 us/op 5.4380 us/op 0.96
mainnet_e58758 - phase0 processHistoricalRootsUpdate 787.00 ns/op 677.00 ns/op 1.16
mainnet_e58758 - phase0 processParticipationRecordUpdates 5.2710 us/op 4.6780 us/op 1.13
mainnet_e58758 - phase0 afterProcessEpoch 162.40 ms/op 160.88 ms/op 1.01
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.9468 ms/op 2.7962 ms/op 0.70
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.1907 ms/op 3.0550 ms/op 0.72
altair processInactivityUpdates - 250000 normalcase 32.695 ms/op 39.745 ms/op 0.82
altair processInactivityUpdates - 250000 worstcase 39.584 ms/op 51.124 ms/op 0.77
phase0 processRegistryUpdates - 250000 normalcase 6.2270 us/op 7.8720 us/op 0.79
phase0 processRegistryUpdates - 250000 badcase_full_deposits 377.71 us/op 433.84 us/op 0.87
phase0 processRegistryUpdates - 250000 worstcase 0.5 181.21 ms/op 227.21 ms/op 0.80
altair processRewardsAndPenalties - 250000 normalcase 107.59 ms/op 133.27 ms/op 0.81
altair processRewardsAndPenalties - 250000 worstcase 74.285 ms/op 119.71 ms/op 0.62
phase0 getAttestationDeltas - 250000 normalcase 12.093 ms/op 13.963 ms/op 0.87
phase0 getAttestationDeltas - 250000 worstcase 12.497 ms/op 13.926 ms/op 0.90
phase0 processSlashings - 250000 worstcase 5.2166 ms/op 5.7049 ms/op 0.91
altair processSyncCommitteeUpdates - 250000 290.73 ms/op 274.60 ms/op 1.06
BeaconState.hashTreeRoot - No change 471.00 ns/op 434.00 ns/op 1.09
BeaconState.hashTreeRoot - 1 full validator 73.111 us/op 61.775 us/op 1.18
BeaconState.hashTreeRoot - 32 full validator 896.31 us/op 632.70 us/op 1.42
BeaconState.hashTreeRoot - 512 full validator 7.2711 ms/op 7.0442 ms/op 1.03
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 89.350 us/op 79.028 us/op 1.13
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2542 ms/op 1.2008 ms/op 1.04
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.846 ms/op 16.916 ms/op 1.00
BeaconState.hashTreeRoot - 1 balances 69.980 us/op 86.493 us/op 0.81
BeaconState.hashTreeRoot - 32 balances 643.46 us/op 550.99 us/op 1.17
BeaconState.hashTreeRoot - 512 balances 6.1917 ms/op 5.4006 ms/op 1.15
BeaconState.hashTreeRoot - 250000 balances 100.26 ms/op 86.618 ms/op 1.16
aggregationBits - 2048 els - zipIndexesInBitList 26.671 us/op 37.728 us/op 0.71
regular array get 100000 times 60.537 us/op 67.388 us/op 0.90
wrappedArray get 100000 times 60.544 us/op 67.378 us/op 0.90
arrayWithProxy get 100000 times 26.227 ms/op 28.211 ms/op 0.93
ssz.Root.equals 949.00 ns/op 1.0920 us/op 0.87
byteArrayEquals 947.00 ns/op 1.1090 us/op 0.85
shuffle list - 16384 els 11.497 ms/op 11.309 ms/op 1.02
shuffle list - 250000 els 169.04 ms/op 166.71 ms/op 1.01
processSlot - 1 slots 14.499 us/op 12.453 us/op 1.16
processSlot - 32 slots 1.9637 ms/op 2.0381 ms/op 0.96
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 318.66 us/op 337.85 us/op 0.94
getCommitteeAssignments - req 1 vs - 250000 vc 5.4549 ms/op 5.6226 ms/op 0.97
getCommitteeAssignments - req 100 vs - 250000 vc 7.9508 ms/op 7.6220 ms/op 1.04
getCommitteeAssignments - req 1000 vs - 250000 vc 8.5007 ms/op 8.1483 ms/op 1.04
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 7.8700 ns/op 9.8000 ns/op 0.80
state getBlockRootAtSlot - 250000 vs - 7PWei 1.1508 us/op 1.3281 us/op 0.87
computeProposers - vc 250000 17.267 ms/op 16.475 ms/op 1.05
computeEpochShuffling - vc 250000 171.11 ms/op 171.03 ms/op 1.00
getNextSyncCommittee - vc 250000 289.58 ms/op 274.77 ms/op 1.05

by benchmarkbot/action

@g11tech g11tech enabled auto-merge (squash) December 23, 2022 12:56
dapplion
dapplion reviewed Dec 24, 2022
View reviewed changes
Copy link
Contributor

@dapplion dapplion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that this function is growing in complexity would make sense to move it to a file with signature

function beaconBlocksMaybeBlobsByRange(config, reqResp, peerId, request) {}

And unit test it by injecting a mock reqResp that resolves with predefined data

@g11tech g11tech merged commit 06b4d17 into unstable Dec 27, 2022
@g11tech g11tech deleted the g11tech/handle-empty-blobs branch December 27, 2022 03:08
dapplion
dapplion reviewed Dec 27, 2022
View reviewed changes
packages/beacon-node/test/unit/network/doBeaconBlocksMaybeBlobsByRange.ts
import {doBeaconBlocksMaybeBlobsByRange, ReqRespBeaconNode} from "../../../src/network/reqresp/index.js";
import {BlockInputType} from "../../../src/chain/blocks/types.js";

describe("doBeaconBlocksMaybeBlobsByRange", function () {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please always append .test.ts extension to test files

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh thats my bad, here is the fix: #4949

@twoeths twoeths mentioned this pull request Dec 29, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion left review comments

@twoeths twoeths twoeths approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@g11tech @twoeths @dapplion