Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Skip checking known participants when publishing ContributionAndProof #4097

Merged
merged 2 commits into from
Jun 8, 2022
Merged

Skip checking known participants when publishing ContributionAndProof #4097

merged 2 commits into from
Jun 8, 2022

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Jun 1, 2022

Motivation

We don't want to check for know participants when publishing ContributionAndProof from api

Description

  • Similar to validateGossipAggregateAndProof, add skipValidationKnownParticipants = false param to validateSyncCommitteeGossipContributionAndProof
  • Fix validateGossipAggregateAndProof: wrong check on AGGREGATOR_ALREADY_KNOWN, should be for ATTESTERS_ALREADY_KNOWN

part of #4089

@twoeths twoeths requested a review from a team as a code owner June 1, 2022 03:24
@github-actions
Copy link
Contributor

github-actions bot commented Jun 1, 2022 •
edited

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 4a8ed4a Previous: 2deb6a2 Ratio
altair processAttestation - 250000 vs - 7PWei normalcase 4.3622 ms/op 3.1169 ms/op 1.40
altair processAttestation - 250000 vs - 7PWei worstcase 6.7567 ms/op 5.0592 ms/op 1.34
altair processAttestation - setStatus - 1/6 committees join 223.98 us/op 171.04 us/op 1.31
altair processAttestation - setStatus - 1/3 committees join 433.86 us/op 351.99 us/op 1.23
altair processAttestation - setStatus - 1/2 committees join 590.07 us/op 510.62 us/op 1.16
altair processAttestation - setStatus - 2/3 committees join 793.33 us/op 667.29 us/op 1.19
altair processAttestation - setStatus - 4/5 committees join 1.0639 ms/op 925.04 us/op 1.15
altair processAttestation - setStatus - 100% committees join 1.3146 ms/op 1.1181 ms/op 1.18
altair processBlock - 250000 vs - 7PWei normalcase 31.301 ms/op 21.138 ms/op 1.48
altair processBlock - 250000 vs - 7PWei normalcase hashState 37.941 ms/op 31.711 ms/op 1.20
altair processBlock - 250000 vs - 7PWei worstcase 100.06 ms/op 78.671 ms/op 1.27
altair processBlock - 250000 vs - 7PWei worstcase hashState 123.75 ms/op 95.919 ms/op 1.29
phase0 processBlock - 250000 vs - 7PWei normalcase 4.0373 ms/op 3.1141 ms/op 1.30
phase0 processBlock - 250000 vs - 7PWei worstcase 62.341 ms/op 49.717 ms/op 1.25
altair processEth1Data - 250000 vs - 7PWei normalcase 825.82 us/op 639.51 us/op 1.29
Tree 40 250000 create 963.09 ms/op 674.28 ms/op 1.43
Tree 40 250000 get(125000) 329.23 ns/op 262.73 ns/op 1.25
Tree 40 250000 set(125000) 2.9432 us/op 2.0852 us/op 1.41
Tree 40 250000 toArray() 34.455 ms/op 25.250 ms/op 1.36
Tree 40 250000 iterate all - toArray() + loop 35.878 ms/op 25.100 ms/op 1.43
Tree 40 250000 iterate all - get(i) 128.76 ms/op 113.41 ms/op 1.14
MutableVector 250000 create 16.275 ms/op 13.583 ms/op 1.20
MutableVector 250000 get(125000) 14.513 ns/op 11.178 ns/op 1.30
MutableVector 250000 set(125000) 811.59 ns/op 564.54 ns/op 1.44
MutableVector 250000 toArray() 7.8989 ms/op 5.4072 ms/op 1.46
MutableVector 250000 iterate all - toArray() + loop 6.4618 ms/op 5.6015 ms/op 1.15
MutableVector 250000 iterate all - get(i) 3.5183 ms/op 3.0087 ms/op 1.17
Array 250000 create 5.8120 ms/op 5.3988 ms/op 1.08
Array 250000 clone - spread 2.4527 ms/op 3.4908 ms/op 0.70
Array 250000 get(125000) 1.2090 ns/op 1.6060 ns/op 0.75
Array 250000 set(125000) 1.2390 ns/op 1.5760 ns/op 0.79
Array 250000 iterate all - loop 144.26 us/op 150.89 us/op 0.96
effectiveBalanceIncrements clone Uint8Array 300000 87.839 us/op 77.238 us/op 1.14
effectiveBalanceIncrements clone MutableVector 300000 831.00 ns/op 654.00 ns/op 1.27
effectiveBalanceIncrements rw all Uint8Array 300000 297.00 us/op 247.30 us/op 1.20
effectiveBalanceIncrements rw all MutableVector 300000 217.44 ms/op 137.71 ms/op 1.58
phase0 afterProcessEpoch - 250000 vs - 7PWei 199.37 ms/op 194.73 ms/op 1.02
phase0 beforeProcessEpoch - 250000 vs - 7PWei 71.996 ms/op 56.913 ms/op 1.27
altair processEpoch - mainnet_e81889 632.84 ms/op 525.80 ms/op 1.20
mainnet_e81889 - altair beforeProcessEpoch 157.14 ms/op 114.35 ms/op 1.37
mainnet_e81889 - altair processJustificationAndFinalization 68.612 us/op 27.869 us/op 2.46
mainnet_e81889 - altair processInactivityUpdates 12.033 ms/op 8.7165 ms/op 1.38
mainnet_e81889 - altair processRewardsAndPenalties 135.14 ms/op 72.155 ms/op 1.87
mainnet_e81889 - altair processRegistryUpdates 15.230 us/op 4.7230 us/op 3.22
mainnet_e81889 - altair processSlashings 4.0630 us/op 1.0440 us/op 3.89
mainnet_e81889 - altair processEth1DataReset 4.3660 us/op 1.1050 us/op 3.95
mainnet_e81889 - altair processEffectiveBalanceUpdates 3.1680 ms/op 2.3843 ms/op 1.33
mainnet_e81889 - altair processSlashingsReset 30.166 us/op 8.1630 us/op 3.70
mainnet_e81889 - altair processRandaoMixesReset 24.236 us/op 6.6440 us/op 3.65
mainnet_e81889 - altair processHistoricalRootsUpdate 4.5320 us/op 1.5500 us/op 2.92
mainnet_e81889 - altair processParticipationFlagUpdates 14.273 us/op 5.4710 us/op 2.61
mainnet_e81889 - altair processSyncCommitteeUpdates 3.8760 us/op 1.1070 us/op 3.50
mainnet_e81889 - altair afterProcessEpoch 214.78 ms/op 217.06 ms/op 0.99
phase0 processEpoch - mainnet_e58758 592.76 ms/op 571.00 ms/op 1.04
mainnet_e58758 - phase0 beforeProcessEpoch 230.84 ms/op 211.68 ms/op 1.09
mainnet_e58758 - phase0 processJustificationAndFinalization 62.637 us/op 23.006 us/op 2.72
mainnet_e58758 - phase0 processRewardsAndPenalties 77.277 ms/op 110.90 ms/op 0.70
mainnet_e58758 - phase0 processRegistryUpdates 37.845 us/op 11.765 us/op 3.22
mainnet_e58758 - phase0 processSlashings 3.2780 us/op 1.0390 us/op 3.15
mainnet_e58758 - phase0 processEth1DataReset 3.4210 us/op 926.00 ns/op 3.69
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.6323 ms/op 2.1251 ms/op 1.24
mainnet_e58758 - phase0 processSlashingsReset 15.613 us/op 6.7370 us/op 2.32
mainnet_e58758 - phase0 processRandaoMixesReset 26.033 us/op 6.9250 us/op 3.76
mainnet_e58758 - phase0 processHistoricalRootsUpdate 4.2240 us/op 1.2150 us/op 3.48
mainnet_e58758 - phase0 processParticipationRecordUpdates 24.461 us/op 5.6170 us/op 4.35
mainnet_e58758 - phase0 afterProcessEpoch 179.40 ms/op 162.54 ms/op 1.10
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.4262 ms/op 2.3392 ms/op 1.04
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.6506 ms/op 2.2974 ms/op 1.15
altair processInactivityUpdates - 250000 normalcase 32.694 ms/op 35.318 ms/op 0.93
altair processInactivityUpdates - 250000 worstcase 38.829 ms/op 35.922 ms/op 1.08
phase0 processRegistryUpdates - 250000 normalcase 29.975 us/op 6.4660 us/op 4.64
phase0 processRegistryUpdates - 250000 badcase_full_deposits 508.06 us/op 363.98 us/op 1.40
phase0 processRegistryUpdates - 250000 worstcase 0.5 228.34 ms/op 218.11 ms/op 1.05
altair processRewardsAndPenalties - 250000 normalcase 146.09 ms/op 116.89 ms/op 1.25
altair processRewardsAndPenalties - 250000 worstcase 88.752 ms/op 118.85 ms/op 0.75
phase0 getAttestationDeltas - 250000 normalcase 15.266 ms/op 10.930 ms/op 1.40
phase0 getAttestationDeltas - 250000 worstcase 14.912 ms/op 11.293 ms/op 1.32
phase0 processSlashings - 250000 worstcase 6.4385 ms/op 5.0960 ms/op 1.26
altair processSyncCommitteeUpdates - 250000 341.78 ms/op 284.84 ms/op 1.20
BeaconState.hashTreeRoot - No change 697.00 ns/op 566.00 ns/op 1.23
BeaconState.hashTreeRoot - 1 full validator 86.430 us/op 71.430 us/op 1.21
BeaconState.hashTreeRoot - 32 full validator 866.93 us/op 800.15 us/op 1.08
BeaconState.hashTreeRoot - 512 full validator 8.5824 ms/op 6.8846 ms/op 1.25
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 125.73 us/op 100.52 us/op 1.25
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.3288 ms/op 1.1288 ms/op 1.18
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 19.294 ms/op 16.126 ms/op 1.20
BeaconState.hashTreeRoot - 1 balances 74.848 us/op 78.058 us/op 0.96
BeaconState.hashTreeRoot - 32 balances 724.34 us/op 672.37 us/op 1.08
BeaconState.hashTreeRoot - 512 balances 6.4797 ms/op 6.0987 ms/op 1.06
BeaconState.hashTreeRoot - 250000 balances 109.53 ms/op 103.55 ms/op 1.06
aggregationBits - 2048 els - zipIndexesInBitList 37.183 us/op 21.629 us/op 1.72
regular array get 100000 times 57.615 us/op 68.118 us/op 0.85
wrappedArray get 100000 times 57.083 us/op 66.211 us/op 0.86
arrayWithProxy get 100000 times 34.748 ms/op 28.819 ms/op 1.21
ssz.Root.equals 612.00 ns/op 499.00 ns/op 1.23
byteArrayEquals 583.00 ns/op 494.00 ns/op 1.18
shuffle list - 16384 els 12.398 ms/op 11.479 ms/op 1.08
shuffle list - 250000 els 180.75 ms/op 166.72 ms/op 1.08
processSlot - 1 slots 17.657 us/op 12.192 us/op 1.45
processSlot - 32 slots 2.6065 ms/op 1.8733 ms/op 1.39
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 414.64 us/op 362.40 us/op 1.14
getCommitteeAssignments - req 1 vs - 250000 vc 5.5231 ms/op 5.4186 ms/op 1.02
getCommitteeAssignments - req 100 vs - 250000 vc 7.9716 ms/op 7.9870 ms/op 1.00
getCommitteeAssignments - req 1000 vs - 250000 vc 8.1330 ms/op 8.5428 ms/op 0.95
computeProposers - vc 250000 22.085 ms/op 17.913 ms/op 1.23
computeEpochShuffling - vc 250000 182.36 ms/op 170.73 ms/op 1.07
getNextSyncCommittee - vc 250000 325.82 ms/op 276.58 ms/op 1.18
pass gossip attestations to forkchoice per slot 3.6177 ms/op 5.9905 ms/op 0.60
computeDeltas 4.3303 ms/op 3.5943 ms/op 1.20
computeProposerBoostScoreFromBalances 889.45 us/op 803.86 us/op 1.11
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.3430 ms/op 1.5292 ms/op 1.53
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 90.768 us/op 63.352 us/op 1.43
BLS verify - blst-native 2.6496 ms/op 2.1670 ms/op 1.22
BLS verifyMultipleSignatures 3 - blst-native 5.7016 ms/op 4.4719 ms/op 1.27
BLS verifyMultipleSignatures 8 - blst-native 12.017 ms/op 9.6977 ms/op 1.24
BLS verifyMultipleSignatures 32 - blst-native 43.309 ms/op 35.222 ms/op 1.23
BLS aggregatePubkeys 32 - blst-native 58.868 us/op 46.832 us/op 1.26
BLS aggregatePubkeys 128 - blst-native 228.57 us/op 182.94 us/op 1.25
getAttestationsForBlock 67.827 ms/op 53.771 ms/op 1.26
isKnown best case - 1 super set check 550.00 ns/op 494.00 ns/op 1.11
isKnown normal case - 2 super set checks 519.00 ns/op 488.00 ns/op 1.06
isKnown worse case - 16 super set checks 520.00 ns/op 480.00 ns/op 1.08
CheckpointStateCache - add get delete 13.470 us/op 10.575 us/op 1.27
validate gossip signedAggregateAndProof - struct 6.4396 ms/op 4.9985 ms/op 1.29
validate gossip attestation - struct 2.9423 ms/op 2.3604 ms/op 1.25
altair verifyImport mainnet_s3766816:31 8.4593 s/op 6.8402 s/op 1.24
pickEth1Vote - no votes 2.5121 ms/op 2.0443 ms/op 1.23
pickEth1Vote - max votes 25.238 ms/op 19.010 ms/op 1.33
pickEth1Vote - Eth1Data hashTreeRoot value x2048 14.986 ms/op 12.211 ms/op 1.23
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 24.560 ms/op 21.480 ms/op 1.14
pickEth1Vote - Eth1Data fastSerialize value x2048 1.8608 ms/op 1.3698 ms/op 1.36
pickEth1Vote - Eth1Data fastSerialize tree x2048 19.276 ms/op 11.513 ms/op 1.67
bytes32 toHexString 1.2610 us/op 940.00 ns/op 1.34
bytes32 Buffer.toString(hex) 831.00 ns/op 713.00 ns/op 1.17
bytes32 Buffer.toString(hex) from Uint8Array 1.0360 us/op 926.00 ns/op 1.12
bytes32 Buffer.toString(hex) + 0x 831.00 ns/op 728.00 ns/op 1.14
Object access 1 prop 0.39200 ns/op 0.33000 ns/op 1.19
Map access 1 prop 0.33200 ns/op 0.30500 ns/op 1.09
Object get x1000 16.487 ns/op 11.017 ns/op 1.50
Map get x1000 0.95900 ns/op 0.92800 ns/op 1.03
Object set x1000 113.89 ns/op 64.981 ns/op 1.75
Map set x1000 78.693 ns/op 42.341 ns/op 1.86
Return object 10000 times 0.42340 ns/op 0.44220 ns/op 0.96
Throw Error 10000 times 6.8506 us/op 5.8200 us/op 1.18
enrSubnets - fastDeserialize 64 bits 3.0500 us/op 2.3850 us/op 1.28
enrSubnets - ssz BitVector 64 bits 894.00 ns/op 762.00 ns/op 1.17
enrSubnets - fastDeserialize 4 bits 410.00 ns/op 413.00 ns/op 0.99
enrSubnets - ssz BitVector 4 bits 855.00 ns/op 765.00 ns/op 1.12
prioritizePeers score -10:0 att 32-0.1 sync 2-0 107.97 us/op 77.051 us/op 1.40
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 150.79 us/op 125.68 us/op 1.20
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 279.00 us/op 187.86 us/op 1.49
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 579.87 us/op 338.07 us/op 1.72
prioritizePeers score 0:0 att 64-1 sync 4-1 592.31 us/op 432.53 us/op 1.37
RateTracker 1000000 limit, 1 obj count per request 204.22 ns/op 170.99 ns/op 1.19
RateTracker 1000000 limit, 2 obj count per request 154.85 ns/op 124.91 ns/op 1.24
RateTracker 1000000 limit, 4 obj count per request 128.58 ns/op 101.83 ns/op 1.26
RateTracker 1000000 limit, 8 obj count per request 115.38 ns/op 90.290 ns/op 1.28
RateTracker with prune 4.9630 us/op 3.3520 us/op 1.48
array of 16000 items push then shift 5.2886 us/op 51.572 us/op 0.10
LinkedList of 16000 items push then shift 28.051 ns/op 15.268 ns/op 1.84
array of 16000 items push then pop 257.15 ns/op 164.07 ns/op 1.57
LinkedList of 16000 items push then pop 23.876 ns/op 13.421 ns/op 1.78
array of 24000 items push then shift 8.2218 us/op 77.333 us/op 0.11
LinkedList of 24000 items push then shift 27.866 ns/op 17.223 ns/op 1.62
array of 24000 items push then pop 217.84 ns/op 179.08 ns/op 1.22
LinkedList of 24000 items push then pop 22.644 ns/op 14.434 ns/op 1.57
intersect bitArray bitLen 8 12.543 ns/op 10.828 ns/op 1.16
intersect array and set length 8 189.71 ns/op 118.25 ns/op 1.60
intersect bitArray bitLen 128 72.530 ns/op 55.673 ns/op 1.30
intersect array and set length 128 2.4054 us/op 1.6609 us/op 1.45

by benchmarkbot/action

wemeetagain
wemeetagain previously approved these changes Jun 3, 2022
View reviewed changes
dapplion
dapplion reviewed Jun 4, 2022
View reviewed changes
packages/lodestar/src/chain/validation/syncCommitteeContributionAndProof.ts Outdated
code: SyncCommitteeErrorCode.SYNC_COMMITTEE_PARTICIPANTS_ALREADY_KNOWN,
});
if (!skipValidationKnownParticipants) {
if (chain.seenContributionAndProof.participantsKnown(contribution)) {
Copy link
Contributor

@dapplion dapplion Jun 4, 2022 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if statements can be merged into one with && (for both contributionAndProof + aggregates)

@twoeths twoeths merged commit a3e286f into unstable Jun 8, 2022
@twoeths twoeths deleted the tuyen/api-skip-validating-known-participants branch June 8, 2022 22:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion left review comments

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@twoeths @wemeetagain @dapplion