Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Delete redundant sync committee duties #3615

Merged
merged 2 commits into from
Jan 17, 2022
Merged

Delete redundant sync committee duties #3615

merged 2 commits into from
Jan 17, 2022

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Jan 13, 2022

Motivation

  • There are redundant sync committee duties because we keep merging it to our map.
  • For example query1 returns [period_n]: [validator1, validator2, validator3]
  • query2 returns [period_n]: [validator2, validator3, validator4]
    => we should delete validator1 out of our duties, and overwrite duties for validator2, validator3, add new duties for validator4

Description

  • When next duties query returned, check for redundant duties and delete

Closes #3572

@codecov
Copy link

codecov bot commented Jan 13, 2022 •
edited

Codecov Report

Merging #3615 (cecd336) into master (1bd730a) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3615      +/-   ##
==========================================
+ Coverage   37.40%   37.42%   +0.02%     
==========================================
  Files         311      311              
  Lines        8374     8373       -1     
  Branches     1299     1298       -1     
==========================================
+ Hits         3132     3134       +2     
+ Misses       5093     5092       -1     
+ Partials      149      147       -2

@codeclimate
Copy link

codeclimate bot commented Jan 13, 2022 •
edited

Code Climate has analyzed commit cecd336 and detected 0 issues on this pull request.

View more on Code Climate.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 13, 2022 •
edited

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 8c10563 Previous: 1bd730a Ratio
BeaconState.hashTreeRoot - No change 586.00 ns/op 489.00 ns/op 1.20
BeaconState.hashTreeRoot - 1 full validator 150.39 us/op 113.05 us/op 1.33
BeaconState.hashTreeRoot - 32 full validator 2.2297 ms/op 1.6612 ms/op 1.34
BeaconState.hashTreeRoot - 512 full validator 28.635 ms/op 22.235 ms/op 1.29
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 144.84 us/op 111.07 us/op 1.30
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.3615 ms/op 1.8161 ms/op 1.30
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 32.387 ms/op 24.740 ms/op 1.31
BeaconState.hashTreeRoot - 1 balances 105.50 us/op 82.062 us/op 1.29
BeaconState.hashTreeRoot - 32 balances 851.76 us/op 683.30 us/op 1.25
BeaconState.hashTreeRoot - 512 balances 8.5914 ms/op 6.6580 ms/op 1.29
BeaconState.hashTreeRoot - 250000 balances 153.71 ms/op 130.69 ms/op 1.18
processSlot - 1 slots 69.785 us/op 42.749 us/op 1.63
processSlot - 32 slots 3.5097 ms/op 2.5287 ms/op 1.39
getCommitteeAssignments - req 1 vs - 250000 vc 4.7261 ms/op 4.7354 ms/op 1.00
getCommitteeAssignments - req 100 vs - 250000 vc 6.5809 ms/op 6.5819 ms/op 1.00
getCommitteeAssignments - req 1000 vs - 250000 vc 7.1986 ms/op 7.0608 ms/op 1.02
computeProposers - vc 250000 25.159 ms/op 18.346 ms/op 1.37
computeEpochShuffling - vc 250000 199.53 ms/op 165.84 ms/op 1.20
getNextSyncCommittee - vc 250000 371.69 ms/op 316.63 ms/op 1.17
altair processAttestation - 250000 vs - 7PWei normalcase 58.690 ms/op 48.067 ms/op 1.22
altair processAttestation - 250000 vs - 7PWei worstcase 51.804 ms/op 43.657 ms/op 1.19
altair processAttestation - setStatus - 1/6 committees join 15.181 ms/op 9.8596 ms/op 1.54
altair processAttestation - setStatus - 1/3 committees join 27.745 ms/op 21.222 ms/op 1.31
altair processAttestation - setStatus - 1/2 committees join 47.284 ms/op 33.184 ms/op 1.42
altair processAttestation - setStatus - 2/3 committees join 56.533 ms/op 43.943 ms/op 1.29
altair processAttestation - setStatus - 4/5 committees join 70.378 ms/op 51.715 ms/op 1.36
altair processAttestation - setStatus - 100% committees join 85.975 ms/op 65.338 ms/op 1.32
altair processAttestation - updateEpochParticipants - 1/6 committees join 15.221 ms/op 11.700 ms/op 1.30
altair processAttestation - updateEpochParticipants - 1/3 committees join 25.446 ms/op 20.190 ms/op 1.26
altair processAttestation - updateEpochParticipants - 1/2 committees join 34.697 ms/op 74.422 ms/op 0.47
altair processAttestation - updateEpochParticipants - 2/3 committees join 29.742 ms/op 24.883 ms/op 1.20
altair processAttestation - updateEpochParticipants - 4/5 committees join 31.196 ms/op 25.706 ms/op 1.21
altair processAttestation - updateEpochParticipants - 100% committees join 33.430 ms/op 27.418 ms/op 1.22
altair processAttestation - updateAllStatus 28.274 ms/op 20.576 ms/op 1.37
altair processBlock - 250000 vs - 7PWei normalcase 49.549 ms/op 43.541 ms/op 1.14
altair processBlock - 250000 vs - 7PWei worstcase 138.25 ms/op 107.79 ms/op 1.28
altair processEpoch - mainnet_e81889 1.0777 s/op 981.54 ms/op 1.10
mainnet_e81889 - altair beforeProcessEpoch 404.29 ms/op 263.85 ms/op 1.53
mainnet_e81889 - altair processJustificationAndFinalization 123.48 us/op 102.17 us/op 1.21
mainnet_e81889 - altair processInactivityUpdates 18.260 ms/op 16.885 ms/op 1.08
mainnet_e81889 - altair processRewardsAndPenalties 153.27 ms/op 169.52 ms/op 0.90
mainnet_e81889 - altair processRegistryUpdates 22.284 us/op 9.8580 us/op 2.26
mainnet_e81889 - altair processSlashings 6.2500 us/op 1.9400 us/op 3.22
mainnet_e81889 - altair processEth1DataReset 7.0900 us/op 1.8930 us/op 3.75
mainnet_e81889 - altair processEffectiveBalanceUpdates 12.188 ms/op 10.354 ms/op 1.18
mainnet_e81889 - altair processSlashingsReset 37.597 us/op 14.625 us/op 2.57
mainnet_e81889 - altair processRandaoMixesReset 51.514 us/op 13.982 us/op 3.68
mainnet_e81889 - altair processHistoricalRootsUpdate 9.5630 us/op 1.1920 us/op 8.02
mainnet_e81889 - altair processParticipationFlagUpdates 150.92 ms/op 96.192 ms/op 1.57
mainnet_e81889 - altair processSyncCommitteeUpdates 6.3510 us/op 1.7250 us/op 3.68
mainnet_e81889 - altair afterProcessEpoch 232.02 ms/op 197.94 ms/op 1.17
altair processInactivityUpdates - 250000 normalcase 67.129 ms/op 62.279 ms/op 1.08
altair processInactivityUpdates - 250000 worstcase 89.008 ms/op 64.489 ms/op 1.38
altair processParticipationFlagUpdates - 250000 anycase 95.477 ms/op 88.480 ms/op 1.08
altair processRewardsAndPenalties - 250000 normalcase 125.31 ms/op 160.61 ms/op 0.78
altair processRewardsAndPenalties - 250000 worstcase 126.98 ms/op 115.26 ms/op 1.10
altair processSyncCommitteeUpdates - 250000 395.35 ms/op 313.08 ms/op 1.26
Tree 40 250000 create 934.66 ms/op 635.17 ms/op 1.47
Tree 40 250000 get(125000) 296.95 ns/op 291.64 ns/op 1.02
Tree 40 250000 set(125000) 2.0911 us/op 2.0900 us/op 1.00
Tree 40 250000 toArray() 56.370 ms/op 36.615 ms/op 1.54
Tree 40 250000 iterate all - toArray() + loop 49.225 ms/op 37.244 ms/op 1.32
Tree 40 250000 iterate all - get(i) 117.02 ms/op 107.64 ms/op 1.09
MutableVector 250000 create 20.861 ms/op 19.215 ms/op 1.09
MutableVector 250000 get(125000) 20.351 ns/op 11.885 ns/op 1.71
MutableVector 250000 set(125000) 571.07 ns/op 516.45 ns/op 1.11
MutableVector 250000 toArray() 8.0423 ms/op 7.6450 ms/op 1.05
MutableVector 250000 iterate all - toArray() + loop 8.5634 ms/op 9.0186 ms/op 0.95
MutableVector 250000 iterate all - get(i) 3.0851 ms/op 2.8673 ms/op 1.08
Array 250000 create 6.0443 ms/op 4.7494 ms/op 1.27
Array 250000 clone - spread 1.8120 ms/op 1.6942 ms/op 1.07
Array 250000 get(125000) 0.91100 ns/op 0.81400 ns/op 1.12
Array 250000 set(125000) 0.92100 ns/op 0.81000 ns/op 1.14
Array 250000 iterate all - loop 122.46 us/op 146.88 us/op 0.83
aggregationBits - 2048 els - readonlyValues 197.43 us/op 194.50 us/op 1.02
aggregationBits - 2048 els - zipIndexesInBitList 33.627 us/op 38.746 us/op 0.87
regular array get 100000 times 51.088 us/op 60.588 us/op 0.84
wrappedArray get 100000 times 50.707 us/op 59.104 us/op 0.86
arrayWithProxy get 100000 times 30.621 ms/op 24.081 ms/op 1.27
ssz.Root.equals 1.1490 us/op 945.00 ns/op 1.22
ssz.Root.equals with valueOf() 1.4630 us/op 1.1140 us/op 1.31
byteArrayEquals with valueOf() 1.4310 us/op 1.0660 us/op 1.34
phase0 processBlock - 250000 vs - 7PWei normalcase 11.620 ms/op 9.1443 ms/op 1.27
phase0 processBlock - 250000 vs - 7PWei worstcase 91.426 ms/op 68.878 ms/op 1.33
phase0 afterProcessEpoch - 250000 vs - 7PWei 213.55 ms/op 185.73 ms/op 1.15
phase0 beforeProcessEpoch - 250000 vs - 7PWei 655.72 ms/op 511.99 ms/op 1.28
phase0 processEpoch - mainnet_e58758 984.84 ms/op 738.30 ms/op 1.33
mainnet_e58758 - phase0 beforeProcessEpoch 599.74 ms/op 406.31 ms/op 1.48
mainnet_e58758 - phase0 processJustificationAndFinalization 122.60 us/op 116.10 us/op 1.06
mainnet_e58758 - phase0 processRewardsAndPenalties 137.49 ms/op 97.549 ms/op 1.41
mainnet_e58758 - phase0 processRegistryUpdates 90.287 us/op 62.719 us/op 1.44
mainnet_e58758 - phase0 processSlashings 7.2440 us/op 1.9920 us/op 3.64
mainnet_e58758 - phase0 processEth1DataReset 5.8180 us/op 2.1140 us/op 2.75
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 9.1591 ms/op 8.5280 ms/op 1.07
mainnet_e58758 - phase0 processSlashingsReset 28.217 us/op 12.673 us/op 2.23
mainnet_e58758 - phase0 processRandaoMixesReset 38.126 us/op 18.716 us/op 2.04
mainnet_e58758 - phase0 processHistoricalRootsUpdate 8.8340 us/op 2.1230 us/op 4.16
mainnet_e58758 - phase0 processParticipationRecordUpdates 31.887 us/op 16.497 us/op 1.93
mainnet_e58758 - phase0 afterProcessEpoch 183.34 ms/op 161.65 ms/op 1.13
phase0 processEffectiveBalanceUpdates - 250000 normalcase 12.220 ms/op 9.4338 ms/op 1.30
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.6452 s/op 1.1935 s/op 1.38
phase0 processRegistryUpdates - 250000 normalcase 97.655 us/op 53.915 us/op 1.81
phase0 processRegistryUpdates - 250000 badcase_full_deposits 3.8564 ms/op 2.7833 ms/op 1.39
phase0 processRegistryUpdates - 250000 worstcase 0.5 2.4397 s/op 1.3599 s/op 1.79
phase0 getAttestationDeltas - 250000 normalcase 34.300 ms/op 30.838 ms/op 1.11
phase0 getAttestationDeltas - 250000 worstcase 33.529 ms/op 31.248 ms/op 1.07
phase0 processSlashings - 250000 worstcase 40.066 ms/op 35.077 ms/op 1.14
shuffle list - 16384 els 12.375 ms/op 11.277 ms/op 1.10
shuffle list - 250000 els 172.12 ms/op 162.87 ms/op 1.06
getEffectiveBalances - 250000 vs - 7PWei 9.9111 ms/op 9.6669 ms/op 1.03
pass gossip attestations to forkchoice per slot 16.865 ms/op 15.148 ms/op 1.11
computeDeltas 2.9238 ms/op 3.3280 ms/op 0.88
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.5207 ms/op 1.9900 ms/op 1.27
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 696.37 us/op 607.62 us/op 1.15
BLS verify - blst-native 1.9367 ms/op 1.6172 ms/op 1.20
BLS verifyMultipleSignatures 3 - blst-native 3.9695 ms/op 3.3096 ms/op 1.20
BLS verifyMultipleSignatures 8 - blst-native 8.9903 ms/op 7.1830 ms/op 1.25
BLS verifyMultipleSignatures 32 - blst-native 30.520 ms/op 26.098 ms/op 1.17
BLS aggregatePubkeys 32 - blst-native 41.156 us/op 34.130 us/op 1.21
BLS aggregatePubkeys 128 - blst-native 165.46 us/op 134.72 us/op 1.23
getAttestationsForBlock 95.572 ms/op 84.195 ms/op 1.14
CheckpointStateCache - add get delete 21.686 us/op 16.435 us/op 1.32
validate gossip signedAggregateAndProof - struct 4.9919 ms/op 3.8632 ms/op 1.29
validate gossip signedAggregateAndProof - treeBacked 4.8730 ms/op 3.8511 ms/op 1.27
validate gossip attestation - struct 2.2407 ms/op 1.8105 ms/op 1.24
validate gossip attestation - treeBacked 2.3717 ms/op 1.8440 ms/op 1.29
bytes32 toHexString 1.8310 us/op 1.4860 us/op 1.23
bytes32 Buffer.toString(hex) 726.00 ns/op 590.00 ns/op 1.23
bytes32 Buffer.toString(hex) from Uint8Array 1.1040 us/op 761.00 ns/op 1.45
bytes32 Buffer.toString(hex) + 0x 754.00 ns/op 599.00 ns/op 1.26
Object access 1 prop 0.38700 ns/op 0.28900 ns/op 1.34
Map access 1 prop 0.32700 ns/op 0.25100 ns/op 1.30
Object get x1000 14.297 ns/op 15.619 ns/op 0.92
Map get x1000 0.90400 ns/op 0.86500 ns/op 1.05
Object set x1000 99.298 ns/op 93.526 ns/op 1.06
Map set x1000 69.564 ns/op 55.827 ns/op 1.25
Return object 10000 times 0.36280 ns/op 0.32350 ns/op 1.12
Throw Error 10000 times 6.1555 us/op 5.2463 us/op 1.17
enrSubnets - fastDeserialize 64 bits 1.2360 us/op 1.0910 us/op 1.13
enrSubnets - ssz BitVector 64 bits 15.372 us/op 14.713 us/op 1.04
enrSubnets - fastDeserialize 4 bits 491.00 ns/op 377.00 ns/op 1.30
enrSubnets - ssz BitVector 4 bits 2.8660 us/op 2.4650 us/op 1.16
RateTracker 1000000 limit, 1 obj count per request 170.09 ns/op 154.57 ns/op 1.10
RateTracker 1000000 limit, 2 obj count per request 128.34 ns/op 115.46 ns/op 1.11
RateTracker 1000000 limit, 4 obj count per request 103.93 ns/op 94.555 ns/op 1.10
RateTracker 1000000 limit, 8 obj count per request 98.400 ns/op 84.281 ns/op 1.17
RateTracker with prune 4.2080 us/op 3.4510 us/op 1.22

by benchmarkbot/action

dapplion
dapplion reviewed Jan 13, 2022
View reviewed changes
Copy link
Contributor

@dapplion dapplion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just completely overwrite the contents of the Map on every request?

@twoeths
Copy link
Contributor Author

twoeths commented Jan 17, 2022

Why not just completely overwrite the contents of the Map on every request?

yeah that would be simpler, will do it 👍

@twoeths twoeths force-pushed the tuyen/overwrite-sync-committee-duties branch from 075b88d to cecd336 Compare January 17, 2022 04:46
@dapplion dapplion merged commit 752d2e6 into master Jan 17, 2022
@dapplion dapplion deleted the tuyen/overwrite-sync-committee-duties branch January 17, 2022 06:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion approved these changes

@g11tech g11tech Awaiting requested review from g11tech

@mpetrunic mpetrunic Awaiting requested review from mpetrunic

@wemeetagain wemeetagain Awaiting requested review from wemeetagain

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Kintsugi] Error on publishContributionAndProofs
2 participants
@twoeths @dapplion