Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add sync committee assertions #4582

Merged
merged 10 commits into from Sep 23, 2022
Merged

Add sync committee assertions #4582

merged 10 commits into from Sep 23, 2022

Conversation

nazarhussain
Copy link
Contributor

Motivation

Sync committee assertions for sim tests.

Resolves #4276

@nazarhussain nazarhussain requested a review from a team as a code owner September 21, 2022 09:53
@nazarhussain nazarhussain self-assigned this Sep 21, 2022
@nazarhussain nazarhussain linked an issue Sep 21, 2022 that may be closed by this pull request
Add assertions to sim tests #4276
Closed
15 tasks
Base automatically changed from nh/4276-assertions-participation-per-slot to unstable September 23, 2022 07:40
@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: c501b00 Previous: bfeb48a Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.7047 ms/op 1.8725 ms/op 0.91
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 63.563 us/op 65.099 us/op 0.98
BLS verify - blst-native 2.1650 ms/op 2.1668 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 4.4727 ms/op 4.4710 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst-native 9.6738 ms/op 9.6744 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 35.151 ms/op 35.182 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 46.537 us/op 46.468 us/op 1.00
BLS aggregatePubkeys 128 - blst-native 182.18 us/op 181.70 us/op 1.00
getAttestationsForBlock 75.607 ms/op 82.447 ms/op 0.92
isKnown best case - 1 super set check 478.00 ns/op 475.00 ns/op 1.01
isKnown normal case - 2 super set checks 468.00 ns/op 473.00 ns/op 0.99
isKnown worse case - 16 super set checks 465.00 ns/op 463.00 ns/op 1.00
CheckpointStateCache - add get delete 8.8740 us/op 8.7840 us/op 1.01
validate gossip signedAggregateAndProof - struct 5.0162 ms/op 5.0137 ms/op 1.00
validate gossip attestation - struct 2.3884 ms/op 2.3670 ms/op 1.01
pickEth1Vote - no votes 2.1273 ms/op 2.1917 ms/op 0.97
pickEth1Vote - max votes 17.410 ms/op 19.999 ms/op 0.87
pickEth1Vote - Eth1Data hashTreeRoot value x2048 12.210 ms/op 12.790 ms/op 0.95
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 19.393 ms/op 21.062 ms/op 0.92
pickEth1Vote - Eth1Data fastSerialize value x2048 1.4073 ms/op 1.5076 ms/op 0.93
pickEth1Vote - Eth1Data fastSerialize tree x2048 11.850 ms/op 12.933 ms/op 0.92
bytes32 toHexString 937.00 ns/op 940.00 ns/op 1.00
bytes32 Buffer.toString(hex) 777.00 ns/op 771.00 ns/op 1.01
bytes32 Buffer.toString(hex) from Uint8Array 1.0230 us/op 1.0430 us/op 0.98
bytes32 Buffer.toString(hex) + 0x 782.00 ns/op 786.00 ns/op 0.99
Object access 1 prop 0.34900 ns/op 0.36600 ns/op 0.95
Map access 1 prop 0.30600 ns/op 0.31000 ns/op 0.99
Object get x1000 11.436 ns/op 11.487 ns/op 1.00
Map get x1000 0.94400 ns/op 0.94800 ns/op 1.00
Object set x1000 70.453 ns/op 71.826 ns/op 0.98
Map set x1000 46.654 ns/op 47.697 ns/op 0.98
Return object 10000 times 0.43870 ns/op 0.44180 ns/op 0.99
Throw Error 10000 times 5.8878 us/op 5.8996 us/op 1.00
enrSubnets - fastDeserialize 64 bits 2.4480 us/op 2.4900 us/op 0.98
enrSubnets - ssz BitVector 64 bits 799.00 ns/op 796.00 ns/op 1.00
enrSubnets - fastDeserialize 4 bits 365.00 ns/op 364.00 ns/op 1.00
enrSubnets - ssz BitVector 4 bits 806.00 ns/op 825.00 ns/op 0.98
prioritizePeers score -10:0 att 32-0.1 sync 2-0 81.404 us/op 80.820 us/op 1.01
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 122.63 us/op 115.27 us/op 1.06
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 199.09 us/op 191.34 us/op 1.04
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 334.89 us/op 374.21 us/op 0.89
prioritizePeers score 0:0 att 64-1 sync 4-1 407.82 us/op 405.22 us/op 1.01
RateTracker 1000000 limit, 1 obj count per request 176.28 ns/op 181.26 ns/op 0.97
RateTracker 1000000 limit, 2 obj count per request 129.55 ns/op 132.86 ns/op 0.98
RateTracker 1000000 limit, 4 obj count per request 104.10 ns/op 107.38 ns/op 0.97
RateTracker 1000000 limit, 8 obj count per request 92.639 ns/op 96.898 ns/op 0.96
RateTracker with prune 3.8090 us/op 3.9200 us/op 0.97
array of 16000 items push then shift 51.572 us/op 51.678 us/op 1.00
LinkedList of 16000 items push then shift 12.290 ns/op 12.189 ns/op 1.01
array of 16000 items push then pop 197.65 ns/op 198.02 ns/op 1.00
LinkedList of 16000 items push then pop 12.077 ns/op 12.093 ns/op 1.00
array of 24000 items push then shift 77.348 us/op 77.445 us/op 1.00
LinkedList of 24000 items push then shift 12.432 ns/op 12.518 ns/op 0.99
array of 24000 items push then pop 190.28 ns/op 191.14 ns/op 1.00
LinkedList of 24000 items push then pop 12.049 ns/op 12.049 ns/op 1.00
intersect bitArray bitLen 8 11.273 ns/op 10.694 ns/op 1.05
intersect array and set length 8 131.94 ns/op 132.83 ns/op 0.99
intersect bitArray bitLen 128 55.609 ns/op 55.702 ns/op 1.00
intersect array and set length 128 1.7230 us/op 1.7732 us/op 0.97
Buffer.concat 32 items 1.7580 ns/op 1.7250 ns/op 1.02
pass gossip attestations to forkchoice per slot 4.1452 ms/op 3.1696 ms/op 1.31
computeDeltas 3.3669 ms/op 3.5918 ms/op 0.94
computeProposerBoostScoreFromBalances 803.64 us/op 805.90 us/op 1.00
altair processAttestation - 250000 vs - 7PWei normalcase 3.3589 ms/op 3.2776 ms/op 1.02
altair processAttestation - 250000 vs - 7PWei worstcase 5.0240 ms/op 5.5270 ms/op 0.91
altair processAttestation - setStatus - 1/6 committees join 184.56 us/op 181.31 us/op 1.02
altair processAttestation - setStatus - 1/3 committees join 359.05 us/op 355.27 us/op 1.01
altair processAttestation - setStatus - 1/2 committees join 507.94 us/op 511.34 us/op 0.99
altair processAttestation - setStatus - 2/3 committees join 665.81 us/op 671.87 us/op 0.99
altair processAttestation - setStatus - 4/5 committees join 927.87 us/op 935.64 us/op 0.99
altair processAttestation - setStatus - 100% committees join 1.1272 ms/op 1.1261 ms/op 1.00
altair processBlock - 250000 vs - 7PWei normalcase 25.504 ms/op 23.350 ms/op 1.09
altair processBlock - 250000 vs - 7PWei normalcase hashState 34.918 ms/op 33.726 ms/op 1.04
altair processBlock - 250000 vs - 7PWei worstcase 79.992 ms/op 80.535 ms/op 0.99
altair processBlock - 250000 vs - 7PWei worstcase hashState 111.53 ms/op 101.75 ms/op 1.10
phase0 processBlock - 250000 vs - 7PWei normalcase 3.1530 ms/op 3.0830 ms/op 1.02
phase0 processBlock - 250000 vs - 7PWei worstcase 50.097 ms/op 51.081 ms/op 0.98
altair processEth1Data - 250000 vs - 7PWei normalcase 713.69 us/op 702.32 us/op 1.02
Tree 40 250000 create 712.46 ms/op 700.71 ms/op 1.02
Tree 40 250000 get(125000) 237.78 ns/op 255.44 ns/op 0.93
Tree 40 250000 set(125000) 2.3941 us/op 2.4014 us/op 1.00
Tree 40 250000 toArray() 27.078 ms/op 28.057 ms/op 0.97
Tree 40 250000 iterate all - toArray() + loop 30.029 ms/op 28.430 ms/op 1.06
Tree 40 250000 iterate all - get(i) 106.94 ms/op 113.91 ms/op 0.94
MutableVector 250000 create 11.754 ms/op 14.259 ms/op 0.82
MutableVector 250000 get(125000) 10.978 ns/op 10.843 ns/op 1.01
MutableVector 250000 set(125000) 459.13 ns/op 560.12 ns/op 0.82
MutableVector 250000 toArray() 5.4951 ms/op 6.1238 ms/op 0.90
MutableVector 250000 iterate all - toArray() + loop 5.6063 ms/op 6.1029 ms/op 0.92
MutableVector 250000 iterate all - get(i) 2.6339 ms/op 2.7083 ms/op 0.97
Array 250000 create 5.4983 ms/op 6.5461 ms/op 0.84
Array 250000 clone - spread 2.7074 ms/op 3.6254 ms/op 0.75
Array 250000 get(125000) 1.2800 ns/op 1.6460 ns/op 0.78
Array 250000 set(125000) 1.2830 ns/op 1.5990 ns/op 0.80
Array 250000 iterate all - loop 153.05 us/op 154.45 us/op 0.99
effectiveBalanceIncrements clone Uint8Array 300000 68.094 us/op 54.780 us/op 1.24
effectiveBalanceIncrements clone MutableVector 300000 671.00 ns/op 1.1850 us/op 0.57
effectiveBalanceIncrements rw all Uint8Array 300000 247.14 us/op 247.41 us/op 1.00
effectiveBalanceIncrements rw all MutableVector 300000 126.47 ms/op 185.62 ms/op 0.68
phase0 afterProcessEpoch - 250000 vs - 7PWei 185.35 ms/op 185.94 ms/op 1.00
phase0 beforeProcessEpoch - 250000 vs - 7PWei 58.642 ms/op 57.841 ms/op 1.01
altair processEpoch - mainnet_e81889 544.20 ms/op 558.13 ms/op 0.98
mainnet_e81889 - altair beforeProcessEpoch 110.28 ms/op 121.92 ms/op 0.90
mainnet_e81889 - altair processJustificationAndFinalization 26.152 us/op 19.954 us/op 1.31
mainnet_e81889 - altair processInactivityUpdates 8.6617 ms/op 8.5836 ms/op 1.01
mainnet_e81889 - altair processRewardsAndPenalties 80.170 ms/op 77.289 ms/op 1.04
mainnet_e81889 - altair processRegistryUpdates 2.8670 us/op 3.0220 us/op 0.95
mainnet_e81889 - altair processSlashings 668.00 ns/op 1.0090 us/op 0.66
mainnet_e81889 - altair processEth1DataReset 642.00 ns/op 904.00 ns/op 0.71
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.2488 ms/op 2.2536 ms/op 1.00
mainnet_e81889 - altair processSlashingsReset 4.3420 us/op 7.1200 us/op 0.61
mainnet_e81889 - altair processRandaoMixesReset 4.1750 us/op 6.4240 us/op 0.65
mainnet_e81889 - altair processHistoricalRootsUpdate 637.00 ns/op 1.1300 us/op 0.56
mainnet_e81889 - altair processParticipationFlagUpdates 2.3210 us/op 3.3090 us/op 0.70
mainnet_e81889 - altair processSyncCommitteeUpdates 688.00 ns/op 865.00 ns/op 0.80
mainnet_e81889 - altair afterProcessEpoch 197.68 ms/op 197.75 ms/op 1.00
phase0 processEpoch - mainnet_e58758 480.25 ms/op 487.05 ms/op 0.99
mainnet_e58758 - phase0 beforeProcessEpoch 178.29 ms/op 174.26 ms/op 1.02
mainnet_e58758 - phase0 processJustificationAndFinalization 26.672 us/op 16.309 us/op 1.64
mainnet_e58758 - phase0 processRewardsAndPenalties 73.111 ms/op 96.087 ms/op 0.76
mainnet_e58758 - phase0 processRegistryUpdates 10.033 us/op 9.9470 us/op 1.01
mainnet_e58758 - phase0 processSlashings 1.2140 us/op 780.00 ns/op 1.56
mainnet_e58758 - phase0 processEth1DataReset 1.4160 us/op 857.00 ns/op 1.65
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.0458 ms/op 1.8530 ms/op 1.10
mainnet_e58758 - phase0 processSlashingsReset 7.0250 us/op 6.2190 us/op 1.13
mainnet_e58758 - phase0 processRandaoMixesReset 8.6510 us/op 6.1850 us/op 1.40
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.4450 us/op 1.0100 us/op 1.43
mainnet_e58758 - phase0 processParticipationRecordUpdates 7.7820 us/op 4.8780 us/op 1.60
mainnet_e58758 - phase0 afterProcessEpoch 162.36 ms/op 162.27 ms/op 1.00
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.2098 ms/op 1.9927 ms/op 1.11
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.9166 ms/op 2.4996 ms/op 1.17
altair processInactivityUpdates - 250000 normalcase 39.873 ms/op 33.618 ms/op 1.19
altair processInactivityUpdates - 250000 worstcase 41.094 ms/op 40.519 ms/op 1.01
phase0 processRegistryUpdates - 250000 normalcase 11.772 us/op 8.5550 us/op 1.38
phase0 processRegistryUpdates - 250000 badcase_full_deposits 373.00 us/op 386.63 us/op 0.96
phase0 processRegistryUpdates - 250000 worstcase 0.5 175.50 ms/op 181.31 ms/op 0.97
altair processRewardsAndPenalties - 250000 normalcase 76.061 ms/op 97.291 ms/op 0.78
altair processRewardsAndPenalties - 250000 worstcase 76.915 ms/op 74.071 ms/op 1.04
phase0 getAttestationDeltas - 250000 normalcase 10.703 ms/op 12.233 ms/op 0.87
phase0 getAttestationDeltas - 250000 worstcase 12.219 ms/op 12.709 ms/op 0.96
phase0 processSlashings - 250000 worstcase 5.0811 ms/op 4.9742 ms/op 1.02
altair processSyncCommitteeUpdates - 250000 288.99 ms/op 289.44 ms/op 1.00
BeaconState.hashTreeRoot - No change 522.00 ns/op 525.00 ns/op 0.99
BeaconState.hashTreeRoot - 1 full validator 65.727 us/op 73.697 us/op 0.89
BeaconState.hashTreeRoot - 32 full validator 711.79 us/op 728.28 us/op 0.98
BeaconState.hashTreeRoot - 512 full validator 7.7384 ms/op 8.2816 ms/op 0.93
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 87.988 us/op 87.319 us/op 1.01
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2875 ms/op 1.3070 ms/op 0.99
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.952 ms/op 16.792 ms/op 1.01
BeaconState.hashTreeRoot - 1 balances 68.457 us/op 70.131 us/op 0.98
BeaconState.hashTreeRoot - 32 balances 631.25 us/op 661.20 us/op 0.95
BeaconState.hashTreeRoot - 512 balances 6.0041 ms/op 6.2234 ms/op 0.96
BeaconState.hashTreeRoot - 250000 balances 97.808 ms/op 103.26 ms/op 0.95
aggregationBits - 2048 els - zipIndexesInBitList 26.729 us/op 24.892 us/op 1.07
regular array get 100000 times 61.050 us/op 60.563 us/op 1.01
wrappedArray get 100000 times 64.100 us/op 60.564 us/op 1.06
arrayWithProxy get 100000 times 28.885 ms/op 28.811 ms/op 1.00
ssz.Root.equals 472.00 ns/op 452.00 ns/op 1.04
byteArrayEquals 462.00 ns/op 437.00 ns/op 1.06
shuffle list - 16384 els 11.215 ms/op 11.193 ms/op 1.00
shuffle list - 250000 els 166.32 ms/op 165.59 ms/op 1.00
processSlot - 1 slots 13.303 us/op 13.620 us/op 0.98
processSlot - 32 slots 1.9084 ms/op 1.9647 ms/op 0.97
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 382.59 us/op 377.17 us/op 1.01
getCommitteeAssignments - req 1 vs - 250000 vc 5.4572 ms/op 5.4692 ms/op 1.00
getCommitteeAssignments - req 100 vs - 250000 vc 7.9729 ms/op 7.9892 ms/op 1.00
getCommitteeAssignments - req 1000 vs - 250000 vc 8.5758 ms/op 8.5466 ms/op 1.00
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 7.6100 ns/op 8.0600 ns/op 0.94
state getBlockRootAtSlot - 250000 vs - 7PWei 1.0854 us/op 1.0025 us/op 1.08
computeProposers - vc 250000 17.046 ms/op 17.032 ms/op 1.00
computeEpochShuffling - vc 250000 170.17 ms/op 171.03 ms/op 0.99
getNextSyncCommittee - vc 250000 285.38 ms/op 284.43 ms/op 1.00

by benchmarkbot/action

g11tech
g11tech approved these changes Sep 23, 2022
View reviewed changes
Copy link
Contributor

@g11tech g11tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@nazarhussain nazarhussain merged commit 8b90b2c into unstable Sep 23, 2022
@nazarhussain nazarhussain deleted the nh/4276-sync-committee branch September 23, 2022 08:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@g11tech g11tech g11tech approved these changes

Assignees

@nazarhussain nazarhussain

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add assertions to sim tests
2 participants
@nazarhussain @g11tech