Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add config matching tests for validator interop with other beacon nodes #4271

Merged
merged 1 commit into from Jul 9, 2022

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Jul 9, 2022

Motivation
We have had the validator interop broken with other bns in the past.

The PR adds test to validate we don't break this behavior.

@g11tech g11tech requested a review from a team as a code owner July 9, 2022 18:43
@g11tech g11tech enabled auto-merge (squash) July 9, 2022 18:43
@g11tech g11tech merged commit 7bf6764 into unstable Jul 9, 2022
@g11tech g11tech deleted the g11tech/validator-interop-tests branch July 9, 2022 19:05
@github-actions
Copy link
Contributor

github-actions bot commented Jul 9, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 7c2f5f6 Previous: 266f9e2 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.9344 ms/op 2.2413 ms/op 1.31
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 98.886 us/op 71.443 us/op 1.38
BLS verify - blst-native 2.4664 ms/op 2.1811 ms/op 1.13
BLS verifyMultipleSignatures 3 - blst-native 5.1989 ms/op 4.4848 ms/op 1.16
BLS verifyMultipleSignatures 8 - blst-native 10.910 ms/op 9.6983 ms/op 1.12
BLS verifyMultipleSignatures 32 - blst-native 40.967 ms/op 35.301 ms/op 1.16
BLS aggregatePubkeys 32 - blst-native 56.291 us/op 46.803 us/op 1.20
BLS aggregatePubkeys 128 - blst-native 214.22 us/op 182.87 us/op 1.17
getAttestationsForBlock 63.608 ms/op 44.203 ms/op 1.44
isKnown best case - 1 super set check 563.00 ns/op 483.00 ns/op 1.17
isKnown normal case - 2 super set checks 565.00 ns/op 464.00 ns/op 1.22
isKnown worse case - 16 super set checks 536.00 ns/op 470.00 ns/op 1.14
CheckpointStateCache - add get delete 15.329 us/op 11.485 us/op 1.33
validate gossip signedAggregateAndProof - struct 5.6784 ms/op 5.0672 ms/op 1.12
validate gossip attestation - struct 2.7466 ms/op 2.3805 ms/op 1.15
altair verifyImport mainnet_s3766816:31 8.3724 s/op 7.0702 s/op 1.18
pickEth1Vote - no votes 2.8776 ms/op 2.1675 ms/op 1.33
pickEth1Vote - max votes 34.297 ms/op 26.938 ms/op 1.27
pickEth1Vote - Eth1Data hashTreeRoot value x2048 17.241 ms/op 13.930 ms/op 1.24
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 30.748 ms/op 26.643 ms/op 1.15
pickEth1Vote - Eth1Data fastSerialize value x2048 2.0490 ms/op 1.4760 ms/op 1.39
pickEth1Vote - Eth1Data fastSerialize tree x2048 25.045 ms/op 14.823 ms/op 1.69
bytes32 toHexString 1.5230 us/op 1.0710 us/op 1.42
bytes32 Buffer.toString(hex) 963.00 ns/op 734.00 ns/op 1.31
bytes32 Buffer.toString(hex) from Uint8Array 1.2920 us/op 1.0010 us/op 1.29
bytes32 Buffer.toString(hex) + 0x 961.00 ns/op 740.00 ns/op 1.30
Object access 1 prop 0.50000 ns/op 0.37500 ns/op 1.33
Map access 1 prop 0.37400 ns/op 0.31600 ns/op 1.18
Object get x1000 19.456 ns/op 11.041 ns/op 1.76
Map get x1000 1.1160 ns/op 0.93800 ns/op 1.19
Object set x1000 146.09 ns/op 80.588 ns/op 1.81
Map set x1000 92.585 ns/op 52.677 ns/op 1.76
Return object 10000 times 0.48060 ns/op 0.43920 ns/op 1.09
Throw Error 10000 times 7.6950 us/op 6.1715 us/op 1.25
enrSubnets - fastDeserialize 64 bits 3.7450 us/op 2.9460 us/op 1.27
enrSubnets - ssz BitVector 64 bits 1.0800 us/op 823.00 ns/op 1.31
enrSubnets - fastDeserialize 4 bits 498.00 ns/op 422.00 ns/op 1.18
enrSubnets - ssz BitVector 4 bits 1.0620 us/op 841.00 ns/op 1.26
prioritizePeers score -10:0 att 32-0.1 sync 2-0 136.59 us/op 87.905 us/op 1.55
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 177.10 us/op 115.48 us/op 1.53
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 322.72 us/op 206.88 us/op 1.56
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 710.36 us/op 396.20 us/op 1.79
prioritizePeers score 0:0 att 64-1 sync 4-1 661.05 us/op 409.19 us/op 1.62
RateTracker 1000000 limit, 1 obj count per request 239.00 ns/op 188.05 ns/op 1.27
RateTracker 1000000 limit, 2 obj count per request 178.97 ns/op 140.23 ns/op 1.28
RateTracker 1000000 limit, 4 obj count per request 147.70 ns/op 116.17 ns/op 1.27
RateTracker 1000000 limit, 8 obj count per request 136.54 ns/op 106.60 ns/op 1.28
RateTracker with prune 6.7540 us/op 4.5840 us/op 1.47
array of 16000 items push then shift 5.9865 us/op 51.709 us/op 0.12
LinkedList of 16000 items push then shift 35.563 ns/op 17.952 ns/op 1.98
array of 16000 items push then pop 282.71 ns/op 200.21 ns/op 1.41
LinkedList of 16000 items push then pop 27.274 ns/op 15.169 ns/op 1.80
array of 24000 items push then shift 8.5948 us/op 77.578 us/op 0.11
LinkedList of 24000 items push then shift 34.203 ns/op 23.246 ns/op 1.47
array of 24000 items push then pop 266.97 ns/op 206.48 ns/op 1.29
LinkedList of 24000 items push then pop 26.776 ns/op 16.434 ns/op 1.63
intersect bitArray bitLen 8 13.552 ns/op 10.777 ns/op 1.26
intersect array and set length 8 210.82 ns/op 147.60 ns/op 1.43
intersect bitArray bitLen 128 78.432 ns/op 59.067 ns/op 1.33
intersect array and set length 128 2.6093 us/op 1.8580 us/op 1.40
pass gossip attestations to forkchoice per slot 6.6462 ms/op 2.9507 ms/op 2.25
computeDeltas 4.9968 ms/op 3.5719 ms/op 1.40
computeProposerBoostScoreFromBalances 975.35 us/op 806.45 us/op 1.21
altair processAttestation - 250000 vs - 7PWei normalcase 6.5363 ms/op 4.2163 ms/op 1.55
altair processAttestation - 250000 vs - 7PWei worstcase 8.8896 ms/op 6.1750 ms/op 1.44
altair processAttestation - setStatus - 1/6 committees join 279.08 us/op 177.22 us/op 1.57
altair processAttestation - setStatus - 1/3 committees join 558.82 us/op 344.18 us/op 1.62
altair processAttestation - setStatus - 1/2 committees join 746.22 us/op 491.84 us/op 1.52
altair processAttestation - setStatus - 2/3 committees join 1.0267 ms/op 642.79 us/op 1.60
altair processAttestation - setStatus - 4/5 committees join 1.3955 ms/op 910.70 us/op 1.53
altair processAttestation - setStatus - 100% committees join 1.7364 ms/op 1.0924 ms/op 1.59
altair processBlock - 250000 vs - 7PWei normalcase 34.395 ms/op 25.300 ms/op 1.36
altair processBlock - 250000 vs - 7PWei normalcase hashState 54.653 ms/op 38.984 ms/op 1.40
altair processBlock - 250000 vs - 7PWei worstcase 101.26 ms/op 80.408 ms/op 1.26
altair processBlock - 250000 vs - 7PWei worstcase hashState 128.39 ms/op 116.94 ms/op 1.10
phase0 processBlock - 250000 vs - 7PWei normalcase 5.2703 ms/op 3.4792 ms/op 1.51
phase0 processBlock - 250000 vs - 7PWei worstcase 62.422 ms/op 51.953 ms/op 1.20
altair processEth1Data - 250000 vs - 7PWei normalcase 1.4422 ms/op 871.57 us/op 1.65
Tree 40 250000 create 1.1467 s/op 867.78 ms/op 1.32
Tree 40 250000 get(125000) 375.00 ns/op 261.31 ns/op 1.44
Tree 40 250000 set(125000) 4.1991 us/op 2.4425 us/op 1.72
Tree 40 250000 toArray() 39.736 ms/op 28.389 ms/op 1.40
Tree 40 250000 iterate all - toArray() + loop 40.216 ms/op 28.538 ms/op 1.41
Tree 40 250000 iterate all - get(i) 147.75 ms/op 112.79 ms/op 1.31
MutableVector 250000 create 20.628 ms/op 14.442 ms/op 1.43
MutableVector 250000 get(125000) 16.006 ns/op 10.689 ns/op 1.50
MutableVector 250000 set(125000) 1.1933 us/op 634.06 ns/op 1.88
MutableVector 250000 toArray() 15.189 ms/op 6.7078 ms/op 2.26
MutableVector 250000 iterate all - toArray() + loop 8.3161 ms/op 6.8745 ms/op 1.21
MutableVector 250000 iterate all - get(i) 3.9149 ms/op 2.7269 ms/op 1.44
Array 250000 create 7.4472 ms/op 6.7593 ms/op 1.10
Array 250000 clone - spread 2.9284 ms/op 3.5649 ms/op 0.82
Array 250000 get(125000) 1.4710 ns/op 1.6020 ns/op 0.92
Array 250000 set(125000) 1.3840 ns/op 1.6170 ns/op 0.86
Array 250000 iterate all - loop 161.14 us/op 157.03 us/op 1.03
effectiveBalanceIncrements clone Uint8Array 300000 111.75 us/op 93.264 us/op 1.20
effectiveBalanceIncrements clone MutableVector 300000 888.00 ns/op 664.00 ns/op 1.34
effectiveBalanceIncrements rw all Uint8Array 300000 333.46 us/op 247.56 us/op 1.35
effectiveBalanceIncrements rw all MutableVector 300000 273.36 ms/op 149.53 ms/op 1.83
phase0 afterProcessEpoch - 250000 vs - 7PWei 213.65 ms/op 188.97 ms/op 1.13
phase0 beforeProcessEpoch - 250000 vs - 7PWei 85.780 ms/op 60.994 ms/op 1.41
altair processEpoch - mainnet_e81889 733.25 ms/op 559.39 ms/op 1.31
mainnet_e81889 - altair beforeProcessEpoch 166.04 ms/op 126.87 ms/op 1.31
mainnet_e81889 - altair processJustificationAndFinalization 67.131 us/op 17.725 us/op 3.79
mainnet_e81889 - altair processInactivityUpdates 12.484 ms/op 9.2526 ms/op 1.35
mainnet_e81889 - altair processRewardsAndPenalties 118.46 ms/op 83.029 ms/op 1.43
mainnet_e81889 - altair processRegistryUpdates 14.949 us/op 2.9210 us/op 5.12
mainnet_e81889 - altair processSlashings 3.9590 us/op 724.00 ns/op 5.47
mainnet_e81889 - altair processEth1DataReset 3.8360 us/op 720.00 ns/op 5.33
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.6437 ms/op 2.1707 ms/op 1.22
mainnet_e81889 - altair processSlashingsReset 27.664 us/op 5.2680 us/op 5.25
mainnet_e81889 - altair processRandaoMixesReset 25.045 us/op 4.5020 us/op 5.56
mainnet_e81889 - altair processHistoricalRootsUpdate 4.2860 us/op 751.00 ns/op 5.71
mainnet_e81889 - altair processParticipationFlagUpdates 16.290 us/op 2.1380 us/op 7.62
mainnet_e81889 - altair processSyncCommitteeUpdates 3.1760 us/op 682.00 ns/op 4.66
mainnet_e81889 - altair afterProcessEpoch 228.41 ms/op 201.10 ms/op 1.14
phase0 processEpoch - mainnet_e58758 669.95 ms/op 503.84 ms/op 1.33
mainnet_e58758 - phase0 beforeProcessEpoch 311.30 ms/op 187.99 ms/op 1.66
mainnet_e58758 - phase0 processJustificationAndFinalization 70.550 us/op 17.826 us/op 3.96
mainnet_e58758 - phase0 processRewardsAndPenalties 159.01 ms/op 112.09 ms/op 1.42
mainnet_e58758 - phase0 processRegistryUpdates 35.180 us/op 8.8050 us/op 4.00
mainnet_e58758 - phase0 processSlashings 3.5460 us/op 644.00 ns/op 5.51
mainnet_e58758 - phase0 processEth1DataReset 3.6710 us/op 627.00 ns/op 5.85
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 3.2449 ms/op 1.9150 ms/op 1.69
mainnet_e58758 - phase0 processSlashingsReset 15.463 us/op 3.7720 us/op 4.10
mainnet_e58758 - phase0 processRandaoMixesReset 25.800 us/op 4.5030 us/op 5.73
mainnet_e58758 - phase0 processHistoricalRootsUpdate 3.8500 us/op 761.00 ns/op 5.06
mainnet_e58758 - phase0 processParticipationRecordUpdates 25.712 us/op 4.0670 us/op 6.32
mainnet_e58758 - phase0 afterProcessEpoch 192.29 ms/op 164.98 ms/op 1.17
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.6261 ms/op 1.9397 ms/op 1.35
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.9813 ms/op 2.1872 ms/op 1.36
altair processInactivityUpdates - 250000 normalcase 58.608 ms/op 37.131 ms/op 1.58
altair processInactivityUpdates - 250000 worstcase 74.463 ms/op 44.121 ms/op 1.69
phase0 processRegistryUpdates - 250000 normalcase 26.519 us/op 6.8870 us/op 3.85
phase0 processRegistryUpdates - 250000 badcase_full_deposits 601.32 us/op 372.22 us/op 1.62
phase0 processRegistryUpdates - 250000 worstcase 0.5 285.29 ms/op 191.17 ms/op 1.49
altair processRewardsAndPenalties - 250000 normalcase 119.32 ms/op 110.13 ms/op 1.08
altair processRewardsAndPenalties - 250000 worstcase 118.78 ms/op 92.466 ms/op 1.28
phase0 getAttestationDeltas - 250000 normalcase 16.921 ms/op 12.855 ms/op 1.32
phase0 getAttestationDeltas - 250000 worstcase 16.749 ms/op 12.687 ms/op 1.32
phase0 processSlashings - 250000 worstcase 7.4397 ms/op 5.2954 ms/op 1.40
altair processSyncCommitteeUpdates - 250000 389.30 ms/op 297.71 ms/op 1.31
BeaconState.hashTreeRoot - No change 619.00 ns/op 529.00 ns/op 1.17
BeaconState.hashTreeRoot - 1 full validator 76.465 us/op 65.836 us/op 1.16
BeaconState.hashTreeRoot - 32 full validator 769.83 us/op 649.94 us/op 1.18
BeaconState.hashTreeRoot - 512 full validator 8.5171 ms/op 7.3551 ms/op 1.16
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 105.50 us/op 81.644 us/op 1.29
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.4733 ms/op 1.1732 ms/op 1.26
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 18.253 ms/op 15.433 ms/op 1.18
BeaconState.hashTreeRoot - 1 balances 77.763 us/op 63.652 us/op 1.22
BeaconState.hashTreeRoot - 32 balances 697.24 us/op 582.54 us/op 1.20
BeaconState.hashTreeRoot - 512 balances 6.1827 ms/op 5.3470 ms/op 1.16
BeaconState.hashTreeRoot - 250000 balances 119.46 ms/op 94.808 ms/op 1.26
aggregationBits - 2048 els - zipIndexesInBitList 45.466 us/op 30.735 us/op 1.48
regular array get 100000 times 64.821 us/op 65.255 us/op 0.99
wrappedArray get 100000 times 63.695 us/op 60.709 us/op 1.05
arrayWithProxy get 100000 times 39.166 ms/op 28.236 ms/op 1.39
ssz.Root.equals 656.00 ns/op 512.00 ns/op 1.28
byteArrayEquals 715.00 ns/op 532.00 ns/op 1.34
shuffle list - 16384 els 13.371 ms/op 11.394 ms/op 1.17
shuffle list - 250000 els 189.65 ms/op 170.79 ms/op 1.11
processSlot - 1 slots 18.258 us/op 13.207 us/op 1.38
processSlot - 32 slots 2.4674 ms/op 1.9592 ms/op 1.26
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 437.25 us/op 428.98 us/op 1.02
getCommitteeAssignments - req 1 vs - 250000 vc 6.1863 ms/op 5.4539 ms/op 1.13
getCommitteeAssignments - req 100 vs - 250000 vc 8.6241 ms/op 8.0161 ms/op 1.08
getCommitteeAssignments - req 1000 vs - 250000 vc 9.7558 ms/op 8.6014 ms/op 1.13
computeProposers - vc 250000 22.813 ms/op 19.138 ms/op 1.19
computeEpochShuffling - vc 250000 194.33 ms/op 173.83 ms/op 1.12
getNextSyncCommittee - vc 250000 383.09 ms/op 292.05 ms/op 1.31

by benchmarkbot/action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants