Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add metrics to LightClientServer #4372

Merged
merged 1 commit into from Aug 6, 2022
Merged

Add metrics to LightClientServer #4372

merged 1 commit into from Aug 6, 2022

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Aug 6, 2022 •
edited

Motivation

After running this implementation of the LightClientServer there are many more edge cases than anticipated, throwing too many errors in the logs.

This is bad for UX, as users must not learn to ignore errors.

Description

Instead of throwing, record un-wanted incidents (like attestedData not available) as metrics, and log to debug. We would still have all data available to understand latter under which conditions that happens, which is even better than just throwing errors.

Closes #3788

@dapplion dapplion requested a review from a team as a code owner August 6, 2022 07:30
g11tech
g11tech approved these changes Aug 6, 2022
View reviewed changes
Copy link
Contributor

@g11tech g11tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀

@github-actions
Copy link
Contributor

github-actions bot commented Aug 6, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: e38bcca Previous: eeb333a Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.2761 ms/op 2.0309 ms/op 1.12
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 82.338 us/op 65.797 us/op 1.25
BLS verify - blst-native 2.6527 ms/op 2.1671 ms/op 1.22
BLS verifyMultipleSignatures 3 - blst-native 5.5932 ms/op 4.4731 ms/op 1.25
BLS verifyMultipleSignatures 8 - blst-native 11.873 ms/op 9.6694 ms/op 1.23
BLS verifyMultipleSignatures 32 - blst-native 42.407 ms/op 35.164 ms/op 1.21
BLS aggregatePubkeys 32 - blst-native 58.325 us/op 46.611 us/op 1.25
BLS aggregatePubkeys 128 - blst-native 229.20 us/op 182.30 us/op 1.26
getAttestationsForBlock 196.44 ms/op 150.22 ms/op 1.31
isKnown best case - 1 super set check 499.00 ns/op 483.00 ns/op 1.03
isKnown normal case - 2 super set checks 506.00 ns/op 475.00 ns/op 1.07
isKnown worse case - 16 super set checks 475.00 ns/op 471.00 ns/op 1.01
CheckpointStateCache - add get delete 10.737 us/op 9.1370 us/op 1.18
validate gossip signedAggregateAndProof - struct 6.2468 ms/op 5.0207 ms/op 1.24
validate gossip attestation - struct 2.9561 ms/op 2.3813 ms/op 1.24
altair verifyImport mainnet_s3766816:31 10.402 s/op 8.7081 s/op 1.19
pickEth1Vote - no votes 2.5923 ms/op 2.1617 ms/op 1.20
pickEth1Vote - max votes 24.739 ms/op 21.007 ms/op 1.18
pickEth1Vote - Eth1Data hashTreeRoot value x2048 14.013 ms/op 12.806 ms/op 1.09
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 23.784 ms/op 21.131 ms/op 1.13
pickEth1Vote - Eth1Data fastSerialize value x2048 1.9299 ms/op 1.5269 ms/op 1.26
pickEth1Vote - Eth1Data fastSerialize tree x2048 16.985 ms/op 14.772 ms/op 1.15
bytes32 toHexString 1.3370 us/op 1.1190 us/op 1.19
bytes32 Buffer.toString(hex) 905.00 ns/op 852.00 ns/op 1.06
bytes32 Buffer.toString(hex) from Uint8Array 1.0840 us/op 1.0500 us/op 1.03
bytes32 Buffer.toString(hex) + 0x 865.00 ns/op 847.00 ns/op 1.02
Object access 1 prop 0.45100 ns/op 0.38300 ns/op 1.18
Map access 1 prop 0.33100 ns/op 0.34200 ns/op 0.97
Object get x1000 16.417 ns/op 11.025 ns/op 1.49
Map get x1000 1.0250 ns/op 1.0480 ns/op 0.98
Object set x1000 125.55 ns/op 77.801 ns/op 1.61
Map set x1000 84.971 ns/op 51.096 ns/op 1.66
Return object 10000 times 0.42930 ns/op 0.44680 ns/op 0.96
Throw Error 10000 times 6.8216 us/op 6.1034 us/op 1.12
enrSubnets - fastDeserialize 64 bits 3.4110 us/op 2.8000 us/op 1.22
enrSubnets - ssz BitVector 64 bits 874.00 ns/op 849.00 ns/op 1.03
enrSubnets - fastDeserialize 4 bits 480.00 ns/op 492.00 ns/op 0.98
enrSubnets - ssz BitVector 4 bits 910.00 ns/op 852.00 ns/op 1.07
prioritizePeers score -10:0 att 32-0.1 sync 2-0 144.75 us/op 87.745 us/op 1.65
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 159.15 us/op 127.19 us/op 1.25
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 296.57 us/op 228.78 us/op 1.30
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 629.22 us/op 335.30 us/op 1.88
prioritizePeers score 0:0 att 64-1 sync 4-1 592.43 us/op 408.56 us/op 1.45
RateTracker 1000000 limit, 1 obj count per request 211.42 ns/op 194.30 ns/op 1.09
RateTracker 1000000 limit, 2 obj count per request 159.75 ns/op 140.80 ns/op 1.13
RateTracker 1000000 limit, 4 obj count per request 130.93 ns/op 117.59 ns/op 1.11
RateTracker 1000000 limit, 8 obj count per request 125.63 ns/op 102.11 ns/op 1.23
RateTracker with prune 5.8850 us/op 4.5870 us/op 1.28
array of 16000 items push then shift 5.2492 us/op 51.584 us/op 0.10
LinkedList of 16000 items push then shift 19.050 ns/op 12.811 ns/op 1.49
array of 16000 items push then pop 283.97 ns/op 228.80 ns/op 1.24
LinkedList of 16000 items push then pop 19.093 ns/op 12.086 ns/op 1.58
array of 24000 items push then shift 7.3607 us/op 77.338 us/op 0.10
LinkedList of 24000 items push then shift 20.036 ns/op 13.285 ns/op 1.51
array of 24000 items push then pop 230.70 ns/op 198.69 ns/op 1.16
LinkedList of 24000 items push then pop 17.382 ns/op 12.167 ns/op 1.43
intersect bitArray bitLen 8 12.670 ns/op 10.877 ns/op 1.16
intersect array and set length 8 202.56 ns/op 144.58 ns/op 1.40
intersect bitArray bitLen 128 69.247 ns/op 58.696 ns/op 1.18
intersect array and set length 128 2.5835 us/op 1.8506 us/op 1.40
Buffer.concat 32 items 2.1460 ns/op 1.8660 ns/op 1.15
pass gossip attestations to forkchoice per slot 8.9792 ms/op 3.1758 ms/op 2.83
computeDeltas 3.5536 ms/op 3.2829 ms/op 1.08
computeProposerBoostScoreFromBalances 864.58 us/op 810.91 us/op 1.07
altair processAttestation - 250000 vs - 7PWei normalcase 4.9465 ms/op 3.6254 ms/op 1.36
altair processAttestation - 250000 vs - 7PWei worstcase 8.0391 ms/op 5.2707 ms/op 1.53
altair processAttestation - setStatus - 1/6 committees join 250.89 us/op 176.33 us/op 1.42
altair processAttestation - setStatus - 1/3 committees join 470.84 us/op 341.83 us/op 1.38
altair processAttestation - setStatus - 1/2 committees join 672.95 us/op 493.44 us/op 1.36
altair processAttestation - setStatus - 2/3 committees join 885.39 us/op 652.99 us/op 1.36
altair processAttestation - setStatus - 4/5 committees join 1.2555 ms/op 903.33 us/op 1.39
altair processAttestation - setStatus - 100% committees join 1.4774 ms/op 1.0921 ms/op 1.35
altair processBlock - 250000 vs - 7PWei normalcase 30.830 ms/op 24.866 ms/op 1.24
altair processBlock - 250000 vs - 7PWei normalcase hashState 43.245 ms/op 39.350 ms/op 1.10
altair processBlock - 250000 vs - 7PWei worstcase 96.710 ms/op 83.543 ms/op 1.16
altair processBlock - 250000 vs - 7PWei worstcase hashState 121.69 ms/op 110.54 ms/op 1.10
phase0 processBlock - 250000 vs - 7PWei normalcase 4.8809 ms/op 3.7716 ms/op 1.29
phase0 processBlock - 250000 vs - 7PWei worstcase 63.522 ms/op 52.525 ms/op 1.21
altair processEth1Data - 250000 vs - 7PWei normalcase 1.0719 ms/op 733.62 us/op 1.46
Tree 40 250000 create 996.41 ms/op 719.30 ms/op 1.39
Tree 40 250000 get(125000) 321.13 ns/op 236.74 ns/op 1.36
Tree 40 250000 set(125000) 3.0508 us/op 2.2366 us/op 1.36
Tree 40 250000 toArray() 34.798 ms/op 27.773 ms/op 1.25
Tree 40 250000 iterate all - toArray() + loop 35.018 ms/op 27.887 ms/op 1.26
Tree 40 250000 iterate all - get(i) 131.84 ms/op 109.53 ms/op 1.20
MutableVector 250000 create 17.491 ms/op 13.344 ms/op 1.31
MutableVector 250000 get(125000) 15.419 ns/op 11.032 ns/op 1.40
MutableVector 250000 set(125000) 705.04 ns/op 567.49 ns/op 1.24
MutableVector 250000 toArray() 7.2272 ms/op 6.1487 ms/op 1.18
MutableVector 250000 iterate all - toArray() + loop 7.1298 ms/op 6.3909 ms/op 1.12
MutableVector 250000 iterate all - get(i) 3.3040 ms/op 2.6894 ms/op 1.23
Array 250000 create 6.6222 ms/op 6.3634 ms/op 1.04
Array 250000 clone - spread 2.8279 ms/op 2.3906 ms/op 1.18
Array 250000 get(125000) 1.2510 ns/op 1.1340 ns/op 1.10
Array 250000 set(125000) 1.3250 ns/op 1.1270 ns/op 1.18
Array 250000 iterate all - loop 134.00 us/op 151.01 us/op 0.89
effectiveBalanceIncrements clone Uint8Array 300000 95.260 us/op 35.902 us/op 2.65
effectiveBalanceIncrements clone MutableVector 300000 799.00 ns/op 690.00 ns/op 1.16
effectiveBalanceIncrements rw all Uint8Array 300000 292.20 us/op 247.35 us/op 1.18
effectiveBalanceIncrements rw all MutableVector 300000 218.23 ms/op 136.63 ms/op 1.60
phase0 afterProcessEpoch - 250000 vs - 7PWei 193.16 ms/op 186.36 ms/op 1.04
phase0 beforeProcessEpoch - 250000 vs - 7PWei 76.369 ms/op 58.419 ms/op 1.31
altair processEpoch - mainnet_e81889 654.18 ms/op 534.03 ms/op 1.22
mainnet_e81889 - altair beforeProcessEpoch 154.71 ms/op 118.23 ms/op 1.31
mainnet_e81889 - altair processJustificationAndFinalization 69.548 us/op 16.415 us/op 4.24
mainnet_e81889 - altair processInactivityUpdates 10.997 ms/op 8.7458 ms/op 1.26
mainnet_e81889 - altair processRewardsAndPenalties 105.01 ms/op 120.46 ms/op 0.87
mainnet_e81889 - altair processRegistryUpdates 15.619 us/op 2.7170 us/op 5.75
mainnet_e81889 - altair processSlashings 4.6020 us/op 512.00 ns/op 8.99
mainnet_e81889 - altair processEth1DataReset 4.1670 us/op 621.00 ns/op 6.71
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.4172 ms/op 2.1033 ms/op 1.15
mainnet_e81889 - altair processSlashingsReset 23.165 us/op 4.3820 us/op 5.29
mainnet_e81889 - altair processRandaoMixesReset 28.042 us/op 4.5630 us/op 6.15
mainnet_e81889 - altair processHistoricalRootsUpdate 5.1870 us/op 690.00 ns/op 7.52
mainnet_e81889 - altair processParticipationFlagUpdates 13.981 us/op 8.2120 us/op 1.70
mainnet_e81889 - altair processSyncCommitteeUpdates 3.3710 us/op 799.00 ns/op 4.22
mainnet_e81889 - altair afterProcessEpoch 208.14 ms/op 217.65 ms/op 0.96
phase0 processEpoch - mainnet_e58758 613.15 ms/op 631.10 ms/op 0.97
mainnet_e58758 - phase0 beforeProcessEpoch 275.23 ms/op 223.45 ms/op 1.23
mainnet_e58758 - phase0 processJustificationAndFinalization 60.930 us/op 16.341 us/op 3.73
mainnet_e58758 - phase0 processRewardsAndPenalties 86.995 ms/op 128.84 ms/op 0.68
mainnet_e58758 - phase0 processRegistryUpdates 34.291 us/op 7.9240 us/op 4.33
mainnet_e58758 - phase0 processSlashings 3.6780 us/op 561.00 ns/op 6.56
mainnet_e58758 - phase0 processEth1DataReset 4.8150 us/op 586.00 ns/op 8.22
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.0675 ms/op 1.9525 ms/op 1.06
mainnet_e58758 - phase0 processSlashingsReset 19.503 us/op 4.2680 us/op 4.57
mainnet_e58758 - phase0 processRandaoMixesReset 31.003 us/op 4.7450 us/op 6.53
mainnet_e58758 - phase0 processHistoricalRootsUpdate 5.0140 us/op 671.00 ns/op 7.47
mainnet_e58758 - phase0 processParticipationRecordUpdates 24.864 us/op 3.9040 us/op 6.37
mainnet_e58758 - phase0 afterProcessEpoch 182.93 ms/op 162.05 ms/op 1.13
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.3663 ms/op 2.0050 ms/op 1.18
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.6120 ms/op 2.2349 ms/op 1.17
altair processInactivityUpdates - 250000 normalcase 48.693 ms/op 50.598 ms/op 0.96
altair processInactivityUpdates - 250000 worstcase 51.950 ms/op 51.535 ms/op 1.01
phase0 processRegistryUpdates - 250000 normalcase 31.104 us/op 6.5030 us/op 4.78
phase0 processRegistryUpdates - 250000 badcase_full_deposits 488.19 us/op 363.53 us/op 1.34
phase0 processRegistryUpdates - 250000 worstcase 0.5 270.03 ms/op 227.99 ms/op 1.18
altair processRewardsAndPenalties - 250000 normalcase 146.83 ms/op 133.81 ms/op 1.10
altair processRewardsAndPenalties - 250000 worstcase 141.25 ms/op 139.67 ms/op 1.01
phase0 getAttestationDeltas - 250000 normalcase 13.995 ms/op 11.899 ms/op 1.18
phase0 getAttestationDeltas - 250000 worstcase 14.431 ms/op 11.725 ms/op 1.23
phase0 processSlashings - 250000 worstcase 6.1148 ms/op 5.1307 ms/op 1.19
altair processSyncCommitteeUpdates - 250000 335.38 ms/op 295.23 ms/op 1.14
BeaconState.hashTreeRoot - No change 588.00 ns/op 516.00 ns/op 1.14
BeaconState.hashTreeRoot - 1 full validator 77.746 us/op 71.457 us/op 1.09
BeaconState.hashTreeRoot - 32 full validator 874.54 us/op 737.57 us/op 1.19
BeaconState.hashTreeRoot - 512 full validator 8.4638 ms/op 8.2199 ms/op 1.03
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 100.87 us/op 81.076 us/op 1.24
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.4790 ms/op 1.2588 ms/op 1.17
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 19.550 ms/op 18.843 ms/op 1.04
BeaconState.hashTreeRoot - 1 balances 76.320 us/op 64.585 us/op 1.18
BeaconState.hashTreeRoot - 32 balances 782.50 us/op 725.12 us/op 1.08
BeaconState.hashTreeRoot - 512 balances 8.3396 ms/op 10.501 ms/op 0.79
BeaconState.hashTreeRoot - 250000 balances 116.83 ms/op 109.78 ms/op 1.06
aggregationBits - 2048 els - zipIndexesInBitList 34.256 us/op 26.153 us/op 1.31
regular array get 100000 times 59.847 us/op 65.111 us/op 0.92
wrappedArray get 100000 times 57.992 us/op 64.688 us/op 0.90
arrayWithProxy get 100000 times 33.269 ms/op 27.565 ms/op 1.21
ssz.Root.equals 577.00 ns/op 541.00 ns/op 1.07
byteArrayEquals 536.00 ns/op 528.00 ns/op 1.02
shuffle list - 16384 els 11.864 ms/op 11.463 ms/op 1.03
shuffle list - 250000 els 176.22 ms/op 165.16 ms/op 1.07
processSlot - 1 slots 17.047 us/op 13.321 us/op 1.28
processSlot - 32 slots 2.2562 ms/op 1.9314 ms/op 1.17
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 561.56 us/op 389.64 us/op 1.44
getCommitteeAssignments - req 1 vs - 250000 vc 5.4450 ms/op 5.4601 ms/op 1.00
getCommitteeAssignments - req 100 vs - 250000 vc 7.9671 ms/op 7.9619 ms/op 1.00
getCommitteeAssignments - req 1000 vs - 250000 vc 8.0054 ms/op 8.5932 ms/op 0.93
computeProposers - vc 250000 19.575 ms/op 18.870 ms/op 1.04
computeEpochShuffling - vc 250000 181.57 ms/op 170.00 ms/op 1.07
getNextSyncCommittee - vc 250000 319.75 ms/op 283.61 ms/op 1.13

by benchmarkbot/action

@g11tech g11tech merged commit b8029a3 into unstable Aug 6, 2022
@g11tech g11tech deleted the dapplion/metrics-lightclient-server branch August 6, 2022 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@g11tech g11tech g11tech approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Error: attestedData not available
2 participants
@dapplion @g11tech