Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Eth1 and builder metrics #4344

Merged
merged 1 commit into from Jul 25, 2022
Merged

Add Eth1 and builder metrics #4344

merged 1 commit into from Jul 25, 2022

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Jul 25, 2022

Add Eth1 and builder metrics in grafana dashboard

Eth1 stats

(below scrape stats)

image
image

Builder Metrics

(below Block production stats)

image

More metrics PRs will follow for

  • Merge stats
  • execution engine stats (this is pending on the retry PR merge)

@g11tech g11tech requested a review from a team as a code owner July 25, 2022 07:47
@g11tech g11tech enabled auto-merge (squash) July 25, 2022 07:47
@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 5d4b8c4 Previous: 0ae8f92 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.2807 ms/op 2.2069 ms/op 1.03
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 85.530 us/op 73.411 us/op 1.17
BLS verify - blst-native 2.7387 ms/op 1.8553 ms/op 1.48
BLS verifyMultipleSignatures 3 - blst-native 5.6292 ms/op 3.8060 ms/op 1.48
BLS verifyMultipleSignatures 8 - blst-native 12.029 ms/op 8.2222 ms/op 1.46
BLS verifyMultipleSignatures 32 - blst-native 43.765 ms/op 29.732 ms/op 1.47
BLS aggregatePubkeys 32 - blst-native 57.736 us/op 39.103 us/op 1.48
BLS aggregatePubkeys 128 - blst-native 229.53 us/op 152.79 us/op 1.50
getAttestationsForBlock 54.296 ms/op 57.313 ms/op 0.95
isKnown best case - 1 super set check 523.00 ns/op 465.00 ns/op 1.12
isKnown normal case - 2 super set checks 479.00 ns/op 426.00 ns/op 1.12
isKnown worse case - 16 super set checks 476.00 ns/op 435.00 ns/op 1.09
CheckpointStateCache - add get delete 11.035 us/op 8.6220 us/op 1.28
validate gossip signedAggregateAndProof - struct 6.3239 ms/op 4.2757 ms/op 1.48
validate gossip attestation - struct 2.9737 ms/op 2.0385 ms/op 1.46
altair verifyImport mainnet_s3766816:31 10.498 s/op 8.9159 s/op 1.18
pickEth1Vote - no votes 2.5353 ms/op 2.1737 ms/op 1.17
pickEth1Vote - max votes 25.195 ms/op 21.966 ms/op 1.15
pickEth1Vote - Eth1Data hashTreeRoot value x2048 14.784 ms/op 11.578 ms/op 1.28
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 23.668 ms/op 21.214 ms/op 1.12
pickEth1Vote - Eth1Data fastSerialize value x2048 1.8147 ms/op 1.5754 ms/op 1.15
pickEth1Vote - Eth1Data fastSerialize tree x2048 16.965 ms/op 16.450 ms/op 1.03
bytes32 toHexString 1.2890 us/op 1.1350 us/op 1.14
bytes32 Buffer.toString(hex) 839.00 ns/op 712.00 ns/op 1.18
bytes32 Buffer.toString(hex) from Uint8Array 1.1480 us/op 955.00 ns/op 1.20
bytes32 Buffer.toString(hex) + 0x 844.00 ns/op 703.00 ns/op 1.20
Object access 1 prop 0.47400 ns/op 0.38800 ns/op 1.22
Map access 1 prop 0.32600 ns/op 0.30300 ns/op 1.08
Object get x1000 15.489 ns/op 17.687 ns/op 0.88
Map get x1000 0.97400 ns/op 0.97600 ns/op 1.00
Object set x1000 119.48 ns/op 134.51 ns/op 0.89
Map set x1000 87.713 ns/op 71.932 ns/op 1.22
Return object 10000 times 0.40860 ns/op 0.38010 ns/op 1.07
Throw Error 10000 times 6.9909 us/op 5.9951 us/op 1.17
enrSubnets - fastDeserialize 64 bits 3.5410 us/op 2.9750 us/op 1.19
enrSubnets - ssz BitVector 64 bits 979.00 ns/op 762.00 ns/op 1.28
enrSubnets - fastDeserialize 4 bits 511.00 ns/op 403.00 ns/op 1.27
enrSubnets - ssz BitVector 4 bits 943.00 ns/op 777.00 ns/op 1.21
prioritizePeers score -10:0 att 32-0.1 sync 2-0 129.88 us/op 105.85 us/op 1.23
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 165.38 us/op 123.80 us/op 1.34
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 308.65 us/op 226.02 us/op 1.37
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 617.11 us/op 479.45 us/op 1.29
prioritizePeers score 0:0 att 64-1 sync 4-1 679.49 us/op 464.74 us/op 1.46
RateTracker 1000000 limit, 1 obj count per request 233.81 ns/op 193.57 ns/op 1.21
RateTracker 1000000 limit, 2 obj count per request 179.88 ns/op 146.85 ns/op 1.22
RateTracker 1000000 limit, 4 obj count per request 160.21 ns/op 123.29 ns/op 1.30
RateTracker 1000000 limit, 8 obj count per request 139.76 ns/op 110.34 ns/op 1.27
RateTracker with prune 6.6430 us/op 5.0400 us/op 1.32
array of 16000 items push then shift 5.7477 us/op 3.0912 us/op 1.86
LinkedList of 16000 items push then shift 19.086 ns/op 28.392 ns/op 0.67
array of 16000 items push then pop 288.28 ns/op 289.37 ns/op 1.00
LinkedList of 16000 items push then pop 17.722 ns/op 23.191 ns/op 0.76
array of 24000 items push then shift 8.1062 us/op 4.5687 us/op 1.77
LinkedList of 24000 items push then shift 21.377 ns/op 28.303 ns/op 0.76
array of 24000 items push then pop 247.28 ns/op 220.88 ns/op 1.12
LinkedList of 24000 items push then pop 17.964 ns/op 23.659 ns/op 0.76
intersect bitArray bitLen 8 12.666 ns/op 11.772 ns/op 1.08
intersect array and set length 8 229.09 ns/op 179.46 ns/op 1.28
intersect bitArray bitLen 128 87.999 ns/op 62.155 ns/op 1.42
intersect array and set length 128 2.9215 us/op 2.4470 us/op 1.19
pass gossip attestations to forkchoice per slot 3.6299 ms/op 3.2563 ms/op 1.11
computeDeltas 3.7414 ms/op 3.6167 ms/op 1.03
computeProposerBoostScoreFromBalances 920.37 us/op 908.65 us/op 1.01
altair processAttestation - 250000 vs - 7PWei normalcase 5.6086 ms/op 4.0307 ms/op 1.39
altair processAttestation - 250000 vs - 7PWei worstcase 8.2178 ms/op 6.2276 ms/op 1.32
altair processAttestation - setStatus - 1/6 committees join 270.06 us/op 207.36 us/op 1.30
altair processAttestation - setStatus - 1/3 committees join 483.35 us/op 395.35 us/op 1.22
altair processAttestation - setStatus - 1/2 committees join 713.70 us/op 552.89 us/op 1.29
altair processAttestation - setStatus - 2/3 committees join 891.15 us/op 708.37 us/op 1.26
altair processAttestation - setStatus - 4/5 committees join 1.2540 ms/op 997.02 us/op 1.26
altair processAttestation - setStatus - 100% committees join 1.5018 ms/op 1.1743 ms/op 1.28
altair processBlock - 250000 vs - 7PWei normalcase 31.124 ms/op 29.532 ms/op 1.05
altair processBlock - 250000 vs - 7PWei normalcase hashState 49.483 ms/op 36.292 ms/op 1.36
altair processBlock - 250000 vs - 7PWei worstcase 107.85 ms/op 85.022 ms/op 1.27
altair processBlock - 250000 vs - 7PWei worstcase hashState 131.37 ms/op 102.13 ms/op 1.29
phase0 processBlock - 250000 vs - 7PWei normalcase 4.6913 ms/op 5.2707 ms/op 0.89
phase0 processBlock - 250000 vs - 7PWei worstcase 67.425 ms/op 49.848 ms/op 1.35
altair processEth1Data - 250000 vs - 7PWei normalcase 1.3075 ms/op 853.88 us/op 1.53
Tree 40 250000 create 1.1679 s/op 843.14 ms/op 1.39
Tree 40 250000 get(125000) 352.61 ns/op 290.10 ns/op 1.22
Tree 40 250000 set(125000) 3.5341 us/op 2.6539 us/op 1.33
Tree 40 250000 toArray() 36.713 ms/op 33.343 ms/op 1.10
Tree 40 250000 iterate all - toArray() + loop 36.811 ms/op 33.799 ms/op 1.09
Tree 40 250000 iterate all - get(i) 133.55 ms/op 111.82 ms/op 1.19
MutableVector 250000 create 17.858 ms/op 15.529 ms/op 1.15
MutableVector 250000 get(125000) 15.125 ns/op 13.316 ns/op 1.14
MutableVector 250000 set(125000) 973.86 ns/op 658.61 ns/op 1.48
MutableVector 250000 toArray() 8.2967 ms/op 7.9899 ms/op 1.04
MutableVector 250000 iterate all - toArray() + loop 8.4711 ms/op 8.0912 ms/op 1.05
MutableVector 250000 iterate all - get(i) 3.6893 ms/op 3.4401 ms/op 1.07
Array 250000 create 8.1578 ms/op 7.2927 ms/op 1.12
Array 250000 clone - spread 3.3400 ms/op 3.2346 ms/op 1.03
Array 250000 get(125000) 1.4840 ns/op 1.1630 ns/op 1.28
Array 250000 set(125000) 1.5040 ns/op 1.1610 ns/op 1.30
Array 250000 iterate all - loop 147.64 us/op 169.24 us/op 0.87
effectiveBalanceIncrements clone Uint8Array 300000 197.67 us/op 72.816 us/op 2.71
effectiveBalanceIncrements clone MutableVector 300000 818.00 ns/op 801.00 ns/op 1.02
effectiveBalanceIncrements rw all Uint8Array 300000 304.22 us/op 252.72 us/op 1.20
effectiveBalanceIncrements rw all MutableVector 300000 225.49 ms/op 180.85 ms/op 1.25
phase0 afterProcessEpoch - 250000 vs - 7PWei 217.34 ms/op 182.19 ms/op 1.19
phase0 beforeProcessEpoch - 250000 vs - 7PWei 77.105 ms/op 79.382 ms/op 0.97
altair processEpoch - mainnet_e81889 645.65 ms/op 603.73 ms/op 1.07
mainnet_e81889 - altair beforeProcessEpoch 155.77 ms/op 174.59 ms/op 0.89
mainnet_e81889 - altair processJustificationAndFinalization 74.530 us/op 37.642 us/op 1.98
mainnet_e81889 - altair processInactivityUpdates 11.285 ms/op 12.008 ms/op 0.94
mainnet_e81889 - altair processRewardsAndPenalties 202.57 ms/op 101.24 ms/op 2.00
mainnet_e81889 - altair processRegistryUpdates 17.196 us/op 5.1430 us/op 3.34
mainnet_e81889 - altair processSlashings 4.4280 us/op 1.3590 us/op 3.26
mainnet_e81889 - altair processEth1DataReset 4.7010 us/op 1.1900 us/op 3.95
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.7245 ms/op 2.5565 ms/op 1.07
mainnet_e81889 - altair processSlashingsReset 28.253 us/op 8.5230 us/op 3.31
mainnet_e81889 - altair processRandaoMixesReset 25.713 us/op 8.5030 us/op 3.02
mainnet_e81889 - altair processHistoricalRootsUpdate 4.9280 us/op 1.2170 us/op 4.05
mainnet_e81889 - altair processParticipationFlagUpdates 17.920 us/op 3.9660 us/op 4.52
mainnet_e81889 - altair processSyncCommitteeUpdates 3.4320 us/op 1.1930 us/op 2.88
mainnet_e81889 - altair afterProcessEpoch 202.63 ms/op 195.12 ms/op 1.04
phase0 processEpoch - mainnet_e58758 686.02 ms/op 556.60 ms/op 1.23
mainnet_e58758 - phase0 beforeProcessEpoch 295.19 ms/op 249.20 ms/op 1.18
mainnet_e58758 - phase0 processJustificationAndFinalization 65.161 us/op 31.956 us/op 2.04
mainnet_e58758 - phase0 processRewardsAndPenalties 143.38 ms/op 145.59 ms/op 0.98
mainnet_e58758 - phase0 processRegistryUpdates 37.492 us/op 13.735 us/op 2.73
mainnet_e58758 - phase0 processSlashings 3.5110 us/op 1.4320 us/op 2.45
mainnet_e58758 - phase0 processEth1DataReset 3.3460 us/op 1.3590 us/op 2.46
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.8044 ms/op 2.0327 ms/op 1.38
mainnet_e58758 - phase0 processSlashingsReset 16.930 us/op 7.1520 us/op 2.37
mainnet_e58758 - phase0 processRandaoMixesReset 24.523 us/op 7.5720 us/op 3.24
mainnet_e58758 - phase0 processHistoricalRootsUpdate 4.2470 us/op 1.3600 us/op 3.12
mainnet_e58758 - phase0 processParticipationRecordUpdates 23.027 us/op 7.4340 us/op 3.10
mainnet_e58758 - phase0 afterProcessEpoch 177.14 ms/op 159.65 ms/op 1.11
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.5323 ms/op 2.7437 ms/op 0.92
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.7550 ms/op 3.1755 ms/op 0.87
altair processInactivityUpdates - 250000 normalcase 48.719 ms/op 42.883 ms/op 1.14
altair processInactivityUpdates - 250000 worstcase 61.103 ms/op 53.668 ms/op 1.14
phase0 processRegistryUpdates - 250000 normalcase 27.505 us/op 13.133 us/op 2.09
phase0 processRegistryUpdates - 250000 badcase_full_deposits 479.92 us/op 557.58 us/op 0.86
phase0 processRegistryUpdates - 250000 worstcase 0.5 251.47 ms/op 226.83 ms/op 1.11
altair processRewardsAndPenalties - 250000 normalcase 148.06 ms/op 134.22 ms/op 1.10
altair processRewardsAndPenalties - 250000 worstcase 100.16 ms/op 92.208 ms/op 1.09
phase0 getAttestationDeltas - 250000 normalcase 15.571 ms/op 13.485 ms/op 1.15
phase0 getAttestationDeltas - 250000 worstcase 16.201 ms/op 13.930 ms/op 1.16
phase0 processSlashings - 250000 worstcase 7.3495 ms/op 5.5093 ms/op 1.33
altair processSyncCommitteeUpdates - 250000 342.45 ms/op 282.34 ms/op 1.21
BeaconState.hashTreeRoot - No change 559.00 ns/op 465.00 ns/op 1.20
BeaconState.hashTreeRoot - 1 full validator 75.948 us/op 63.250 us/op 1.20
BeaconState.hashTreeRoot - 32 full validator 833.85 us/op 702.08 us/op 1.19
BeaconState.hashTreeRoot - 512 full validator 7.7336 ms/op 5.9460 ms/op 1.30
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 111.31 us/op 75.799 us/op 1.47
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.5725 ms/op 1.1435 ms/op 1.38
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 20.264 ms/op 15.520 ms/op 1.31
BeaconState.hashTreeRoot - 1 balances 77.578 us/op 57.965 us/op 1.34
BeaconState.hashTreeRoot - 32 balances 731.15 us/op 586.68 us/op 1.25
BeaconState.hashTreeRoot - 512 balances 7.5404 ms/op 6.0055 ms/op 1.26
BeaconState.hashTreeRoot - 250000 balances 123.49 ms/op 98.896 ms/op 1.25
aggregationBits - 2048 els - zipIndexesInBitList 34.164 us/op 32.240 us/op 1.06
regular array get 100000 times 57.881 us/op 67.530 us/op 0.86
wrappedArray get 100000 times 60.145 us/op 67.469 us/op 0.89
arrayWithProxy get 100000 times 39.072 ms/op 29.322 ms/op 1.33
ssz.Root.equals 597.00 ns/op 473.00 ns/op 1.26
byteArrayEquals 555.00 ns/op 452.00 ns/op 1.23
shuffle list - 16384 els 11.826 ms/op 11.123 ms/op 1.06
shuffle list - 250000 els 171.39 ms/op 163.58 ms/op 1.05
processSlot - 1 slots 17.521 us/op 12.073 us/op 1.45
processSlot - 32 slots 2.3939 ms/op 1.7609 ms/op 1.36
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 1.0361 ms/op 418.16 us/op 2.48
getCommitteeAssignments - req 1 vs - 250000 vc 5.3289 ms/op 5.2841 ms/op 1.01
getCommitteeAssignments - req 100 vs - 250000 vc 7.8994 ms/op 7.2660 ms/op 1.09
getCommitteeAssignments - req 1000 vs - 250000 vc 8.1276 ms/op 7.7652 ms/op 1.05
computeProposers - vc 250000 19.722 ms/op 18.425 ms/op 1.07
computeEpochShuffling - vc 250000 182.67 ms/op 167.04 ms/op 1.09
getNextSyncCommittee - vc 250000 327.62 ms/op 269.27 ms/op 1.22

by benchmarkbot/action

tuyennhv
tuyennhv approved these changes Jul 25, 2022
View reviewed changes
Copy link
Contributor

@tuyennhv tuyennhv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me, not sure why Builder Metric "Rest API response time" is so big 👀

@g11tech g11tech merged commit f26df84 into unstable Jul 25, 2022
@g11tech g11tech deleted the g11tech/eth1-builder-metrics branch July 25, 2022 12:40
@g11tech
Copy link
Contributor Author

g11tech commented Jul 25, 2022

looks good to me, not sure why Builder Metric "Rest API response time" is so big eyes

yea, registerValidator has like 500 vals in it in one query so its understandable, but there are a few produceBlindedBlocks calls which call builder's getPayloadHeader api which consume like 3+sec

@g11tech g11tech mentioned this pull request Jul 29, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tuyennhv tuyennhv tuyennhv approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@g11tech @tuyennhv