Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attestation performance metrics #7709

Merged

Conversation

potuz
Copy link
Contributor

@potuz potuz commented Nov 3, 2020

What type of PR is this?

Feature

Fixes #7694

What does this PR do? Why is it needed?

This PR exports 5 metrics by pubkey: the attested slot, the inclusion distance, the correctness of the vote on head,target and source. In order to be correct however this is now based on #7708 since otherwise it would suffer from #7699

Other notes for review

The attested slot is always one epoch ahead with this PR. This has to be taken into account when producing tables like this one

import

I have checked the above for over a day with beaconcha.in now that it is working.

@potuz potuz requested a review from a team as a code owner November 3, 2020 11:46
@codecov
Copy link

codecov bot commented Nov 3, 2020

Codecov Report

Merging #7709 (4c85e00) into master (8638e2c) will increase coverage by 0.01%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #7709      +/-   ##
==========================================
+ Coverage   62.10%   62.12%   +0.01%     
==========================================
  Files         429      429              
  Lines       30365    30379      +14     
==========================================
+ Hits        18859    18873      +14     
  Misses       8566     8566              
  Partials     2940     2940              

Copy link
Member

@terencechain terencechain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm slightly concerned on how this will increase the cardinality of the metrics page significantly and require tsdb to store much more data than before.

Is a gauge the most appropriate format?
We track overall inclusion distance with a histogram in the beacon node, should we use a histogram?

"pubkey",
},
)
// ValidatorCorrectlyVotedHeadGaugeVec used to keep track of validator's accuracy on voting target by public key.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// ValidatorCorrectlyVotedHeadGaugeVec used to keep track of validator's accuracy on voting target by public key.
// ValidatorCorrectlyVotedHeadGaugeVec used to keep track of validator's accuracy on voting head by public key.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh thanks for catching this. As for the gauge @prestonvanloon is worried about the same. Gauge is needed to get a time series which you need if you want to collect the table above. Getting a histogram is only useful to get averages. This as gauge replicates the table in beaconcha.in. The prometheus dbase can then later be pruned can't it?

Copy link
Member

@prestonvanloon prestonvanloon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few changes that seem unrelated. Please explain additional changes

Comment on lines 180 to 182
if (slot+1)%params.BeaconConfig().SlotsPerEpoch != 0 || slot <= params.BeaconConfig().SlotsPerEpoch {
// Do nothing unless we are at the end of the epoch, and not in the first epoch.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why change this?

validator/client/runner.go Outdated Show resolved Hide resolved
@potuz
Copy link
Contributor Author

potuz commented Nov 6, 2020

Fixes #7694

@rauljordan
Copy link
Contributor

@terencechain PTAL

@rauljordan rauljordan added OK to merge Ready For Review A pull request ready for code review labels Nov 10, 2020
@prylabs-bulldozer prylabs-bulldozer bot merged commit e22dd37 into prysmaticlabs:master Nov 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ready For Review A pull request ready for code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Inclusion Distance as prometheus metrics (maybe Attestation Effectiveness)
4 participants