New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
log auto and manual DTT starts, green DTTs and red DTTs #19998
Conversation
# | name | metadata | value | | ||
# ----------------------------------------------------------------------------- | ||
# | dtt_start | commit hash | 0 - if automatic | | ||
# | | | 1 - if manual | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few file-level questions/comments:
- the values of 0 and 1 for automatic and manual should really be defined as constants in this file
- Does the "value" column have to be a number? It would be helpful to add some documentation for the parameters to
self.write_metric
, especially since there is no easy way to see the definition of the underlying mysql table.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The value column is a double. Guidance from Will and Jeremy is that the data table should be as versatile as possible so we can chuck in any type of metric we want to measure; the columns are intentionally very general with the idea that metadata is flexible enough to hold whatever information makes sense.
If this works, then I'll write up some documentation outlining the structure and how to add new metrics. Where should that info live? The wiki? Part of the README?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The write_metric
method in this file should definitely have documentation including (1) what this method does and (2) the types of each argument. Then, some high-level documentation pointing to this method could either go in the wiki (if the audience is developers) or in the github readme (if the audience is developers + contributors).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I updated to include documentation for the write_metric
method consistent with commenting for methods in other lib/cdo files.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new docs look great!
# | name | metadata | value | | ||
# ----------------------------------------------------------------------------- | ||
# | dtt_start | commit hash | 0 - if automatic | | ||
# | | | 1 - if manual | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new docs look great!
This PR begins the process of collecting metrics about DTTs as a measure of deployment pipeline health.
*NOTE: Marking a DTT red really just means that someone has manually looked at the DTT and determined that it will never be marked green. We log this because we will want to differentiate between runs marked red and those that have not ever been marked or were killed etc.
Data collected in the metrics table so far is also now summarized in metrics_helper.rb.