Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

log auto and manual DTT starts, green DTTs and red DTTs #19998

Merged
merged 3 commits into from Jan 15, 2018

Conversation

Erin007
Copy link
Contributor

@Erin007 Erin007 commented Jan 13, 2018

This PR begins the process of collecting metrics about DTTs as a measure of deployment pipeline health.

  • When the robo-DOTD automatically kicks off a new DTT, log a new dtt_start with a value of 0 (to indicate that it was automatic).
  • When a new DTT is manually kicked off, log a new dtt_start with a value of 1 (manual).
  • When a DTT is automatically marked green, log a new dtt_green with a value of 0.
  • When a DTT is manually marked green, log a new dtt_green with a value of 1.
  • When a DTT is marked red, log a new dtt_red with a value of 1 *.

*NOTE: Marking a DTT red really just means that someone has manually looked at the DTT and determined that it will never be marked green. We log this because we will want to differentiate between runs marked red and those that have not ever been marked or were killed etc.

Data collected in the metrics table so far is also now summarized in metrics_helper.rb.

# | name | metadata | value |
# -----------------------------------------------------------------------------
# | dtt_start | commit hash | 0 - if automatic |
# | | | 1 - if manual |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few file-level questions/comments:

  1. the values of 0 and 1 for automatic and manual should really be defined as constants in this file
  2. Does the "value" column have to be a number? It would be helpful to add some documentation for the parameters to self.write_metric, especially since there is no easy way to see the definition of the underlying mysql table.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The value column is a double. Guidance from Will and Jeremy is that the data table should be as versatile as possible so we can chuck in any type of metric we want to measure; the columns are intentionally very general with the idea that metadata is flexible enough to hold whatever information makes sense.
If this works, then I'll write up some documentation outlining the structure and how to add new metrics. Where should that info live? The wiki? Part of the README?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The write_metric method in this file should definitely have documentation including (1) what this method does and (2) the types of each argument. Then, some high-level documentation pointing to this method could either go in the wiki (if the audience is developers) or in the github readme (if the audience is developers + contributors).

Copy link
Contributor Author

@Erin007 Erin007 Jan 15, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated to include documentation for the write_metric method consistent with commenting for methods in other lib/cdo files.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new docs look great!

# | name | metadata | value |
# -----------------------------------------------------------------------------
# | dtt_start | commit hash | 0 - if automatic |
# | | | 1 - if manual |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new docs look great!

@Erin007 Erin007 merged commit de3064d into staging Jan 15, 2018
@Erin007 Erin007 deleted the mark-red-green-dtts branch January 15, 2018 22:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants