Spike: Identify TestGrid Metrics/KPIs - Test Performance

[MichaelClifford]

As Data Scientists our first task is to convert the raw data generated by the CI processes into some meaningful KPI's/Metrics/Features (numbers) that we can track and use to describe the state or behavior of the CI process over time. One of the key elements of the CI process are the tests. Understanding the health and behavior of these test suites is critical.

Test KPI's could be things like "Test Pass Rate", "Test Run Rate", "Number of Correlated Failures with Test", etc. Sippy is currently quantifying these types of metrics and might be a good place to start looking for examples. But, as data scientists, we must admit that we are not currently subject matter experts in CI monitoring and do not know the best metrics to track for monitoring CI platform test health. As such, we need to perform a research spike and look for example KPI's used in the industry that we could collect and monitor from the TestGrid Data.

Acceptance Criteria:

• create a markdown document of KPI's. Issue must include link to resource used to discover the metric, an explanation of why it would be useful to track, and a brief outline describing how we could generate it from our existing data sources.

[oindrillac]

#177