Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Additional EDA on TestGrid Data set #17

Closed
1 task done
MichaelClifford opened this issue Sep 22, 2020 · 0 comments
Closed
1 task done

Additional EDA on TestGrid Data set #17

MichaelClifford opened this issue Sep 22, 2020 · 0 comments

Comments

@MichaelClifford
Copy link
Member

MichaelClifford commented Sep 22, 2020

To close issue #15 and build upon the initial EDA work done in #16 there are a number of additional questions that we would like answered about the TestGrid dataset. Specifically:

  • How comparable are the testgrids?
  • How do we analyze them in aggregate to learn from their combined behavior?
  • How many/ which tests do they all have in common?
  • Are their time series dates comparable?
  • Are there sub-groups that should only be compared with one another?
  • Is looking at the grid matrices independent of test names a valid approach for issue identification?
  • What is the expected behavior of a test over time across multiple jobs.
  • How does the entire test platform/specific tests perform on a given day?
  • How does the entire test platform behavior evolve over time.
  • Is there sufficient data here for useful ML approaches?
  • Can we develop some meaningful alerting/ problem identification with the results of the above questions?

Acceptance Criteria:

  • Notebook that address the questions above.
tumido added a commit to tumido/ocp-ci-analysis that referenced this issue Nov 3, 2020
Add AICoE CI configuration example
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants