Random failure analysis
The goal of this project is to make it possible to identify flaky tests by aggregating test failure data from RHBA Jenkins.
The project consists of 2 parts:
- A Haskell program which
- downloads test results of all unstable builds of jobs from master pullrequests folder of RHBA Jenkins. For each failure it saves 5 items: job URL, test class and test method name, stack trace and date of build.
- searches for the path of each test class within local filesystem (starting from folder where all kiegroup repositories are cloned) in order to provide GitHub link functionality.
- A single page Elm application which enables browsing data scraped by the above script. This is deployed at janhrcek.cz/random-failures/ and updated with new data on weekly basis.
Updating the report
RHBA Jenkins is only archiving last 14 days of builds. So it's necessary to periodically (~ once a week) scrape test failure data. The data are persisted in a JSON file, which is then deployed to the gh-pages branch of this repository together with the interactive HTML application for browsing it.
The process of scraping is automated, everything can be done by just running
./cli.sh at the root of this project. The script will
- build and run the scraper program, which outputs all the failures into
- build and runs the front end report
- copy the contents of the
frontend/distto the root directory of this repo at
- push the updated report to the