This repo contains test reports for Jetpack e2e tests and the code used to create and maintain such reports.
All reports are stored in the docs folder which is served by Github Pages. A different folder exists for each report, the name used being the pull request number or the branch name for which the tests ran.
The reports urls have the following pattern: https://automattic.github.io/jetpack-e2e-reports/{PR_NUMBER}/report
Use the PR number or branch name (only for main branch - currently master) to access a test report.
A dashboard was implemented to ease the discovery of the stored reports. The dashboard gets updated every time a new report gets pushed.
The tests run in Jetpack monorepo CI. At the end of the E2E tests job suite a repository dispatch event is sent to this repo with all the required information about the test run, triggering the report workflow.
The reports are generated using Allure framework. Allure results in json format are being created by the tests and are stored as artefacts in Github after each test run. The workflow in this repo will download the artefacts, use the results to generate a new reports and then push the results and the newly generated report.
A cleanup job runs twice every day, deleting:
- full reports for pull requests that are closed
- full reports that were not updated in the last 30 days
- test results and attachments referencing tests that are not included in the history list (for each test we keep the last 20 results)