[SUREFIRE-1345] Support flakyFailure and flakyError in TestSuiteXmlParser

[gsmet]

Hey there,

In the Quarkus project, we make good use of the Surefire reports to provide a full report of a pull request run.

We will soon enable the ability to retry tests but we need to make sure we can easily identify flaky tests thus why we included the ability to get flakyFailures and flakyErrors from the report.

This is related to https://issues.apache.org/jira/browse/SUREFIRE-1345 but does not entirely implement it as I haven't implemented the reruns. I could be convinced to if you think it's crucial to get this PR in but, in our case, we don't see it as useful.

Tests are included to make sure the data is properly extracted.

I think I already signed the Apache CLA a long time ago but can fill it again if needed.

Happy to iterate on this to get it merged soon.

Thanks for having a look!

Following this checklist to help us incorporate your
contribution quickly and easily:

• Make sure there is a JIRA issue filed
  for the change (usually before you start working on it). Trivial changes like typos do not
  require a JIRA issue. Your pull request should address just this issue, without
  pulling in other changes.
• Each commit in the pull request should have a meaningful subject line and body.
• Format the pull request title like [SUREFIRE-XXX] - Fixes bug in ApproximateQuantiles,
  where you replace SUREFIRE-XXX with the appropriate JIRA issue. Best practice
  is to use the JIRA issue title in the pull request title and in the first line of the
  commit message.
• Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
• Run mvn clean install to make sure basic checks pass. A more thorough check will
  be performed on your pull request automatically.
• You have run the integration tests successfully (mvn -Prun-its clean install).

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.

To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

• I hereby declare this contribution to be licenced under the Apache License Version 2.0, January 2004

• In any other case, please file an Apache Individual Contributor License Agreement.

[michael-o]

Yet another question: How do you want to utilize the data? In the Maven Surefire Report Plugin or somewhere else?

[gsmet]

The plan is to consume them in the Quarkus GitHub Bot for this feature: https://github.com/quarkusio/quarkus-github-bot#workflow-run-report .

It analyses all the build reports from our CI workflow and post a comment in the pull request.

We wanted to try to rerun tests but we need the feedback to contain the flakiness information.

[michael-o]

The plan is to consume them in the Quarkus GitHub Bot for this feature: https://github.com/quarkusio/quarkus-github-bot#workflow-run-report .

It analyses all the build reports from our CI workflow and post a comment in the pull request.

We wanted to try to rerun tests but we need the feedback to contain the flakiness information.

I see, let's focus on the issue and handle std streams separately, if you don't mind.

[michael-o]

@kriegaex Any objections before I go over to your PR?

[kriegaex]

@michael-o, thanks for considering that the original version of this PR collided with #692. But it is good to get confirmation from a third party that my idea of capturing stdOut and stdErr is useful to other users, too. No objections.

[michael-o]

Merged.

[gsmet]

Thanks for the review and for merging!