CB-15495 TestNg reports are not generated for timed out test cases

[aszegedi]

Our E2E tests are running in parallel by 8 threads (with the help of TestNG parallel attribute):

parallel="methods": TestNG will run all your test methods in separate threads. Dependent methods will also run in separate threads but they will respect the order that you specified.

This helps us to reduce the 'execution time' as tests are executed simultaneously in different threads.

Beyond this we applied a 100 minutes timeout for our test cases by default, on this way we can guarantees that none of the threads will block on a stuck test thread forever.

Unfortunately test case's thread is terminated if it takes longer than it's timeout setting. I found some similar issues at TestNG:

• TestNg not generating reports in case of timeout #1215
• [selenium:testng] Total tests run: 0, Failures: 0, Skips: 0 on suite timeout with hardAssert? #669
• Test Timeout not respected in parallel="methods" mode #2009
• timeout setting not working in parallel="tests" mode #811

For now we can introduce an extra validation in our check-results.sh as a workaround for this.