You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We recently started noticing the following behavior in our test runs and are struggling to troubleshoot:
For classes that have multiple JUnit tests, sometimes (not always) when a single test in the class fails, other tests in the class that have not yet been executed are just missed (as in, they do not show up in the Serenity HTML report at all, and do not appear to have been executed).
For some additional context, we are executing our tests in a parallel fashion using the Maven Failsafe plugin. We have it configured to run classes in parallel, so only a single test from a given class should be executing at a time, and most (if not all) of our classes containing multiple tests use the following annotation to execute the tests in a predictable order:
When this problem occurs, not only are the missing/skipped tests not shown in the report, but I do not see them in the console output in Jenkins, either. I ran a small subset of our test classes as part of debugging this (few enough that there are more forks than classes). Immediately after the stack trace from the first failed test in a class, I see the message below:
[DEBUG] Closing the fork 4 after saying GoodBye.
I'm not sure this is a Serenity issue, but I thought I'd post here in case anyone had any insight into this. If there is any lacking information I can provide that might be helpful, please let me know.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
We recently started noticing the following behavior in our test runs and are struggling to troubleshoot:
For classes that have multiple JUnit tests, sometimes (not always) when a single test in the class fails, other tests in the class that have not yet been executed are just missed (as in, they do not show up in the Serenity HTML report at all, and do not appear to have been executed).
For some additional context, we are executing our tests in a parallel fashion using the Maven Failsafe plugin. We have it configured to run classes in parallel, so only a single test from a given class should be executing at a time, and most (if not all) of our classes containing multiple tests use the following annotation to execute the tests in a predictable order:
Failsafe configuration from pom.xml:
When this problem occurs, not only are the missing/skipped tests not shown in the report, but I do not see them in the console output in Jenkins, either. I ran a small subset of our test classes as part of debugging this (few enough that there are more forks than classes). Immediately after the stack trace from the first failed test in a class, I see the message below:
I'm not sure this is a Serenity issue, but I thought I'd post here in case anyone had any insight into this. If there is any lacking information I can provide that might be helpful, please let me know.
Additional version details:
Beta Was this translation helpful? Give feedback.
All reactions