Allow Launcher to randomize tests after discovery

[LeggoMahEggo]

My test classes are annotated with @TestMethodOrder using the Random method orderer; they are (of course) performed in a random order, but when I discover and execute them from a Launcher instance, they run in the order that they were discovered. For example if the order of test discovery is TestClassA followed by TestClassB, both test classes have their tests randomized, but TestClassA's tests will always run before TestClassB's tests. I would like to be able to randomize all tests so that test execution order is a mix of both TestClassA and TestClassB tests (I gather though that it may be problematic due to needing to instantiate multiple test classes at the same time).

[marcphilipp]

You can also randomize the order of test classes but that would still run all tests from one class before starting on the second class. The only way to achieve what you're after is to enable parallel execution. Would that be an option for you?

[LeggoMahEggo]

Unfortunately not; I use Junit with Selenium to test my company's browser tools, so I don't think parallel execution would help in that regard.

[marcphilipp]

Team decision: Since test classes also have lifecycle methods etc. we run them one by one unless parallel execution is enabled. Therefore, this proposal is not possible due to the architecture of JUnit 5.