Attempting to rerun failing JUnit 4 Parameterized tests in IDEA fails

[paulduffin]

Overview

• ( x) Bug report.

Am using version 5.1.1

In IDEA run a JUnit 4 Parameterized test class which has one test method that fails. Attempt to rerun the failing test and it fails with the following error:

org.junit.platform.commons.util.PreconditionViolationException: Could not find method with name [foo[parameter]] in class [ParameterizedTest].

Although this happens in IDEA I don't think that this is a bug in IDEA. It seems as though there is an inconsistency between how org.junit.runners.Parameterized test formats a parameterized test and how org.junit.platform.engine.discovery.DiscoverySelectors#selectMethod(java.lang.String) expects a parameterized test to be formatted.

The former uses "class#method[parameters]", the latter expects "class#method(parameters)". That results in the methodName field of the MethodSelector object being set to "method[parameters]" instead of "method" which means that the org.junit.vintage.engine.discovery.MethodSelectorResolver#resolve(...) method fails to find method.

Couple of possible approaches:

1. Change org.junit.runners.Parameterized to use (...) instead of [...]. That could be a hard coded change or maybe configurable - such as if the test is being by JUnit 5.
2. Change DiscoverySelectors#selectMethod(java.lang.String) to handle both [...] and (...).
3. Have some way for engines to handle their own parsing of a String to a MethodSelector.
4. Change MethodSelectorResolver#resolve(...) to check to see if the methodName contains a [] and if it does then extract the parameters and recreate the MethodSelector.

The latter approach seems the best one to me as it places the logic for handling JUnit 4 code in the vintage engine which is where it belongs.

Deliverables

• - Resolve inconsistency so can rerun failed parameterized tests in IDEA

[sbrannen]

IDEA should actually use the unique ID of the test to select an individual invocation of a parameterized test to re-run it.

Thus, if IDEA is selecting the individual parameterized invocation via DiscoverySelectors#selectMethod, I would consider that a bug in IDEA.

@akozlova, thoughts?

[akozlova]

IDEA started to support uniqueIDs starting from 2018.1 (before that uniqueIds were not supported in standard engines). In other words, works fine here with latest released IDEA version.

[sbrannen]

Thanks, @akozlova! 👍

I assumed that was the case (just wasn't sure of the version of IDEA that supported unique IDs).

I am therefore closing this issue.

@paulduffin, if you are using a version of IDEA prior to 2018.1, please upgrade to that version.

[Panthro]

The issue is happening to me when I change the name of the test.

@Parameterized.Parameters(name = "{index} - {1} : {0}")

Then I get:

java.lang.Exception: No tests found matching Tests from org.junit.runner.Request$1@5ae50ce6

	at org.junit.internal.requests.FilterRequest.getRunner(FilterRequest.java:40)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:49)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

If I leave the @Parameterized.Parameters without name it works, any idea?

Junit version 4.12, can this be related or should I open another issue?

[sbrannen]

@Panthro, this is the issue tracker for JUnit 5, and your stack trace appears to be unrelated to JUnit 5.

If you wish to raise an issue against JUnit 4, please do that here.