Unable to run single test that inherits all test methods with surefire

[lutovich]

Hello JUnit team,

I noticed that running a single test which inherits all test methods does not work with Maven.
JUnit 5.3.0-M1, surefire 2.22.0.

Base test class:

abstract class AbstractTestBase {
  @Test
  void theTest() {
    assertTrue(getValue());
  }

  protected abstract boolean getValue();
}

child class:

class ConcreteTest extends AbstractTestBase {
  @Override
  protected boolean getValue() {
    return true;
  }
}

Maven command mvn test -Dtest=ConcreteTest.

Surefire provider is now part of the surefire plugin but this issue might be JUnit related. I think this is what happens:

1. surefire parses the command like into a class patttern "**/ConcreteTest"
2. JUnit performs test discovery and notices ConcreteTest#theTest()
3. MethodBasedTestDescriptor is created with MethodSource as its test source
4. MethodSource is created using MethodSource#from(Method) and the discovered ConcreteTest class is not used
5. MethodSource constructor tries to figure out the class name using method.getDeclaringClass().getName()
6. resulting MethodSource is for class AbstractTestBase and method #theTest()
7. surefire's TestMethodFilter tries to match MathodSource(AbstractTestBase, theTest) with "**/ConcreteTest"
8. class name does not match and the test is skipped; at this point there is no way for surefire to undestand that AbstractTestBase#theTest() and ConcreteTest#theTest() is the same thing

Running mvn test -Dtest=ConcreteTest results in an error saying "No tests were executed!".
Command mvn test -Dtest=ConcreteTest,AbstractTestBase can be used as a workaround. It results in a single executed test, as expected.

Problem can be fixed by making constructor of MethodBasedTestDescriptor use MethodSource#from(String className, String methodName) static factory instead of MethodSource#from(Method). This preserves "ConcreteTest" name for future checks executed by surefire.

VintageTestDescriptor also uses MethodSource#from(Method) and might have the same problem.

Example project from this gist can be used to reproduce the problem.

I'd be happy to provide a fix for this problem if it is really in JUnit code and my findings are correct.

Thanks in advance!

[sormuras]

Duplicate of #1406 ?

[lutovich]

It does look like a duplicate. Sorry I did not check existing issues.
Does my description of the problem look right and it should be fixed in JUnit code?

[sbrannen]

Closing this issue as a duplicate of #1406.

[sbrannen]

Does my description of the problem look right

Appears so, but I have not verified it.

and it should be fixed in JUnit code?

If the above is true, then yes. 😉