Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The combination of JUnit 5.3.0, maven-surefire-plugin 2.22.0 and reuseForks=false fails to find tests #1579

Closed
mureinik opened this issue Sep 6, 2018 · 21 comments

Comments

@mureinik
Copy link
Contributor

mureinik commented Sep 6, 2018

Overview

I'm not sure if this bug should be reported here or on maven-surefire-plugin's repo, but I'll give it a try here.
When using a combination of JUnit 5.3.0 together with maven-surefire-plugin 2.22.0 configured with reuseForks=false, surefire fails to find any JUnit tests.

Downgrading JUnit to 5.2.0 makes it work (i.e., tests are properly found), as does setting reuseForks=true.

The pom.xml file:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.example</groupId>
  <artifactId>junit.jupiter.reuse-forks</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>junit.jupiter.reuse-forks</name>
  <url>http://maven.apache.org</url>

  <dependencies>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <version>5.3.0</version> <!-- If replaced by 5.2.0, the test executes -->
      <scope>test</scope>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.22.0</version>
        <configuration>
          <reuseForks>false</reuseForks> <!-- If replace by true, the test executes -->
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

A complete example (just unzip and run mvn install):
reuser-forks.zip

With JUnit 5.3.0, you'll see the output:

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

With JUnit 5.2.0, you'll see the expected output:

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
@sormuras
Copy link
Member

sormuras commented Sep 6, 2018

I'm not sure if this bug should be reported here or on maven-surefire-plugin's repo, but I'll give it a try here.

As the entire code now resides in Surefire's repo, please open new Surefire-related issues here: https://issues.apache.org/jira/browse/SUREFIRE

Seems to be related to apache/maven-surefire#193 -- see also the discussion at https://mail-archives.apache.org/mod_mbox/maven-dev/201809.mbox/browser "issue with junit 5 integration". I'm not sure, what a possible solution could be. The PR 193 seems to fix the issue.

@mureinik
Copy link
Contributor Author

mureinik commented Sep 7, 2018

As the entire code now resides in Surefire's repo, please open new Surefire-related issues here: https://issues.apache.org/jira/browse/SUREFIRE

I opened https://issues.apache.org/jira/browse/SUREFIRE-1566 according to this comment.
If apache/maven-surefire#193 fixes it it may be a mute point, but better have this as a top level issue on it's own right to make sure the surefire devs consider this usecase.

@sormuras
Copy link
Member

sormuras commented Sep 7, 2018

Thanks, Allon.

... the surefire devs consider this usecase.

We do. ;-)

@winterbe
Copy link

winterbe commented Sep 11, 2018

Any news on this? Does the latest 5.3.1 update fix this issue? Unfortunately there doesn't seem to be any activity on the Maven Surefire issue.

@mureinik
Copy link
Contributor Author

@winterbe I've tested with JUnit 5.3.1 too, and it does not fix the issue.

@baron1405
Copy link

baron1405 commented Sep 17, 2018

I have experienced a similar issue but using Gradle 4.9 and 4.10 junit platform integration with Junit 5.3.0 and 5.3.1. Everything works with 5.2.0.

@sormuras
Copy link
Member

It is a bug in how Surefire resolves the underlying JUnit Platform dependencies. At the moment, it always loads version 5.2.0. We're working to solve the issue in the upcoming Surefire release.

For details see https://issues.apache.org/jira/browse/SUREFIRE-1564

@baron1405
Copy link

baron1405 commented Sep 17, 2018

@sormuras would this fix the issue with Gradle as well?

@sormuras
Copy link
Member

sormuras commented Sep 17, 2018

No, don't think so. Gradle has it's own integration of the platform.

Correct, @marcphilipp?

@baron1405
Copy link

In any case the fix required in Surefire might be required in Gradle.

@sormuras
Copy link
Member

Gradle seems not affected. The generated @classpath file (gradle-worker-classpath17288247877244964343txt) for the starter sample project looks good:

-cp
...
junit-jupiter-params-5.3.1.jar
junit-jupiter-engine-5.3.1.jar
junit-jupiter-api-5.3.1.jar
junit-platform-engine-1.3.1.jar
junit-platform-commons-1.3.1.jar
apiguardian-api-1.0.0.jar
opentest4j-1.1.1.jar

...configured via:

https://github.com/junit-team/junit5-samples/blob/c155a7f0d8683d2274856569388720e44841abcb/junit5-jupiter-starter-gradle/build.gradle#L11-L22

Running with --debug option active shows that Gradle internally uses JUnit Platform 1.0.3, though. That doesn't however prevent the tests to be found or getting executed.

application classpath

09:43:21.333 [DEBUG] [o.g.p.i.w.DefaultWorkerProcessBuilder] Using application classpath
...
junit-jupiter-params-5.3.1.jar
junit-jupiter-engine-5.3.1.jar
junit-jupiter-api-5.3.1.jar
junit-platform-engine-1.3.1.jar
junit-platform-commons-1.3.1.jar
apiguardian-api-1.0.0.jar
opentest4j-1.1.1.jar

implementation classpath

09:43:21.333 [DEBUG] [o.g.p.i.w.DefaultWorkerProcessBuilder] Using implementation classpath
...
gradle-core-api-4.8.jar
gradle-core-4.8.jar
gradle-logging-4.8.jar
gradle-messaging-4.8.jar
gradle-base-services-4.8.jar
gradle-cli-4.8.jar
gradle-native-4.8.jar
plugins/gradle-testing-base-4.8.jar
plugins/gradle-testing-jvm-4.8.jar
plugins/gradle-testing-junit-platform-4.8.jar
plugins/junit-platform-engine-1.0.3.jar
plugins/junit-platform-launcher-1.0.3.jar
plugins/junit-platform-commons-1.0.3.jar
gradle-process-services-4.8.jar
slf4j-api-1.7.16.jar
jul-to-slf4j-1.7.16.jar
native-platform-0.14.jar
kryo-2.20.jar
commons-lang-2.6.jar
plugins/junit-4.12.jar

@baron1405
Copy link

After trying a number of my Gradle projects, it appears to only effect my Spring Boot projects. I need to look more deeply. Any insights would be appreciated.

@sbrannen
Copy link
Member

Then that's almost certainly due to a version conflict.

spring-boot-dependencies defines a specific version for JUnit Jupiter. So if you're using a more recent version, you'll have to make sure you override the version managed by Spring Boot.

@sbrannen
Copy link
Member

Sounds like a recent thread on Twitter I participated in.

See the history of this person's build.gradle to see how to fix it.

https://github.com/TimRiemer/junit5_springboot_test/commits/master/build.gradle

@sbrannen
Copy link
Member

Basically, you need the following:

ext['junit-jupiter.version'] = '5.3.1' // or '5.3.0'

test {
    useJUnitPlatform()
}

dependencies {
    compile('org.springframework.boot:spring-boot-starter')
    testCompile('org.junit.jupiter:junit-jupiter-api')
    testRuntime('org.junit.jupiter:junit-jupiter-engine')
    testCompile('org.springframework.boot:spring-boot-starter-test') {
        exclude group: 'junit', module: 'junit'
    }
}

Of course, you don't have to exclude JUnit 4 if you are using it, but then you'll need the junit-vintage-engine as well.

@baron1405
Copy link

@sbrannen that did it. I did not consider this because it is the first time I have encountered anomalous behavior. Going forward I will use the property to set the version and let the Spring Boot plugin do the rest. Thanks to all for their help!

@sbrannen
Copy link
Member

Glad to hear that works for you!

@mureinik
Copy link
Contributor Author

It is a bug in how Surefire resolves the underlying JUnit Platform dependencies. At the moment, it always loads version 5.2.0. We're working to solve the issue in the upcoming Surefire release.

For details see https://issues.apache.org/jira/browse/SUREFIRE-1564

Quick note:
Yesterday I upgraded the project where I originally encountered this issue (Synopsys' Seeker, not open source, unfortunately) to JUnit Jupiter 5.3.1 and maven-surefire-plugin 2.22.1 (released last weekend), and I can confirm that it indeed solves the problems.

Kudos!

@TamasNeumer
Copy link

I was also facing the same issue (using spring-boot).
However as @sbrannen pointed out previously I had to override the junit dependencies provided by spring-boot-starter, which then solved the issue. Thanks!

<properties>
   ....
    <junit-jupiter.version>5.3.1</junit-jupiter.version>
    <junit-platform.version>1.3.1</junit-platform.version>
   ....
</properties>

tommytroen added a commit to navikt/token-support that referenced this issue Oct 28, 2019
…versions and surefire version 2.22.0 causes no tests to run

* ref - junit-team/junit5#1579
tommytroen added a commit to navikt/token-support that referenced this issue Oct 28, 2019
…versions and surefire version 2.22.0 causes no tests to run

* ref - junit-team/junit5#1579
@aorlowskiddm
Copy link

aorlowskiddm commented Nov 10, 2022

For anyone seeking a more clear Maven-oriented answer, find where in your pom.xml spring-boot-starter-test or spring-boot-starter is declared and add an exclusion for junit-jupiter, then specify your own version (the latest, 5.9.0 seemed to work for me):

               <dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
			<exclusions>
				<!-- Exclude jupiter version provided with dependency and use version specified elsewhere -->
				<exclusion>
					<groupId>org.junit.jupiter</groupId>
					<artifactId>junit-jupiter</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<!-- Override JUnit Jupiter provided by spring-boot-starter-test -->
		<dependency>
			<groupId>org.junit.jupiter</groupId>
			<artifactId>junit-jupiter</artifactId>
			<scope>test</scope>
			<version>5.9.0</version>
		</dependency>

@daivanov
Copy link

The adding junit-jupiter:5.9.0 did not work for me. I had to downgrade maven-surefire-plugin from 2.22.2 to 2.21.0

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.21.0</version>
            </plugin>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants