Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JUnit bump 4 -> 5 #5123

Merged
merged 3 commits into from Nov 14, 2022
Merged

JUnit bump 4 -> 5 #5123

merged 3 commits into from Nov 14, 2022

Conversation

bvfalcon
Copy link
Contributor

@bvfalcon bvfalcon commented Aug 9, 2022 •
edited

Migrating unit-tests from JUnit 4 to JUnit 5 (Issue).

Notes:

  1. IMPORTANT: Annotations @Before and @After of JUnit 4 work a little another way, as @BeforeEach and @AfterEach of JUnit 5. In case of overriding annotations @Before and @After will work, annotations @BeforeEach and @AfterEach - not. See example for demonstration.
  2. jersey-release-notes-maven-plugin still uses JUnit 4 because of used in unit-tests maven-plugin-testing-harness has no support for JUnit5. And because of it is comparatively isolated part of jersey project.
  3. jersey-tests-osgi-functional, osgi-helloworld-webapp, osgi-http-service and extended-wadl-webapp still use JUnit 4 because of pax-exam has no support for JUnit 5 (issues one, two)
  4. I suppose, test monitoring never worked: dependencies without version cannot work in projects without parent.

@bvfalcon bvfalcon force-pushed the junit-switch-4-to-5 branch 24 times, most recently from 9f33b87 to 1c7adb7 Compare August 17, 2022 15:39
@bvfalcon bvfalcon force-pushed the junit-switch-4-to-5 branch 6 times, most recently from 1970cfb to 735a13a Compare August 19, 2022 10:20
This was referenced Mar 17, 2023
Closed
Open
Open
Closed
@joschi joschi mentioned this pull request Mar 25, 2023
Closed
This was referenced May 11, 2023
Open
Open
This was referenced Jun 1, 2023
Merged
Closed
This was referenced Jun 4, 2023
Merged
Closed
This was referenced Jun 15, 2023
Merged
Merged
This was referenced Jun 26, 2023
Closed
Open
This was referenced Jul 6, 2023
Open
Open
@LuciferYang LuciferYang mentioned this pull request Jul 11, 2023
Closed
LuciferYang added a commit to apache/spark that referenced this pull request Jul 12, 2023
@LuciferYang
[SPARK-44377][BUILD] Exclude Junit5 dependencies from `jersey-test-fr…
443b49e 
…amework-provider-simple`

### What changes were proposed in this pull request?
The following situation has led to Java tests no longer be executed when performing maven tests on the core module:

1.  SPARK-44316 upgrade Jersey from 2.36 to 2.40.
2. Jersey 2.38 start to use [Junit5 instead of Junit4](eclipse-ee4j/jersey#5123).
3. Spark core module uses `org.glassfish.jersey.test-framework.providers:jersey-test-framework-provider-simple:2.40:test`, which cascades and introduces the dependencies of Junit5.

So this pr manual exclude `Junit5` dependencies from `org.glassfish.jersey.test-framework.providers:jersey-test-framework-provider-simple:2.40:test` to make Java tests of core module can be tested using Maven.

Additionally, this pr leaves a TODO to remove this exclusion after Spark  migrates to Junit5 for Java testing:

- [SPARK-44170](https://issues.apache.org/jira/browse/SPARK-44170): Migrating Junit4 to Junit5

### Why are the changes needed?
Make Java tests of core module can be tested using Maven after SPARK-44316. This issue only affects Maven, not SBT.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
- Pass Git Hub Action
- Manual test:

run `mvn clean install -pl core -am -Dtest.exclude.tags=org.apache.spark.tags.ExtendedLevelDBTest`

**Before**

The core module is not running any Java tests

```
[INFO] --- maven-surefire-plugin:3.1.2:test (default-test)  spark-core_2.12 ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-surefire-plugin:3.1.2:test (test)  spark-core_2.12 ---
[INFO] Skipping execution of surefire because it has already been run for this configuration
```

**After**

The Java test run of the core module was successful

```
[INFO] --- maven-surefire-plugin:3.1.2:test (default-test)  spark-core_2.12 ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitcore.JUnitCoreProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running test.org.apache.spark.Java8RDDAPISuite
[INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.845 s -- in test.org.apache.spark.Java8RDDAPISuite
[INFO] Running test.org.apache.spark.JavaAPISuite
[INFO] Tests run: 92, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.40 s -- in test.org.apache.spark.JavaAPISuite
[INFO] Running test.org.apache.spark.JavaSparkContextSuite
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.587 s -- in test.org.apache.spark.JavaSparkContextSuite
[INFO] Running org.apache.spark.launcher.SparkLauncherSuite
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.977 s -- in org.apache.spark.launcher.SparkLauncherSuite
[INFO] Running org.apache.spark.JavaJdbcRDDSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.491 s -- in org.apache.spark.JavaJdbcRDDSuite
[INFO] Running org.apache.spark.shuffle.sort.PackedRecordPointerSuite
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s -- in org.apache.spark.shuffle.sort.PackedRecordPointerSuite
[INFO] Running org.apache.spark.shuffle.sort.ShuffleInMemoryRadixSorterSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 s -- in org.apache.spark.shuffle.sort.ShuffleInMemoryRadixSorterSuite
[INFO] Running org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite
[INFO] Tests run: 27, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.829 s -- in org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite
[INFO] Running org.apache.spark.shuffle.sort.ShuffleInMemorySorterSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.052 s -- in org.apache.spark.shuffle.sort.ShuffleInMemorySorterSuite
[INFO] Running org.apache.spark.memory.TaskMemoryManagerSuite
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s -- in org.apache.spark.memory.TaskMemoryManagerSuite
[INFO] Running org.apache.spark.util.SerializableConfigurationSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.058 s -- in org.apache.spark.util.SerializableConfigurationSuite
[INFO] Running org.apache.spark.util.collection.unsafe.sort.UnsafeInMemorySorterRadixSortSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 s -- in org.apache.spark.util.collection.unsafe.sort.UnsafeInMemorySorterRadixSortSuite
[INFO] Running org.apache.spark.util.collection.unsafe.sort.UnsafeInMemorySorterSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s -- in org.apache.spark.util.collection.unsafe.sort.UnsafeInMemorySorterSuite
[INFO] Running org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorterSuite
[INFO] Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.488 s -- in org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorterSuite
[INFO] Running org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorterRadixSortSuite
[INFO] Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.433 s -- in org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorterRadixSortSuite
[INFO] Running org.apache.spark.io.ReadAheadInputStreamSuite
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.145 s -- in org.apache.spark.io.ReadAheadInputStreamSuite
[INFO] Running org.apache.spark.io.NioBufferedInputStreamSuite
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.540 s -- in org.apache.spark.io.NioBufferedInputStreamSuite
[INFO] Running org.apache.spark.unsafe.map.BytesToBytesMapOnHeapSuite
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.771 s -- in org.apache.spark.unsafe.map.BytesToBytesMapOnHeapSuite
[INFO] Running org.apache.spark.unsafe.map.BytesToBytesMapOffHeapSuite
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.647 s -- in org.apache.spark.unsafe.map.BytesToBytesMapOffHeapSuite
[INFO] Running org.apache.spark.api.java.OptionalSuite
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s -- in org.apache.spark.api.java.OptionalSuite
[INFO] Running org.apache.spark.resource.JavaResourceProfileSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s -- in org.apache.spark.resource.JavaResourceProfileSuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 264, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-surefire-plugin:3.1.2:test (test)  spark-core_2.12 ---
[INFO] Skipping execution of surefire because it has already been run for this configuration
...

Run completed in 21 minutes, 45 seconds.
Total number of tests run: 3196
Suites: completed 300, aborted 0
Tests: succeeded 3196, failed 0, canceled 5, ignored 10, pending 0
All tests passed.
```

Closes #41944 from LuciferYang/SPARK-44377.

Authored-by: yangjie01 <yangjie01@baidu.com>
Signed-off-by: yangjie01 <yangjie01@baidu.com>
This was referenced Aug 28, 2023
Merged
Merged
This was referenced Nov 30, 2023
Closed
Closed
@renovate renovate bot mentioned this pull request Mar 1, 2024
Merged
1 task
ragnarok56 pushed a commit to ragnarok56/spark that referenced this pull request Mar 2, 2024
@LuciferYang @ragnarok56
[SPARK-44377][BUILD] Exclude Junit5 dependencies from `jersey-test-fr…
916baef 
…amework-provider-simple`

### What changes were proposed in this pull request?
The following situation has led to Java tests no longer be executed when performing maven tests on the core module:

1.  SPARK-44316 upgrade Jersey from 2.36 to 2.40.
2. Jersey 2.38 start to use [Junit5 instead of Junit4](eclipse-ee4j/jersey#5123).
3. Spark core module uses `org.glassfish.jersey.test-framework.providers:jersey-test-framework-provider-simple:2.40:test`, which cascades and introduces the dependencies of Junit5.

So this pr manual exclude `Junit5` dependencies from `org.glassfish.jersey.test-framework.providers:jersey-test-framework-provider-simple:2.40:test` to make Java tests of core module can be tested using Maven.

Additionally, this pr leaves a TODO to remove this exclusion after Spark  migrates to Junit5 for Java testing:

- [SPARK-44170](https://issues.apache.org/jira/browse/SPARK-44170): Migrating Junit4 to Junit5

### Why are the changes needed?
Make Java tests of core module can be tested using Maven after SPARK-44316. This issue only affects Maven, not SBT.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
- Pass Git Hub Action
- Manual test:

run `mvn clean install -pl core -am -Dtest.exclude.tags=org.apache.spark.tags.ExtendedLevelDBTest`

**Before**

The core module is not running any Java tests

```
[INFO] --- maven-surefire-plugin:3.1.2:test (default-test)  spark-core_2.12 ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-surefire-plugin:3.1.2:test (test)  spark-core_2.12 ---
[INFO] Skipping execution of surefire because it has already been run for this configuration
```

**After**

The Java test run of the core module was successful

```
[INFO] --- maven-surefire-plugin:3.1.2:test (default-test)  spark-core_2.12 ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitcore.JUnitCoreProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running test.org.apache.spark.Java8RDDAPISuite
[INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.845 s -- in test.org.apache.spark.Java8RDDAPISuite
[INFO] Running test.org.apache.spark.JavaAPISuite
[INFO] Tests run: 92, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.40 s -- in test.org.apache.spark.JavaAPISuite
[INFO] Running test.org.apache.spark.JavaSparkContextSuite
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.587 s -- in test.org.apache.spark.JavaSparkContextSuite
[INFO] Running org.apache.spark.launcher.SparkLauncherSuite
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.977 s -- in org.apache.spark.launcher.SparkLauncherSuite
[INFO] Running org.apache.spark.JavaJdbcRDDSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.491 s -- in org.apache.spark.JavaJdbcRDDSuite
[INFO] Running org.apache.spark.shuffle.sort.PackedRecordPointerSuite
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s -- in org.apache.spark.shuffle.sort.PackedRecordPointerSuite
[INFO] Running org.apache.spark.shuffle.sort.ShuffleInMemoryRadixSorterSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 s -- in org.apache.spark.shuffle.sort.ShuffleInMemoryRadixSorterSuite
[INFO] Running org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite
[INFO] Tests run: 27, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.829 s -- in org.apache.spark.shuffle.sort.UnsafeShuffleWriterSuite
[INFO] Running org.apache.spark.shuffle.sort.ShuffleInMemorySorterSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.052 s -- in org.apache.spark.shuffle.sort.ShuffleInMemorySorterSuite
[INFO] Running org.apache.spark.memory.TaskMemoryManagerSuite
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s -- in org.apache.spark.memory.TaskMemoryManagerSuite
[INFO] Running org.apache.spark.util.SerializableConfigurationSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.058 s -- in org.apache.spark.util.SerializableConfigurationSuite
[INFO] Running org.apache.spark.util.collection.unsafe.sort.UnsafeInMemorySorterRadixSortSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 s -- in org.apache.spark.util.collection.unsafe.sort.UnsafeInMemorySorterRadixSortSuite
[INFO] Running org.apache.spark.util.collection.unsafe.sort.UnsafeInMemorySorterSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s -- in org.apache.spark.util.collection.unsafe.sort.UnsafeInMemorySorterSuite
[INFO] Running org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorterSuite
[INFO] Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.488 s -- in org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorterSuite
[INFO] Running org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorterRadixSortSuite
[INFO] Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.433 s -- in org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorterRadixSortSuite
[INFO] Running org.apache.spark.io.ReadAheadInputStreamSuite
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.145 s -- in org.apache.spark.io.ReadAheadInputStreamSuite
[INFO] Running org.apache.spark.io.NioBufferedInputStreamSuite
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.540 s -- in org.apache.spark.io.NioBufferedInputStreamSuite
[INFO] Running org.apache.spark.unsafe.map.BytesToBytesMapOnHeapSuite
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.771 s -- in org.apache.spark.unsafe.map.BytesToBytesMapOnHeapSuite
[INFO] Running org.apache.spark.unsafe.map.BytesToBytesMapOffHeapSuite
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.647 s -- in org.apache.spark.unsafe.map.BytesToBytesMapOffHeapSuite
[INFO] Running org.apache.spark.api.java.OptionalSuite
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s -- in org.apache.spark.api.java.OptionalSuite
[INFO] Running org.apache.spark.resource.JavaResourceProfileSuite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s -- in org.apache.spark.resource.JavaResourceProfileSuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 264, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-surefire-plugin:3.1.2:test (test)  spark-core_2.12 ---
[INFO] Skipping execution of surefire because it has already been run for this configuration
...

Run completed in 21 minutes, 45 seconds.
Total number of tests run: 3196
Suites: completed 300, aborted 0
Tests: succeeded 3196, failed 0, canceled 5, ignored 10, pending 0
All tests passed.
```

Closes apache#41944 from LuciferYang/SPARK-44377.

Authored-by: yangjie01 <yangjie01@baidu.com>
Signed-off-by: yangjie01 <yangjie01@baidu.com>
@renovate renovate bot mentioned this pull request Mar 18, 2024
Merged
1 task
@mend-for-github-com mend-for-github-com bot mentioned this pull request Mar 24, 2024
Open
1 task
This was referenced Apr 13, 2024
Open
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jansupol jansupol jansupol approved these changes

@senivam senivam senivam approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
2.38
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@bvfalcon @jansupol @senivam