`@MethodSource` fails to load class with custom `ClassLoader` arrangement

[chrisr3]

JUnit tests with @MethodSource fail inside an OSGi framework after upgrading from 5.9.2 to 5.9.3.

# Execution Started: testMethodSource(String) - [engine:bnd-bundle-engine]/[bundle:test-junit5-tests;1.0.0.SNAPSHOT]/[sub-engine:junit-jupiter]/[class:org.testing.junit.TestJUnit]/[test-template:testMethodSource(java.lang.String)]
# Execution Finished: testMethodSource(String) - [engine:bnd-bundle-engine]/[bundle:test-junit5-tests;1.0.0.SNAPSHOT]/[sub-engine:junit-jupiter]/[class:org.testing.junit.TestJUnit]/[test-template:testMethodSource(java.lang.String)] - TestExecutionResult [status = FAILED, throwable = org.junit.platform.commons.JUnitException: Could not load class [org.testing.junit.TestJUnit]]
org.junit.platform.commons.JUnitException: Could not load class [org.testing.junit.TestJUnit]
        at org.junit.jupiter.params.provider.MethodArgumentsProvider.lambda$loadRequiredClass$8(MethodArgumentsProvider.java:181)
        at org.junit.platform.commons.function.Try$Failure.getOrThrow(Try.java:335)
        at org.junit.jupiter.params.provider.MethodArgumentsProvider.loadRequiredClass(MethodArgumentsProvider.java:180)
        at org.junit.jupiter.params.provider.MethodArgumentsProvider.findFactoryMethodByFullyQualifiedName(MethodArgumentsProvider.java:117)
        at org.junit.jupiter.params.provider.MethodArgumentsProvider.findFactoryMethod(MethodArgumentsProvider.java:82)
        ...
Caused by: java.lang.ClassNotFoundException: org.testing.junit.TestJUnit
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at org.junit.platform.commons.util.ReflectionUtils.lambda$tryToLoadClass$9(ReflectionUtils.java:831)
        at org.junit.platform.commons.function.Try.lambda$call$0(Try.java:57)
        at org.junit.platform.commons.function.Try.of(Try.java:93)
        ...

Steps to reproduce

Run attached Gradle project:

$ ./gradlew clean build

Context

• Used versions (Jupiter/Vintage/Platform):
  Jupiter 5.9.3
  Platform 1.9.3
• Build Tool/IDE:
  Gradle 8.1.1
  Java 11

[marcphilipp]

Could be a red herring but there's also a suppressed exception that could be related to #3131 or #3266:

org.junit.platform.commons.JUnitException: Could not load class [org.testing.junit.TestJUnit]
        at org.junit.jupiter.params.provider.MethodArgumentsProvider.lambda$loadRequiredClass$8(MethodArgumentsProvider.java:181)
        at org.junit.platform.commons.function.Try$Failure.getOrThrow(Try.java:335)
        at org.junit.jupiter.params.provider.MethodArgumentsProvider.loadRequiredClass(MethodArgumentsProvider.java:180)
        at org.junit.jupiter.params.provider.MethodArgumentsProvider.findFactoryMethodByFullyQualifiedName(MethodArgumentsProvider.java:117)
        at org.junit.jupiter.params.provider.MethodArgumentsProvider.findFactoryMethod(MethodArgumentsProvider.java:82)
        at org.junit.jupiter.params.provider.MethodArgumentsProvider.lambda$provideArguments$1(MethodArgumentsProvider.java:59)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:274)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:274)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:110)
        at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:44)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
        at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
        at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
        at aQute.tester.bundle.engine.BundleDescriptor.executeChild(BundleDescriptor.java:49)
        at aQute.tester.bundle.engine.BundleEngine.lambda$executeBundle$7(BundleEngine.java:120)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
        at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at aQute.tester.bundle.engine.BundleEngine.executeBundle(BundleEngine.java:120)
        at aQute.tester.bundle.engine.BundleEngine.lambda$execute$5(BundleEngine.java:100)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
        at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at aQute.tester.bundle.engine.BundleEngine.execute(BundleEngine.java:100)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
        at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
        at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
        at aQute.tester.junit.platform.Activator.test(Activator.java:441)
        at aQute.tester.junit.platform.Activator.automatic(Activator.java:345)
        at aQute.tester.junit.platform.Activator.run(Activator.java:217)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at aQute.launcher.Launcher.handleMainCallable(Launcher.java:465)
        at aQute.launcher.Launcher.launch(Launcher.java:438)
        at aQute.launcher.Launcher.run(Launcher.java:186)
        at aQute.launcher.Launcher.main(Launcher.java:162)
        at aQute.launcher.pre.EmbeddedLauncher.executeWithRunPath(EmbeddedLauncher.java:170)
        at aQute.launcher.pre.EmbeddedLauncher.findAndExecute(EmbeddedLauncher.java:135)
        at aQute.launcher.pre.EmbeddedLauncher.main(EmbeddedLauncher.java:52)
        Suppressed: org.junit.platform.commons.PreconditionViolationException: Configuration error: You must configure at least one set of arguments for this @ParameterizedTest
                at org.junit.platform.commons.util.Preconditions.condition(Preconditions.java:299)
                at org.junit.jupiter.params.ParameterizedTestExtension.lambda$provideTestTemplateInvocationContexts$5(ParameterizedTestExtension.java:98)
                at java.base/java.util.stream.AbstractPipeline.close(AbstractPipeline.java:323)
                at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271)
                ... 87 more
Caused by: java.lang.ClassNotFoundException: org.testing.junit.TestJUnit
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at org.junit.platform.commons.util.ReflectionUtils.lambda$tryToLoadClass$9(ReflectionUtils.java:831)
        at org.junit.platform.commons.function.Try.lambda$call$0(Try.java:57)
        at org.junit.platform.commons.function.Try.of(Try.java:93)
        at org.junit.platform.commons.function.Try.call(Try.java:57)
        at org.junit.platform.commons.util.ReflectionUtils.tryToLoadClass(ReflectionUtils.java:794)
        at org.junit.platform.commons.util.ReflectionUtils.tryToLoadClass(ReflectionUtils.java:750)
        ... 111 more

[marcphilipp]

There were definitely some changes to MethodArgumentsProvider in 5.9.3 but I suspect the root cause for the class-loading issue is in bnd-bundle-engine.

@chrisr3 Could you please raise an issue with the BND folks and cross-reference it here?

[chrisr3]

Done.

My first impression after examining the exception stack trace was that JUnit was (suddenly?) expecting the test class to be found in the Application classloader:

Caused by: java.lang.ClassNotFoundException: org.testing.junit.TestJUnit
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at org.junit.platform.commons.util.ReflectionUtils.lambda$tryToLoadClass$9(ReflectionUtils.java:831)
        ...

I would consider this to be unlikely when executed within an OSGi framework.

---

According to git bisect:

07f2bf349e45f5a63228cb6002ae7bdbd3445ce1 is the first bad commit
commit 07f2bf349e45f5a63228cb6002ae7bdbd3445ce1
Author: Sam Brannen <sam@sambrannen.com>
Date:   Sat Apr 22 12:20:31 2023 +0200

    Treat overloaded local & external @MethodSource factory methods equally

[chrisr3]

FWIW this small change fixes my test-case:

diff --git a/junit-jupiter-params/src/main/java/org/junit/jupiter/params/provider/MethodArgumentsProvider.java b/junit-jupiter-params/src/main/java/org/junit/jupiter/params/provider/MethodArgumentsProvider.java
index ddcb16b5c4..ba3d9d4874 100644
--- a/junit-jupiter-params/src/main/java/org/junit/jupiter/params/provider/MethodArgumentsProvider.java
+++ b/junit-jupiter-params/src/main/java/org/junit/jupiter/params/provider/MethodArgumentsProvider.java
@@ -108,7 +108,7 @@ class MethodArgumentsProvider extends AnnotationBasedArgumentsProvider<MethodSou
                String className = methodParts[0];
                String methodName = methodParts[1];
                String methodParameters = methodParts[2];
-               Class<?> clazz = loadRequiredClass(className);
+               Class<?> clazz = testMethod.getDeclaringClass();
 
                // Attempt to find an exact match first.
                Method factoryMethod = ReflectionUtils.findMethod(clazz, methodName, methodParameters).orElse(null);

The problem is that loadRequiredClass(className) uses either the TCCL or the Application classloader, neither of which is likely to be able to load the test class within an OSGi framework.

If testMethod.getDeclaringClass() isn't what you want either then maybe:

Class<?> clazz = loadRequiredClass(className, testMethod.getDeclaringClass().getClassLoader());

where

private static Class<?> loadRequiredClass(String className, ClassLoader classLoader) {
    return ReflectionUtils.tryToLoadClass(className, classLoader).getOrThrow(
        cause -> new JUnitException(format("Could not load class [%s]", className), cause));
}

will work instead?