4.6-rc-2: JUnit 5 test engine exceptions in initialization are not propagated and do fail the Test task

[mkobit]

JUnit 5 supports extensions automatic registration via ServiceLoader (https://junit.org/junit5/docs/current/user-guide/#extensions-registration-automatic). If in the same projects resources then

Expected Behavior

They should be included.

Current Behavior

They are not included, and an internal exception occurs but the build does not fail.

Context

Migrating from JUnit 5 plugin to Gradle support. The project I am working on uses service loading feature by defining extensions in the same project test resources.

Steps to Reproduce (for bugs)

This is supported in the current JUnit 5 Gradle plugin (https://junit.org/junit5/docs/current/user-guide/#extensions-registration), but does not appear to be working in the migration.

1. Create an extension. For example:

   package mkobit;
   import org.junit.jupiter.api.extension.BeforeEachCallback;
   import org.junit.jupiter.api.extension.ExtensionContext;
   
   class MyDumbExtension implements BeforeEachCallback {
     @Override
     public void beforeEach(final ExtensionContext context) throws Exception {
       System.out.println("We should fail");
       throw new RuntimeException("Things should fail!");
     }
   }

2. Add an extry to a file META-INF/services/org.junit.jupiter.api.extension.Extension in the resources directory. In my project, this is in src/test/resources/META-INF/services/org.junit.jupiter.api.extension.Extension

   mkobit.MyDumbExtension
   

3. Enable auto registration with system property systemProperty('junit.jupiter.extensions.autodetection.enabled', 'true')

4. Run tests, see internal exception and build succeeds

5. Extension is not included

Repo example at https://github.com/mkobit/gradle-4.6-junit-5-issue

Your Environment

Gradle 4.6.1

Scan: https://scans.gradle.com/s/zojscmisf4icq

[oehme]

@mkobit thanks for the report! The repo is empty, did you forget to push your example?

[blindpirate]

Thanks for reporting. This is because your class is package-private and java.util.ServiceLoader cannot instantiate it. An internal exception is not handled properly.

Of course you can change your class to public as a workaround. Anyway Gradle should take this situation into consideration. I'll fix it soon.

[mkobit]

Sorry! Mistakenly did not push up the example.

I think @blindpirate figured out the unhandled exception and my mistake with the ServiceLoader example. I think I may have had a different case for the extension not being discovered, so let me try and reproduce that, too.

[mkobit]

Ok, I tried to reproduce the other not getting discovered issue, and I haven't been able to. I'll adjust the title of this issue to reflect the discovered problem.