KAFKA-15150: Add ServiceLoaderScanner implementation

[gharris1727]

Add the ServiceLoaderScanner, a companion to the ReflectionScanner which does not use reflection to discover plugins.

This will be utilized later in the Plugins class for the configurable-discovery-mode startup scanning, and in the connect-plugin-path script for manifest generation. In this PR, the PluginScannerTest is parameterized to ensure that the ServiceLoaderScanner has parity on the TestPlugins. This required adding manifests for the valid TestPlugins.

Also fixes a bug where the ServiceLoader handles plugin LinkageErrors differently than Reflections. To summarize:

Before:

• Reflections WARN logs LinkageError internally
• ServiceLoader propagates LinkageError and kills the worker

After:

• Reflections behaves the same as before
• JREs with ServiceLoader implementations that make progress before throwing LinkageError:
  • PluginScanner will typically log one error per unique error message, potentially up to 100 errors.
  • If there are >100 consecutive plugins failing with LinkageError, PluginScanner propagates the LinkageError and kills the worker.
• JREs with ServiceLoader implementations that do not make progress before throwing a LinkageError:
  • PluginScanner will typically log a single error, potentially up to 100 errors.
  • PluginScanner will propagate the LinkageError and kills the worker.

Committer Checklist (excluded from commit message)

• Verify design and implementation
• Verify test coverage and CI build status
• Verify documentation (including upgrade notes)

[gharris1727]

@C0urante Could you review this? Thanks!

[C0urante]

Thanks Greg! A couple of nits and one question about testing methodology, then this should be good to go.

[gharris1727]

This appears to have some JDK8 failures that look consistent that i'll have to replicate locally. I've been testing with JDK11, so it's possible that the JDK8 ServiceLoader has some slightly different behavior, since it was overhauled in JDK9.

[C0urante]

Tested the latest commit locally with OpenJDK 8 and 17; everything passes fine. Left one last comment but once that's addressed, this is good to go. Thanks for the diligent work on this, Greg!

[C0urante]

Thanks Greg. I really want to merge this but have some final thoughts. I promise this is it, though!

[C0urante]

LGTM (pending CI build). Thanks Greg, this was a tricky one!

[gharris1727]

Flaky test failures appear unrelated.