[META] Java/Maven Plugin Packaging

[andrewvc]

This is taking over from: #9227

Abstract

We're working on a Java API in #9215, but we need a Java friendly way to distribute and install these Java Plugins. We, additionally, want to isolate them via parent-last class loaders.

Our goals are to:

1. Provide an idiomatic way for Java plugin developers to allow them to contribute to Logstash as easily as possible, thus broadening our developer base and improving the developer experience for these users.
2. To make it possible for us to write pure Java plugins (we can already write mixed ones today with a wrapper)

Spec

Note, this isn't super strict, we want to leave some flexibility here to be determined at implementation time

• Artifacts to be packaged as .zip files in a similar manner to [WIP] Java Plugin Experiments #9338 . Final internal layout is TBD.
• Java classes to be loaded via parent-last classloaders to maintain isolation.
• Artifacts to be distributed via a Maven repo. Maven central will be supported for community plugins. We will determine whether official plugins will be distributed through central, or through our own maven repo.
• Existing bin/logstash-plugin commands will continue work as before.
  • Installing a plugin like bin/logstash-plugin install logstash-input-file will first search maven under the co.elastic.logstash.plugins group for an artifact named logstash-input-file, it will fall back to rubygems.org if not found.
  • We will support new addressing to support maven repos of the form repo_url:group_id:artifact_id:version, group_id:artifact:id, and group_id:artifact_id:version for community plugins.

Tasks

• Build sample maven/gradle projects with plugins for creating a LS compatible package
• Make changes to LS plugin management features to support these new commands
• Determine where official LS plugins should be hosted, whether maven central, or our own repo
• Ensure this works with the new Java Plugin API, and consolidate plugin discovery code.

[praseodym]

How will codec plugins work when classloader isolation is used? I think they would need to be loaded in the same classloader as the input/output plugin that will be using it, but that does mean that Logstash cannot just leave codec initialisation up to the input/output plugin itself.

[danhermann]

This issue is obsolete. Java plugins will still be packaged and distributed as Ruby gems. This is nearing completion and its progress is tracked on the top-level meta issue: #9215