Issues building against Logstash 7.0.0

[whmeitzler]

Posting this for discussion and to document my steps for other users; I appreciate that this repo is currently experimental.
It seems this repo has gotten out of sync with the plugin interface in Logstash Core.

1. The steps to build the project are unclear. Currently,build.gradle includes a reference to LOGSTASH_CORE_PATH, which goes nowhere, causing the build to fail. In order to hack things into working, I ended up cloning the core logstash project, generating the requisite jar by calling ~/src/logstash $ gradle jar and replacing LOGSTASH_CORE_PATH with the absolute path of ~/src/logstash/logstash-core/build/libs. Is there a preferred (read: less hacky) way to manage this artifact dependency?

2. It appears the plugin interface in Logstash 7.0.0 has diverged from the state of this repo. The method signature of Filter.filter has changed, as have the steps to generate JavaFilterExample.SOURCE_CONFIG.

[danhermann]

@whmeitzler, thank you for trying out the Java plugin API and reporting back on your experience. Apologies for your trouble as there are clearly a couple deficiencies in the documentation that I will fix based on your feedback. Briefly:

1. The requirement to have a local copy of the Logstash codebase and to properly set the LOGSTASH_CORE_PATH in Gradle is not documented. The best way to do this is clone the Logstash repo and then create a gradle.properties file in the root of your plugin project with the line LOGSTASH_CORE_PATH=/path/to/logstash_repo/logstash-core.
2. The correspondence between the various phases of the Java plugin API and the different releases of Logstash is not clearly documented. The experimental phase of the Java plugin API corresponds to the 6.6.0 release of Logstash which is available in the 6.6 branch of its Github repo. The beta phase of the Java plugin API which did introduce some API changes, especially with filters, corresponds to the upcoming 6.7 release of Logstash which you can find in the 6.7 branch of the Logstash repo. It's also currently in master as no further changes have been made to it. The example plugins have been updated to the beta version of the API in their respective beta branches. E.g., the updated filter plugin.

Please let me know if you have any other questions or run into any other difficulties with this.

[danhermann]

I've updated the README with a new section on setting up the environment. Let me know if that does not address the problems you encountered.

[nielsbasjes]

Yes, I got it to build this way.
In fact I was able to create a first maven pom.xml file that seems to work.

Request: Please push the logstash-core jar file into maven central to make file easier.

[danhermann]

@nielsbasjes, our plan is to eventually publish a jar containing the classes and interfaces in the co.elastic.logstash.api package which are all that are required for Java plugins and much smaller than the logstash-core.jar file.

[holgerbrandl]

@danhermann Any update on when such a jar might land on jcenter/central? Still shipping with the latest 6.7 release or later?

[danhermann]

@holgerbrandl, we have some challenges around the packaging and deployment of Java plugins that we need to resolve before we can determine how best to isolate and publish the logstash-core dependencies for Java plugins. It will definitely be after 6.7 since that went out yesterday.

[holgerbrandl]

Thanks @danhermann for the update