Update default plugins for 5.0

[suyograo]

5.0 is a chance for us to review and remove any plugins that we don't have to bundle by default. These plugins are still supported and maintained, but the default package should only contain most popular, most used plugins.

Removed plugins:

logstash-filter-anonymize: Replaced with newer logstash-filter-fingerprint
logstash-input-zeromq/output: Not a lot of users. Most users seem to prefer rabbitmq, redis, kafka as broker
logstash-codec-netflow: not sure about this.
logstash-filter-multiline: deprecated
logstash-output-hipchat: not maintained. Still at v1 of API.
logstash-output-juggernaut: less usage

Please comment if you'd like to remove more/less

/cc @acchen97 @jordansissel

[ph]

Concerning logstash-codec-netflow, we have a new community maintainer for it, and I can say he is really active.

logstash-output-hipchat Was completely rewritten by me a few month ago and It does support the v2 api and we use the official hipchat gem for it.

logstash-input-zeromq/output and juggernaut I am OK with it.
logstash-filter-multiline, this one is the elephant in the room, but since we are moving away from having a single worker it doesnt make sense to keep it in the core.

[suyograo]

@ph we also need to measure if folks are actively using it. Remember these plugins can still be installed via bin/logstash-plugin install

[ph]

@suyograo Agree, we need to find a way to make the rubygems stats meaningful, right now theses stats are obfuscated by our build process.

[acchen97]

@suyograo I'm +1 on keeping netflow codec in for now. You've removed most of the ones I had in mind, here's a few others proposals.

Remove:

• ganglia output
• gelf output
• nagios output
• nagios_nsca output
• null output

Add:

• translate filter
• webhdfs output

[ph]

@acchen97 I am +1 with your proposed changes

[jsvd]

I know this has been discussed already but I'm a bit on the fence with removing the multiline filter, as we don't have an alternative solution to dealing with multiline after codec, even if it means using a single input and -w 1

Do we have any statistics on how much this plugin is still used today?

Also, I suggest removing the checksum filter, should have been gone a long time ago logstash-plugins/logstash-filter-checksum#2

[acchen97]

@jsvd this is just for removing multiline filter from default package.

Also, I suggest removing the checksum filter, should have been gone a long time ago logstash-plugins/logstash-filter-checksum#2

+1 on removing this from logstash-plugins org, removing output-elasticsearch_http makes sense too.

[suyograo]

@jsvd we are proposing to remove multiline filter from the defaults. Since it is deprecated, if someone has the edge case of using it, then can always install it

[suyograo]

@ph @acchen97 can you guys take another look? Updated

[acchen97]

LGTM

[ph]

LGTM

[elasticsearch-bot]

Suyog Rao merged this into the following branches!

Branch	Commits
master	96c09ea, 33771c1
5.0	1f775e6, e2e995a