Deprecating Node Protocol

[andrewvc]

The node protocol is a giant pain for both Logstash users and Logstash developers.

Why node is bad for users:
Users think they can use it to gain a speed boost (which it doesn't actually provide). After trying node they then wind up posting bug reports due to broken configurations (setting up a valid node is more complex than HTTP or transport). This makes for disappointed users and disappointed developers.

Why node is bad for developers:

We waste time debugging node configuration errors
We currently have to maintain an extra logstash-output-elasticsearch-license plugin JUST for the use case of shield / marvel + logstash. Additionally, the error messages generated in this scenario are plain ridiculous to debug.
Why user like node

They can see which Logstashes are active via the Elasticsearch node list (the next major of Logstash will cover this with Metrics however)
perceived speed
multicast discoverability (though we now discourage this in Elasticsearch)
IMHO we should deprecate node soon, and consider removing it in the next major release. Perhaps a community member would be interested in maintaining that as a separate plugin.

[suyograo]

The plan is to discourage/deprecate node protocol in 2.2 and remove support for it in 3.0, the next major version.

[markwalkom]

I think this is a good idea.

But we need to be aware that some users choose node so that they have visibility into LS metrics via Marvel, so we'd need to make sure we have metrics APIs to replace this.

[jordansissel]

some users choose node so that they have visibility

Understood, though this was never an intended effect. I hope we don't upset too many folks by removing node for this purpose.

Metrics in Logstash will come soon.

[acchen97]

+1 on discouraging/deprecating node protocol in 2.2 and removing support in the next major.

[djschny]

While the http protocol is now our official stance and I agree with the push of it, there are two items that make me hesitant about doing an action such as a plan to remove all together the node protocol:

• The node protocol is the only one out of the three that has the ability to support multicast discovery (unless I'm missing a way to do it with the other two). I know we recommend unicast heavily, but there is still no denying that some of our user base like and still use multicast. I recall using it before for years on 5 node or less deployments with absolutely no issues. It the benefit of not having to maintain node seed lists was appealing. Unless we offer this ability a pre-step for the transport or http protocols I think it should be deprecated, but not planned for removal.
• it may feel better for us to make the move to EOL our commitment to maintaining this plugin code base all together and let the community decide whether they want to continue to maintain the elasticsearch_java plugin which supports both node and transport.