Add a unique ID on every plugin and allow user to override it.

[ph]

In the configuration we can define multiples time the same plugin with different configurations, but to associate the metrics to a specific plugins we need to make sure we can correctly identify it. This ID has a side effect to allow us to update the configuration of a specific plugin if needed via an API..

I propose:

• By default every plugins (input, filter, output and codec) generate a unique ID.
• This ID should be still human readable to understand what it refer too
• Users should be allowed to override it.
• Logstash should be smart enough to raise an issue if two plugins has the same ID.
• If would be awesome if a codec ID contains information about the plugin defining it.
• --debug should show the ID in the comment.
• The ID need to be reproducable to associated with the same plugin between each restart.

This is a what a config would look like with user defined ids.

input {
 sqs {
  id => "fast-queue"
  codec  => json { id => "json-fast" }
 }
}

[ph]

I am open to naming suggestion, could be id, uuid or identifier ?

[jsvd]

id is fine.

what's your idea to auto generate a human readable plugin instance id? some concatenation of options + auto increment id?

as for codecs, if we have a pipeline level structure to register the plugin ids then we can nest them, thus when we get a full id of a codec it comes with the parent plugin id too

[ph]

Yes, for that.

what's your idea to auto generate a human readable plugin instance id? some concatenation of options + auto increment id?

[ph]

Agree about the register, this is a good time to finally use/fixed the Logstash Registry class.

[suyograo]

Why would a user care about plugin ID? IOW, when would a user override a plugin ID? A user can query LS to get all grok plugins and this would list IDs. You can then re-query LS using this ID. Will this work?

[ph]

@suyograo This was mostly for a human convenience, to allow them to easily spot where metrics are emitted by which part of their configuration they maintain, since they are the expert in their config/topology.

A common use could be a user with multiples different grok filters, this would make it easy to spot a slower pattern and they could improve it and compare before and after their changes, since creating ID that stay the same could be complicated if the config change, this allow the ID to be resilient to config modification (change in pattern) or adding new plugins to an existing configuration.