Add support for Nomad

[jsoriano]

Nomad is an orchestrator for heterogeneous workloads maintained by Hashicorp.

This issue keeps track of efforts to support Nomad in Beats.

Nomad support could follow a similar approach to the available support for Kubernetes, for that the following components should be implemented:

• nomad autodiscover provider.
• add_nomad_metadata processor.
• Try to use the events API for watching https://www.nomadproject.io/api-docs/events (it is polling now).
• nomad metricbeat module.
• nomad filebeat module for nomad agents.
• Filebeat input if current ones are not enough.
• Documentation and reference job files to deploy beats in nomad.

The code for these features should be placed under x-pack (licensed as Basic).

At the moment of creating this issue there are some open community PRs that provide some of the required components, we have to review and continue with them:

• Add nomad processor #14500 - Adds an add_nomad_metadata processor.
• Autodiscover provider for Nomad #14954 - Adds an add_nomad_metadata processor too, and a nomad autodiscover provider.

There are also some known issues regarding Nomad support (though they can be general issues):

• Stops sending logs after harvester error #12988 - Filebeat fails to collect logs from containers started by nomad
• [Heartbeat] Docker autodiscovery generates monitors for non-published ports #12178 - Problems reaching non-published ports with Docker autodiscover

// cc: @jorgelbg and @jorgemarey as authors of current open PRs

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)

[jsoriano]

Hey @jorgelbg and @jorgemarey,

I have been taking a look to your pull requests and the truth is that both look great, I would love to see Nomad support in Beats. Thanks both for your efforts!

Both PRs contain an add_nomad_metadata processor, and they both work pretty well, but #14954 is more complete, it has support for indexers and matchers, what I think will help to support more Beats and nomad runtimes in the future, and it also includes an autodiscover provider. So I think that it'd be better to go on with this PR only.
In any case, @jorgemarey, if you have some time, it'd be great if you could also take a look to #14954, I am sure you can give some ideas there! For example I like that your processor in #14500 automatically gets the node ID of the local agent.

Thanks again!

[jsoriano]

Initial support merged, what includes an autodiscover provider and an add_nomad_metadata processor. It will be released in principle as experimental feature in 7.12.

[Cobular]

Thank you! Really looking forward to this being fully released.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[joaquinjsb]

👍🏼

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[matya]

The hope dies last 😅