[Logstash] Add Node Dashboards to Logstash integration package

[robbavey]

What does this PR do?

This PR adds dashboards to the Logstash integration.

These dashboards are intended as the first phase in a long term replacement of the
dedicated stack monitoring UI with standard Kibana visualizations, and are intended to
allow serverless users to continue to monitor Logstash.

This PR includes a Logstash overview dashboard, an overview of all the Logstash nodes, allowing
drill downs to individual nodes, with dashboards for general information on a single node, and a more
advanced view of a single node.

The intention of this commit is to mirror the existing stack monitoring UI as much as possible, using the
current node.stats data available.

Limitations:

• Dashboards related to pipelines will be addressed in follow-up PRs - the current data in the integration does
  not fit well with standard Kibana visualizations, and will be addressed separately
• The Advanced node view differs from the Stack Monitoring UI:
  • The size of the persistent queue is not presented here. PQ size statistics are only presented in the node.state
    document, which we do not want to use going forward. Additionally, the existing UI has significant limitations, in that
    only the total size across all PQs is shown, which is misleading when multiple PQs are available.
  • Additional graphs showing OS and Process CPU usage, and file descriptor usage are added

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Related issues

Screenshots

Logstash Overview

This PR:

Existing Stack Monitoring UI:

Nodes Overview

This PR:

Existing Stack Monitoring UI:

Single Node Overview

This PR:

Existing Stack Monitoring UI:

Single Node Advanced View

This PR:

Existing Stack Monitoring UI

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-09-08T17:53:51.381+0000

• Duration: 18 min 34 sec

Test stats 🧪

Test	Results
Failed	0
Passed	29
Skipped	0
Total	29

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (2/2)	💚
Files	100.0% (6/6)	💚
Classes	100.0% (6/6)	💚
Methods	100.0% (46/46)	💚 24.242
Lines	87.558% (190/217)	👎 -12.442
Conditionals	100.0% (0/0)	💚

[robbavey]

/test

[miltonhultgren]

I'll review this now, have you considered linking these new dashboards into the SM UI, similar to what was done recently for Ingest Pipeline monitoring?
elastic/kibana#149721

[miltonhultgren]

Not much to review given the generated dashboard JSON!
The only thing to note is that it should be possible to make the dashboards also read from the .monitoring-* data streams to make it work with Beats collection as well, which would make it available to use also for ESS users.

[robbavey]

@miltonhultgren Thank you for the fast turnaround!

I'll review this now, have you considered linking these new dashboards into the SM UI, similar to what was done recently for Ingest Pipeline monitoring?

This commit is the starting phase of a project to replace the stack monitoring UI for Logstash with standard Kibana visualizations, rather than rely on the custom UI.
In particular, we are doing this now to allow Logstash to be monitored with the serverless solution, which does not include any stack monitoring UI at all.
Eventually, when we port over the missing functionality from the stack monitoring UI, I would anticipate that we would replace that with a link to the Overview dashboard here.

The only thing to note is that it should be possible to make the dashboards also read from the .monitoring-* data streams to make it work with Beats collection as well, which would make it available to use also for ESS users.

Done - I've added an index pattern along the same lines as that done in the elasticsearch package.

[miltonhultgren]

Eventually, when we port over the missing functionality from the stack monitoring UI, I would anticipate that we would replace that with a link to the Overview dashboard here.

Sounds good to me!

[robbavey]

@miltonhultgren Is this good to go after the index pattern changes?

[miltonhultgren]

Yes, this LGTM!

[elasticmachine]

Package logstash - 2.3.3 containing this change is available at https://epr.elastic.co/search?package=logstash

[elasticmachine]

Package logstash - 2.3.4 containing this change is available at https://epr.elastic.co/search?package=logstash