[8.14] Fix the telemetry collection of Logstash with metricbeat monitoring. (#182304) #183331
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…lastic#182304) ## Summary Telemetry data collection is broken for Logstash, monitoring with metricbeat. This PR change covers following issues faced: **1) Resolve cluster UUID** - With self monitoring, KB creates `.monitoring-es*` index with mapping `type` field and defaults to [`type:cluster_state`](https://github.com/elastic/kibana/blob/main/packages/kbn-apm-synthtrace-client/src/lib/monitoring/cluster_stats.ts#L25) key-value. It uses [`type:cluster_state` condition when fetching cluster UUID](https://github.com/elastic/kibana/blob/main/x-pack/plugins/monitoring/server/telemetry_collection/get_cluster_uuids.ts#L48). - However, with metricbeat, this [_`type` field doesn't exist_ under mapping](https://github.com/elastic/beats/blob/v8.13.2/metricbeat/module/elasticsearch/_meta/fields.yml) which metricbeat creates, so cluster UUID will not be resolved as query is wrong (results empty output). **2) `type` field mismatch in (especially in _state_) queries, also collapse field** - metricbeat sends the _state_ event with `metricset.name:node` and state fetch query doesn't care about this condition, instead uses [legacy `type:logstash_state` condition](https://github.com/elastic/kibana/blob/main/x-pack/plugins/monitoring/server/telemetry_collection/get_logstash_stats.ts#L349) which is incorrect. - in the queries, `collapse` field is not correct: it is due to data shape change from legacy to metricbeat monitoring and queries are tightly coupled with legacy one ([1](https://github.com/elastic/kibana/blob/main/x-pack/plugins/monitoring/server/telemetry_collection/get_logstash_stats.ts#L355), [2](https://github.com/elastic/kibana/blob/main/x-pack/plugins/monitoring/server/telemetry_collection/get_logstash_stats.ts#L346), [3](https://github.com/elastic/kibana/blob/main/x-pack/plugins/monitoring/server/telemetry_collection/get_logstash_stats.ts#L301-L302)) - in the queries, `filter_path` is also not correct: in both [_state_ query](https://github.com/elastic/kibana/blob/main/x-pack/plugins/monitoring/server/telemetry_collection/get_logstash_stats.ts#L332) and [_stats_ query](https://github.com/elastic/kibana/blob/main/x-pack/plugins/monitoring/server/telemetry_collection/get_logstash_stats.ts#L273) **3) Logstash state data frequency** - metricbeat sends _state_ event (_node/stats/pipeline?graph=true) data once but legacy frequently sends. KB telemetry fetcher [queries against ES with latest update period](https://github.com/elastic/kibana/blob/main/x-pack/plugins/monitoring/server/telemetry_collection/get_logstash_stats.ts#L343-L344) where state data will not be available. It might be a reasonable design which results network efficiency. If that's the case, we should decide the expectation - to still collect: we have to tune the query to collect the state data. - leave it as an empty assuming state didn't change (personally, I would not align with this option since collecting once it a data loss risky) --------- Co-authored-by: Alejandro Fernández Haro <afharo@gmail.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 26f6977)
2 tasks
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
Note: this might fail since #183332 is not merged yet. We might need to update this PR after it. |
💚 Build Succeeded
Metrics [docs]Unknown metric groupsESLint disabled line counts
Total ESLint disabled count
To update your PR or re-run it, just comment with: cc @mashhurs |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto8.14:Questions ?
Please refer to the Backport tool documentation