Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Alerting] Index task manager health data into Kibana SM monitoring indices #98373

Closed
chrisronline opened this issue Apr 26, 2021 · 3 comments
Closed
Labels
enhancement New value added to drive a business result estimate:needs-research Estimated as too large and requires research to break down into workable issues Feature:Alerting Feature:Task Manager Team:Monitoring Stack Monitoring team Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)

Comments

@chrisronline
Copy link
Contributor

chrisronline commented Apr 26, 2021

Relates to #95197

As Kibana is responsible for more and more in the Stack, we need to ensure users have the necessary insights into the performance of Kibana from the Stack Monitoring UI.

Alerting currently has a health api for the task manager (api/task_manager/_health) but there are two main problems with it:

  1. There is no corresponding UI that consumes the data and presents it in a useful way to users
  2. The API is unusable if the Kibana instance is under too much load or is unresponsive (which is usually when you want to use the API!)

We should look to moving this data to also exist within the Kibana SM indices so we can eventually visualize it (and alert on it) within Stack Monitoring.

During 7.x, we'll need to make changes to two locations:

  1. Internal collection -> https://github.com/elastic/kibana/blob/master/x-pack/plugins/monitoring/server/kibana_monitoring/bulk_uploader.ts
  2. MB collection -> https://github.com/elastic/kibana/blob/ca17d931deaff7217dc7f75ae1ad6ff3be680543/src/plugins/usage_collection/server/routes/stats/stats.ts

We'll also need to update the mappings (found here) so we can perform aggregations across various fields.

It's not advised to just stick as much data as possible into the index, rather we need to be deliberate with which fields we want to index and map.

@chrisronline chrisronline added enhancement New value added to drive a business result Team:Monitoring Stack Monitoring team Feature:Alerting Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) labels Apr 26, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/stack-monitoring (Team:Monitoring)

@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

@mikecote
Copy link
Contributor

Adding to backlog but up for consideration for 7.14 planning.

@gmmorris gmmorris added the loe:needs-research This issue requires some research before it can be worked on or estimated label Jul 15, 2021
@gmmorris gmmorris added the estimate:needs-research Estimated as too large and requires research to break down into workable issues label Aug 18, 2021
@gmmorris gmmorris removed the loe:needs-research This issue requires some research before it can be worked on or estimated label Sep 2, 2021
@mikecote mikecote removed this from Backlog in Kibana Alerting Jan 6, 2022
@kobelb kobelb added the needs-team Issues missing a team label label Jan 31, 2022
@botelastic botelastic bot removed the needs-team Issues missing a team label label Jan 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New value added to drive a business result estimate:needs-research Estimated as too large and requires research to break down into workable issues Feature:Alerting Feature:Task Manager Team:Monitoring Stack Monitoring team Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)
Projects
No open projects
Development

No branches or pull requests

5 participants