Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce budget for the Telemetry collectors #119291

Closed
mshustov opened this issue Nov 22, 2021 · 2 comments
Closed

Introduce budget for the Telemetry collectors #119291

mshustov opened this issue Nov 22, 2021 · 2 comments
Labels
Feature:Telemetry performance Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc

Comments

@mshustov
Copy link
Contributor

mshustov commented Nov 22, 2021

Telemetry API allows plugins to query the Elasticsearch server to retrieve stored metadata. In some cases, an Elasticsearch response might have an enormous size as we saw in #115217 that affects the Kibana performance due to the necessity to parse JSON and perform some math operations over the data.

This issue is created to start an investigation on possible solutions to minimize the impact of telemetry on Kibana and Elasticsearch performance. As discussed on October's SDH issues grooming, we might need to enforce more strict rules for querying the Elasticsearch server from Telemetry collectors by introducing a budget for such queries. Telemetry collectors already have dedicated Elasticsearch and SO clients, so we have an extension point to adjust both client configurations to restrict access to API causing performance problems.

Before we take this task on, we need to check the metrics for telemetry collector queries on Cloud to see how far this problem affects our customers.

@mshustov mshustov added Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc performance Feature:Telemetry labels Nov 22, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-core (Team:Core)

@mshustov
Copy link
Contributor Author

closed in favour of #119468

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Telemetry performance Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc
Projects
None yet
Development

No branches or pull requests

2 participants