Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gather Elasticsearch and Kibana monitoring data during Cloud load test runs #38

Open
LeeDr opened this issue Jan 27, 2021 · 0 comments
Open
Labels
enhancement New feature or request

Comments

@LeeDr
Copy link

LeeDr commented Jan 27, 2021

Similar to #39
It would be really useful to get the monitoring metrics during load test runs. This would show things like the Elasticsearch search queue and rejected searches. It would also show Kibana memory and response times.

It would be a little challenging to look at the Monitoring page along side of our dashboard so we might re-build some of the monitoring visualizations from the monitoring indices so we can add them to one dashboard.

The problem is, we want to run load tests on the latest master snapshot builds and those are only deployed to Cloud staging. But our kibana-stats.elastic.dev is on Cloud production. We can't configure our load test deployment to send monitoring data from staging to a deployment id in production.
We would need to configure our load test deployment to use the kibana-stats URL, username, password and we don't think we have a way to do that right now.
https://www.elastic.co/guide/en/cloud/current/Deployment_-_CRUD.html#ec_request_example_10

We thought about configuring the load test deployment to keep the monitoring data local and at the end of the job, save a snapshot to cloud storage, and immediately restore that snapshot into kibana-stats.

We also thought about running kibana-stats in Cloud staging so that we could use the deployment id.

/cc @marius-dr @dmlemeshko

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant