New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fetching metrics from ingester is really slow #283

Closed

tomwilkie opened this issue Feb 10, 2017 · 1 comment

Labels

component/ingester

Contributor

tomwilkie commented Feb 10, 2017

tomwilkie added this to Query Optimisation in Cortex TODO

Contributor Author

tomwilkie commented Apr 9, 2018

We're seeing this again, super annoying. Its on an ingester with a lot of series in memory, which suggests it could be the counting of the number of series: https://github.com/weaveworks/cortex/blob/master/pkg/ingester/user_state.go#L101

Or it could be the minimum of 3 locks that we have to grab to fetch metrics from the ingesters; the block profiler might help us find out.

tomwilkie added the component/ingester label

tomwilkie mentioned this issue

Use global metrics for the ingester #930

Merged

tomwilkie closed this as completed in #930

tomwilkie removed this from Query Optimisation in Cortex TODO

bboreham mentioned this issue

/metrics calls on ingesters taking multiple seconds #1281

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment