Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aggregate: Expire splay should be configurable #435

Closed
gpiucco opened this issue Jun 16, 2021 · 2 comments
Closed

aggregate: Expire splay should be configurable #435

gpiucco opened this issue Jun 16, 2021 · 2 comments

Comments

@gpiucco
Copy link

gpiucco commented Jun 16, 2021

If you have high every/expire values, the aggregator will wait too long before producing metrics.

This is because the random splay added is based on the bucket size.

For example, this could take up to 600 seconds before producing metrics:

aggregate ^foo\..+\.bar$
  every 300 seconds
  expire after 301 seconds
    compute sum write to foo.bar
  send to main
  stop
  ;

In most setups, a few seconds of splay should already be enough to avoid the "thundering herd of expirations" problem.
A simple solution would be to make the value configurable.

@grobian
Copy link
Owner

grobian commented Jun 21, 2021

I can see your point, 600 seems too long with this config indeed

@grobian
Copy link
Owner

grobian commented Apr 27, 2024

how about we just limit the splay to a few seconds instead of anywhere on its interval?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants