Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add worker_check_interval configuration option #2759

Merged
merged 1 commit into from Dec 11, 2021
Merged

Add worker_check_interval configuration option #2759

merged 1 commit into from Dec 11, 2021

Conversation

ob-stripe
Copy link
Contributor

@ob-stripe ob-stripe commented Nov 23, 2021

Description

This PR adds support for changing the worker check interval. Previously, the interval was hardcoded to 5 seconds.

Your checklist for this pull request

  • I have reviewed the guidelines for contributing to this repository.
  • I have added (or updated) appropriate tests if this PR fixes a bug or adds a feature.
  • My pull request is 100 lines added/removed or less so that it can be easily reviewed.
  • If this PR doesn't need tests (docs change), I added [ci skip] to the title of the PR.
  • If this closes any issues, I have added "Closes #issue" to the PR description or my commit messages.
  • I have updated the documentation accordingly.
  • All new and existing tests passed, including Rubocop.

@nateberkopec nateberkopec added feature waiting-for-review Waiting on review from anyone labels Dec 10, 2021
@nateberkopec
Copy link
Member

@nateberkopec nateberkopec commented Dec 10, 2021

Why did the default interval not work for your use case?

@nateberkopec
Copy link
Member

@nateberkopec nateberkopec commented Dec 10, 2021

Also, you marked this as WIP - what remains to be done?

@ob-stripe ob-stripe changed the title [WIP] Add worker_check_interval configuration option Add worker_check_interval configuration option Dec 10, 2021
@ob-stripe
Copy link
Contributor Author

@ob-stripe ob-stripe commented Dec 10, 2021

Why did the default interval not work for your use case?

Currently, Stripe runs multiple Puma servers in single mode on each host, using Einhorn as the socket manager / controller process. Our monitoring system queries Einhorn every 2 seconds to get the list of all live workers, then queries each worker's control server to compute overall availability.

Einhorn is deprecated and we're in the process of replacing it with Puma cluster mode. While updating the monitoring system to support querying the Puma cluster main process rather than Einhorn, I noticed that worker statuses are only updated every 5 seconds. I raised this internally and there were concerns around changing the granularity from 2 seconds to 5 seconds. For the time being we're monkey-patching Puma's WORKER_CHECK_INTERVAL constant but having a dedicated configuration option would be much cleaner.

Also, you marked this as WIP - what remains to be done?

Nothing, actually. This is ready for review. Happy to answer any questions or address feedback!

@nateberkopec nateberkopec merged commit 1008c36 into puma:master Dec 11, 2021
47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature waiting-for-review Waiting on review from anyone
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants