Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a chapter on how to prepare for production. #525

Merged
merged 2 commits into from Mar 3, 2022
Merged

Added a chapter on how to prepare for production. #525

merged 2 commits into from Mar 3, 2022

Conversation

stas
Copy link
Contributor

@stas stas commented Feb 18, 2022

Based on the feedback from #523 I tried to put together a chapter with best practices on how to prepare running GoodJob in production.

@bensheldon @reczy would appreciate you taking a look at this. Thank you!

Closes #523

README.md Outdated
Comment on lines 333 to 349
The number of the database connections your setup needs is directly dependent
on the execution mode. A good way to estimate the size of the connections pool
per process running as `:external` execution mode is:

* 1 connection dedicated to the scheduler aka `LISTEN/NOTIFY`
* 1 connection per thread * `GOOD_JOB_MAX_THREADS`
* 1 connection per every extra thread if your application runs these as part
of a job
* 1 connection dedicated to the cron scheduler if you're running it
* 1 connection dedicated to the cron executor if you're running cron

If you're running the queue along with the Rails process, please adjust
the pool size taking into account the number of web server threads!

It is important to understand that the queue process won't crash if the
connections pool is exhausted, instead it will report an exception (eg.
`ActiveRecord::ConnectionTimeoutError`)!
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like this to be isolated inside of the section on database connection pool size.

Overall, I'm imagining the section you're proposing here is more of a checklist of items that explains why you should configure them for production, but the how and details lives in the section related to that specific feature.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bensheldon thanks for reviewing this, I followed your suggestions and moved it as the Database Connections sub-section.

As for the reasoning behind it, we basically were wrapping our heads around trying to compile a production-ready setup and still failed to grasp the whole connections usage part. It can be seen as a checklist, though to me, I just want to make sure our findings are documented, as I'll definitely be returning to them as I use good_jobs in more projects 😉

Copy link
Owner

@bensheldon bensheldon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Thank you!

@bensheldon bensheldon merged commit ed3cf91 into bensheldon:main Mar 3, 2022
@bensheldon bensheldon added the documentation Improvements or additions to documentation label Apr 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Best practices in deploying and monitoring a queue
2 participants