Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Dynamic scale down #2
Hey, this is a really nice fork.
You may however, want to document in this fork/pull request that when setting WORKLESS_MAX_WORKERS to a value greater than one for heroku cedar, Delayed::Job needs to be configured with:
Delayed::Worker.raise_signal_exceptions = :term
otherwise, a race condition occurs whereby if you have multiple workers running and the first worker finishes but a second worker is still going, the call to ::Heroku::API.post_ps_scale is always incorrectly killing the last worker.
This results in a Delayed::Job record that still thinks it's locked to a worker process, however that worker process has been SIGKILL'ed by heroku.
Because this hanging job remains locked (even though it's not being processed), workless doesn't spin down the remaining worker (which isn't doing anything) and you get a race condition with a job that doesn't finish and a worker that doesn't spin down.