Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Ent Rolling Restarts
Do you have long-running jobs which might take minutes or hours to finish? Do you like to deploy frequently? Sidekiq's traditional TSTP+TERM shutdown process can cause jobs to rerun, especially if they are long-running.
Sidekiq Enterprise 1.7.0 adds support for rolling restarts. In this mode, a Sidekiq process will quiet immediately but will not exit until all jobs are complete. There is no limit to the time it can continue running. Upon signalling a rolling restart, a new process will be started to pick up new jobs.
einhorn is a special process manager which manages the rolling restart. Use it to run
einhorn -m10 -- bundle exec sidekiqswarm [...cli args...]
To trigger a rolling restart, you tell einhorn to upgrade the binary after deploying the new code:
einhornsh --execute upgrade
If the new process is still running after 10 seconds, the old process will get USR2 and gracefully shutdown.
Modern init systems implement
restart as explicit
stop operations. You can use
reload to trigger a rolling restart.
Ideally you'll have two types of deploys: the default, "minor" deploy which reloads Sidekiq and a "major" deploy which stops Sidekiq completely, runs any migrations and then starts it back up.
# For Ubuntu systemd users # /lib/systemd/system/sidekiq.service ExecReload=/usr/local/bin/bundle exec einhornsh --execute upgrade
Then a rolling restart is as simple as:
# systemd systemctl reload sidekiq
- Old and new processes will be running at the same time so memory usage will increase.
- Consider compatibility between old and new code versions; take care around database migrations.
- Rolling restarts work with both the
- Einhorn has lots of options which might be useful for various edge cases, e.g. if your app uses Capistrano to deploy.