Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
To safely shut down Sidekiq, you need to send it the TSTP signal as early as possible in your deploy process and the TERM signal as late as possible. TSTP tells Sidekiq to stop pulling new work and finish all current work. TERM tells Sidekiq to exit within N seconds, where N is set by the
-t timeout option and defaults to 8. Using TSTP+TERM in your deploy process gives your jobs the maximum amount of time to finish before exiting.
If any jobs are still running when the timeout is up, Sidekiq will push those jobs back to Redis so they can be rerun later.
I recommend running 1 or more Sidekiqs per app server in your production cluster. At The Clymb, we run two Sidekiqs on each of our three app servers, for six total processes. With default concurrency of 25, this gives us 150 worker threads. All six processes talk to the same master Redis server and we use
low queues to keep things as simple as possible.
Running separate machines for Sidekiq or using many different queues adds complexity where it is not needed for us.
Using sidekiq with Heroku is simple, add a
Procfile to your Rails app to start a sidekiq worker process:
web: bundle exec puma ... worker: bundle exec sidekiq -t 25 ...
To connect to your Redis addon, you'll need to tell Sidekiq which environment variable name to use by setting REDIS_URL to the name of that variable, e.g.
heroku config:set REDIS_PROVIDER=REDISTOGO_URL. Make sure you're using discrete Redis instances for any other purposes such as a Rails cache store.
Keep in mind that Heroku puts a hard limit of 30 seconds on a process restart, the
-t 25 tells Sidekiq to give the jobs 25 seconds to finish before starting the "force shutdown" procedure. You cannot send
the TSTP signal but you can optionally use
Sidekiq::ProcessSet.new.each(&:quiet!) in a pre-deploy hook to give your jobs even more time to finish. After that 25 second limit, Sidekiq will push any unfinished jobs back to Redis; make sure your jobs are idempotent so it can restart them when the process starts back up.
See this page for more details about Procfiles, Foreman and Heroku.
Setting this environment variable will greatly reduce Sidekiq's memory usage and is highly recommended:
heroku config:set MALLOC_ARENA_MAX=2
Running your own Process
If you want to run Sidekiq on your own server, use Upstart or Systemd to start Sidekiq as a system service. This will ensure the process is restarted if Sidekiq crashes.
Here's an example Upstart config used to manage Sidekiq instances. These files go in
/etc/init and will automatically startup Sidekiq when the machine is booted. The Sidekiq process group can be very simply managed with
[start | stop | restart] workers.
Here's an example systemd unit file.
sidekiqswarm binary makes it trivial to start N Sidekiq processes with a single Upstart/Systemd service. Take a look at Foreman for a useful tool to automate your application processes in development and production.
capistrano-sidekiq gem (github). Integrated support has been removed. Warning: Capistrano uses daemonization by default so if the Sidekiq process crashes, it will not restart automatically.
Sidekiq fires process lifecycle events when starting up and shutting down:
Sidekiq.configure_server do |config| # runs after your app has finished initializing but before any jobs are dispatched. config.on(:startup) do make_some_singleton end config.on(:quiet) do puts "Got TSTP, stopping further job processing..." end config.on(:shutdown) do puts "Got TERM, shutting down process..." stop_the_world end end
This can be extremely useful if you want to start/stop your own threads/actors or signal.
Sidekiq Pro users can use
:quiet to signal long-running jobs that they need to prepare for an upcoming restart (this requires super_fetch to be enabled).
# config/initializers/sidekiq.rb $shutdown_pending = false Sidekiq.configure_server do |config| config.on(:quiet) do $shutdown_pending = true end end # some_job.rb def perform # This job might process 1000s of items and take an hour. # Have each iteration check for shutdown. big_list_of_items # should only return unprocessed items so the loop will continue # where it left off. big_list_of_items.find_each do |item| process(item) # Sidekiq Pro will restart job immediately on process restart raise Sidekiq::Shutdown if $shutdown_pending end end