Heroku

Using sidekiq with Heroku is simple, use Procfile with your Rails app to start a worker process:

web: bundle exec puma ...
worker: bundle exec sidekiq -c 10

See this page for more details about Procfiles, Foreman and Heroku.

WARNING: don't ever use a concurrency value greater than 10 without thorough testing. I've seen dozens of customers crushing their dyno CPUs with concurrency: 25 or higher. This will only make jobs slower.

Redis

To connect to your Redis addon, you'll need to tell Sidekiq which environment variable name to use by setting REDIS_PROVIDER to the name of that variable, e.g. heroku config:set REDIS_PROVIDER=REDISTOGO_URL. Make sure you're using discrete Redis instances for any other purposes such as a Rails cache store. Lacking the REDIS_PROVIDER variable Sidekiq can also use the REDIS_URL environment variable.

Heroku Redis 6.0 and later on production tiers requires TLS encryption but uses self-signed certificates. With the default SSL configuration, connections will fail with a self-signed-certificate error. This behavior can be configured for Sidekiq in an initializer file. See Using Redis: Using an Initializer.

An example:

SIDEKIQ_REDIS_CONFIGURATION = {
  url: ENV.fetch(ENV.fetch("REDIS_PROVIDER", "REDIS_URL"), nil), # use REDIS_PROVIDER for Redis environment variable name, defaulting to REDIS_URL 
  ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_NONE }, # we must trust Heroku and AWS here
}

Sidekiq.configure_server do |config|
  config.redis = SIDEKIQ_REDIS_CONFIGURATION
end

Sidekiq.configure_client do |config|
  config.redis = SIDEKIQ_REDIS_CONFIGURATION
end

Earlier versions of Heroku Redis use stunnel to secure their connection, in conjunction with the heroku/redis buildpack; this requires no custom configuration in Sidekiq, but the buildpack will attempt to stunnel all Heroku Redis connections, including any Redis 6 instances. It is much simpler to keep all Redis instances on the later generation.

The redis gem has supported the verify_mode key since version 4.0.2. The hiredis gem does not work in this environment, so do not include it in your Gemfile if you are using redis with rediss:// and OpenSSL::SSL::VERIFY_NONE.

Configuration

Memory

Setting this environment variable will greatly reduce Sidekiq's memory usage and is highly recommended:

heroku config:set MALLOC_ARENA_MAX=2

As of September 2019 Heroku changed MALLOC_ARENA_MAX to default to 2 for new Ruby applications. Ruby apps created after then shouldn't need to set this environment variable.

Concurrency

I don't recommend setting Sidekiq's concurrency greater than 10. At larger values, you will see poor performance due to CPU contention. With 1x and 2x dynos, I would set concurrency to 5.

Environment Variables

Sidekiq will pay attention to a few well-known environment variables:

• REDIS_PROVIDER - should contain the name of the environment variable for your chosen Redis provider
• RAILS_ENV - "production", "staging", "development", alternative to -e
• RAILS_MAX_THREADS - controls process concurrency, alternative to -c

Dyno Types

The best value dyno types for Rails apps are the 2x and Performance-L dynos. I don't suggest using 1x or Performance-M dynos unless you have a specific reason. If you wish to use a 2x dyno, keep your concurrency low, e.g. -c 3, as 1GB is not much memory for large Rails app. If you are using a Performance-L dyno, Sidekiq will only utilize one of the eight cores of the dyno unless you are using Sidekiq Enterprise's Multi-Process feature.

worker: SIDEKIQ_MAXMEM_MB=1750 bundle exec sidekiqswarm ...

With SIDEKIQ_MAXMEM_MB, the swarm parent process will restart any Sidekiq child processes which bloat past 1.75GB of memory. Since P-Ls have 14GB of memory (8 * 1.75 = 14), this should keep the entire thing within RAM with no more R14 memory errors.

Connection Pool Sizing

Don't tune the pool manually. All modern connection pools in Ruby are lazy, they will use as many connections as they need but no more. Lower Sidekiq's concurrency if you need to use fewer connections.

Shutdown Timeout

Heroku puts a hard limit of 30 seconds on a process restart, Sidekiq's default of -t 25 gives jobs 25 seconds to finish before starting the "force shutdown" procedure. After 25 seconds, Sidekiq will push any remaining unfinished jobs back to Redis and exit. Make sure your jobs are idempotent so they will safely restart when the process starts back up.

You can optionally use require 'sidekiq/api';Sidekiq::ProcessSet.new.each(&:quiet!) in a pre-deploy hook to give your jobs even more time to finish. If your pre-deploy process fails for some reason, you must still terminate Sidekiq using Sidekiq::ProcessSet.new.each(&:stop!). Heroku will start new processes and they will start fetching jobs again. See #5047.

Debugging

If you need to get backtraces from a running Heroku worker process, you can use the Sidekiq API to deliver a TTIN signal:

❯ bundle exec rails c
irb(main):001:0> require 'sidekiq/api'
irb(main):003:0> Sidekiq::ProcessSet.new.each {|ps| ps.dump_threads }

Within 5 seconds, the process should log a backtrace for every thread in the process.