Should Sidekiq use a dedicated Redis pool?

[collimarco]

Currently we use this configuration:

# concurrency = 25

$redis = ConnectionPool.new(size: 30) { Redis.new(url: ENV['REDIS_URL']) }
Sidekiq.configure_server do |config|
  config.redis = $redis
end
Sidekiq.configure_client do |config|
  config.redis = $redis
end

Note that the same $redis pool is also used in our own application code, for example:

$redis.with do |conn|
  # ...
end

Now I wonder if that is correct, or if that may create a bottleneck.

Is it better to use a separate connection pool for Sidekiq and a different one for the application?

E.g.

# use this pool only in my own code
$redis = ConnectionPool.new(size: 30) { Redis.new(url: ENV['REDIS_URL']) }

# and I define a separate connection pool only for Sidekiq
Sidekiq.configure_server do |config|
  config.redis = ConnectionPool.new(size: 30) { Redis.new(url: ENV['REDIS_URL']) }
end
Sidekiq.configure_client do |config|
  config.redis = ConnectionPool.new(size: 30) { Redis.new(url: ENV['REDIS_URL']) }
end

The answer depends on the Sidekiq usage of that connections (e.g. does it use a connection for each thread? Does it release them immediately or keeps them for the whole execution of a job?).

The answer should probably be documented somewhere (I would be happy to update the wiki).

[mperham]

Your app is using Sidekiq's 25 processor threads to run that code. It will be fine with the single, shared pool.