Very slow perform_later call on ActiveJob

[elohr]

Ruby version: 2.3.3
Sidekiq / Pro / Enterprise version(s): Sidekiq 5.1.0

Initializer:

Sidekiq.configure_server do |config|
  config.redis = ConnectionPool.new(size: Sidekiq.options[:concurrency] + 4, timeout: 1) do
    $REDIS
  end
end

Sidekiq.configure_client do |config|
  config.redis = ConnectionPool.new(size: Sidekiq.options[:concurrency] + 4, timeout: 1) do
    $REDIS
  end
end

I have a SchedulerJob:

class SchedulerJob < ActiveJob::Base
  queue_as :scheduler

  def perform
    logger.debug("Start")
    DelayedJob.set(wait: 10.seconds).perform_later
    logger.debug("After Job 1")
    DelayedJob.set(wait: 20.seconds).perform_later
    logger.debug("After Job 2")
    DelayedJob.set(wait: 30.seconds).perform_later
    logger.debug("End")
  end
end

and a DelayedJob:

class DelayedJob < ActiveJob::Base
  queue_as :delayed_jobs

  def perform
    puts "I'm done"
  end
end

If I call SchedulerJob.new.perform the job runs in just a few milliseconds. If I call SchedulerJob.perform_later to run the job in Sidekiq it takes about 90 seconds to finish, and by looking at the logs I can tell that each of those .perform_later calls takes about 30 seconds each.

Why would this happen?

[mperham]

My guess is you’ve destroyed all concurrency by hacking in one global $REDIS connection into the pool. Don’t do that. Let Sidekiq create and manage the pool of connections.

…

On May 2, 2018, at 14:43, Esteban Lohr ***@***.***> wrote: Ruby version: 2.3.3 Sidekiq / Pro / Enterprise version(s): Sidekiq 5.1.0 Initializer: Sidekiq.configure_server do |config| config.redis = ConnectionPool.new(size: Sidekiq.options[:concurrency] + 4, timeout: 1) do $REDIS end end Sidekiq.configure_client do |config| config.redis = ConnectionPool.new(size: Sidekiq.options[:concurrency] + 4, timeout: 1) do $REDIS end end I have a SchedulerJob: class SchedulerJob < ActiveJob::Base queue_as :scheduler def perform logger.debug("Start") DelayedJob.set(wait: 10.seconds).perform_later logger.debug("After Job 1") DelayedJob.set(wait: 20.seconds).perform_later logger.debug("After Job 2") DelayedJob.set(wait: 30.seconds).perform_later logger.debug("End") end end and a DelayedJob: class DelayedJob < ActiveJob::Base queue_as :delayed_jobs def perform puts "I'm done" end end If I call SchedulerJob.new.perform the job runs in just a few milliseconds. If I call SchedulerJob.perform_later to run the job in Sidekiq it takes about 90 seconds to finish, and by looking at the logs I can tell that each of those .perform_later calls takes about 30 seconds each. Why would this happen? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[elohr]

That fixed it. Thank you!