Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is resque-retry compatible with Rails ActiveJob? #140

Closed
rhymes opened this issue Nov 7, 2016 · 5 comments

Comments

Projects
None yet
3 participants
@rhymes
Copy link

commented Nov 7, 2016

I have a bunch of jobs working perfectly fine with resque, resque-scheduler and resque-pool.

Unfortunately though I can't get resque-retry to work. I queue a thousand jobs, send a SIGHUP to resque-pool (which in turns sends a SIGTERM to the workers) and in the interface I find some of them in the failed queue and they are never ever retried (I waited a few minutes to make sure :D)

Setup

config/initializers/resque.rb:

require 'resque-retry/server'

require 'resque-retry'
require 'resque/failure/redis'
require 'rollbar'

Resque.logger.level = Logger::DEBUG if Rails.env.development?

Resque::Failure::MultipleWithRetrySuppression.classes = [
  Resque::Failure::Redis#, Resque::Failure::Rollbar
]
Resque::Failure.backend = Resque::Failure::MultipleWithRetrySuppression

lib/tasks/resque.rake:

require 'resque/pool/tasks'
require 'resque/scheduler/tasks'

task 'resque:setup' => :environment do
end

task 'resque:pool:setup' => :environment do
  # close any sockets or files in pool manager
  ActiveRecord::Base.connection.disconnect!
  # and re-open them in the resque worker parent
  Resque::Pool.after_prefork do |_job|
    ActiveRecord::Base.establish_connection
  end
end

job:

class DoSomethingJob < ActiveJob::Base
  include Rollbar::ActiveJob
  extend Resque::Plugins::Retry
  @retry_limit = 3
  @retry_delay = 3
  @retry_exceptions = [Resque::TermException]

  queue_as :default

  def perform(argument)
    ...do the stuff...
  end
end

What happens after sending SIGHUP to the pool is that I end up with some jobs in the Resque failed queue and they all failed due to SIGTERM / Resque::TermException without ever retrying.

screen shot 2016-11-07 at 1 41 26 pm

screen shot 2016-11-07 at 1 42 06 pm

Could it be that ActiveJob and resque-retry do not play nice?

I'm using Rails 4.2.7.1 and the current released versions of resque, resque-pool, resque-scheduler and resque-retry

@rhymes

This comment has been minimized.

Copy link
Author

commented Nov 7, 2016

I would like to add that I converted such job to a "Resque job" (a plain class not descending from ActiveJob::Base) and it all works correctly

@lantins

This comment has been minimized.

Copy link
Owner

commented Nov 7, 2016

ActiveJob specifically abstracts away the underlying queue/job system, so I'm not surprised it doesn't work.

We don't have any tests for ActiveJob and I'm not familiar with it myself, that doesn't mean it resque-retry can't work with it or be adapted to.

I'd accept a PR to make it work so long as it doesn't effect vanilla Resque.

Perhaps you could do some research and report back with your findings? :)

Luke

Sent from my iPhone

On 7 Nov 2016, at 13:53, rhymes notifications@github.com wrote:

I would like to add that I converted such job to a "Resque job" (a plain class not descending from ActiveJob::Base) and it all works correctly


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@rhymes

This comment has been minimized.

Copy link
Author

commented Nov 13, 2016

@lantins I've actually converted them to be "pure" Resque jobs, I'm using Rails 4.2.7.1, I reckon stuff has changed now that Rails 5.1 is in the works. I'll close this for now. Thank you!

@rhymes rhymes closed this Nov 13, 2016

@lantins

This comment has been minimized.

Copy link
Owner

commented Nov 14, 2016

Thanks for the update :)

Good luck!

@drn

This comment has been minimized.

Copy link

commented Aug 9, 2017

Old thread, but for what it's worth, I included ActiveJob support by doing this:

# /config/initializers/resque.rb
wrapper_klass = ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper
wrapper_klass.extend(Resque::Plugins::Retry)
wrapper_klass.instance_variable_set(:@retry_limit, 3)
wrapper_klass.instance_variable_set(:@retry_delay, 1.minute)
wrapper_klass.instance_variable_set(:@expire_retry_key_after, 12.hours)
wrapper_klass.instance_variable_set(:@ignore_exceptions, [Resque::TermException])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.