Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ActiveJob: queue_adapter can be inherited #16992

Merged
merged 1 commit into from Mar 12, 2015

Conversation

tamird
Copy link
Contributor

@tamird tamird commented Sep 20, 2014

@tamird tamird force-pushed the configurable-job-queue-adapter branch from a91699a to 3092006 Compare Sep 24, 2014
@tamird tamird force-pushed the configurable-job-queue-adapter branch 2 times, most recently from b0f3a44 to 4275f40 Compare Sep 24, 2014
@tamird tamird changed the title allow overriding sub job queue_adapter without affecting the base job or... AJ::Base.queue_adapter improvements Sep 24, 2014
@tamird tamird force-pushed the configurable-job-queue-adapter branch from 4275f40 to 208c9b9 Compare Sep 24, 2014
performed_jobs << {job: job.class, args: job.arguments, queue: job.queue_name}
job.perform_now
else
enqueued_jobs << {job: job.class, args: job.arguments, queue: job.queue_name}
Copy link
Member

@lukaszx0 lukaszx0 Sep 24, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have 4 lines like this. Maybe let's return it from a method like job_hash(job) ?

Copy link
Member

@lukaszx0 lukaszx0 Sep 24, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually even more natural would be job.to_hash

Copy link
Contributor Author

@tamird tamird Sep 24, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@tamird tamird force-pushed the configurable-job-queue-adapter branch from 208c9b9 to 22118ce Compare Sep 24, 2014
@cristianbica
Copy link
Member

cristianbica commented Sep 24, 2014

The test adapter was made like this so we can run tests in parallel. @seuros you worked on this
Anyway if you are going to do this make a separate PR

require 'active_support/core_ext/string/inflections'

module ActiveJob
module QueueAdapter
extend ActiveSupport::Concern

included do
class_attribute :_queue_adapter, instance_accessor: false, instance_predicate: false
Copy link
Member

@cristianbica cristianbica Sep 24, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rename _queue_adapter to something else _queue_adapter_class or even queue_adapter_class to be more clear

Copy link
Contributor Author

@tamird tamird Sep 24, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not always a class, though. It may be a callable. I'll change the name to reflect that.

@cristianbica
Copy link
Member

cristianbica commented Sep 24, 2014

Here's the PR when we switch the test adapter to instance methods #16724 (diff)

@tamird tamird force-pushed the configurable-job-queue-adapter branch from 22118ce to 1d5b389 Compare Sep 24, 2014
@tamird tamird mentioned this pull request Sep 24, 2014
@@ -7,6 +7,7 @@

class EnqueuedJobsTest < ActiveJob::TestCase
setup { queue_adapter.perform_enqueued_at_jobs = true }
teardown { queue_adapter.perform_enqueued_at_jobs = false }
Copy link
Member

@seuros seuros Sep 24, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need these teardowns. The test adapter sould to allow parallel test instantiated for every test

Copy link
Contributor Author

@tamird tamird Sep 24, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not anymore, it's a singleton now. See existing discussion in this PR.

@tamird tamird force-pushed the configurable-job-queue-adapter branch 4 times, most recently from d933fb3 to 6f851ec Compare Sep 24, 2014
@cristianbica
Copy link
Member

cristianbica commented Sep 24, 2014

Ok. So ... as discussed the ability to pass a proc to queue_adapter is still under discussion so I suggest to leave this PR with the initial scope of being able to override sub-jobs adapters and start a new PR with the proc thing.
On the other hand we need to document the current change so we need to add documentation to queue_adapter= and queue_adapter. What would be nicer is if you have some time to update the AJ guide :)

@cristianbica
Copy link
Member

cristianbica commented Sep 24, 2014

Also you need to rebase before we're ready to merge

@cristianbica
Copy link
Member

cristianbica commented Sep 24, 2014

I meant to rebase so you can squash your commits

@seuros
Copy link
Member

seuros commented Sep 24, 2014

Then you should split them into multiple PRs.
I don't like the singleton conversion. we had it like that and changed it to instance . cc @jeremy

@lukaszx0
Copy link
Member

lukaszx0 commented Sep 24, 2014

No need to squash or split it into multiple PRs.

Looks good to me 👍

@jeremy
Copy link
Member

jeremy commented Sep 25, 2014

I'd keep TestAdapter as a class with a single instance rather than switching to using a class object as the adapter. Nice cleanup of the adapter internals in either case.

@cristianbica
Copy link
Member

cristianbica commented Oct 8, 2014

@tamir this is the same as #16977 so we can close #16977, right?

@tamir
Copy link

tamir commented Oct 8, 2014

I was erroneously ccd. the mail should have gone to @tamird i think.

On 08 Oct 2014, at 09:02, Cristian Bica notifications@github.com wrote:

@tamir this is the same as #16977 so we can close #16977, right?


Reply to this email directly or view it on GitHub.

@tamird tamird force-pushed the configurable-job-queue-adapter branch from 6f851ec to c80f403 Compare Feb 23, 2015
test_adapter = ActiveJob::QueueAdapters::TestAdapter.new

@old_queue_adapters = (ActiveJob::Base.subclasses << ActiveJob::Base).select do |klass|
klass.respond_to?(:queue_adapter)
Copy link
Member

@jeremy jeremy Mar 12, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every ActiveJob::Base subclass will respond to :queue_adapter since it's a singleton method on Base

Copy link
Contributor Author

@tamird tamird Mar 12, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

derp, you're right. fixed

@tamird tamird force-pushed the configurable-job-queue-adapter branch from 9515e8a to cbaa77d Compare Mar 12, 2015
[klass, klass.queue_adapter].tap do
klass.queue_adapter = test_adapter
end
end
Copy link
Member

@jeremy jeremy Mar 12, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since class attributes no longer inherit from superclass once they're written, assigning #queue_adapter on classes the didn't have it explicitly set will leak side effects between test cases (since changing a superclass attr won't be inherited by its subclass afterward)

Copy link
Contributor Author

@tamird tamird Mar 12, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great catch, fixed

@tamird tamird force-pushed the configurable-job-queue-adapter branch 3 times, most recently from fabae1a to 2a30528 Compare Mar 12, 2015
# only override explicitly set adapters, a quirk of `class_attribute`
klass.singleton_class.public_instance_methods(false).include?(:_queue_adapter)
end.map do |klass|
[klass, klass._queue_adapter].tap do
Copy link
Member

@jeremy jeremy Mar 12, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefer to use the public-API reader?

Copy link
Contributor Author

@tamird tamird Mar 12, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

This allows different `queue_adapters` to be used in each `ActiveJob`
class heirarchy. Previously, all subclasses used a single global queue
adapter.
@tamird tamird force-pushed the configurable-job-queue-adapter branch from 2a30528 to 1f8558f Compare Mar 12, 2015
jeremy added a commit that referenced this pull request Mar 12, 2015
ActiveJob: queue_adapter can be inherited
@jeremy jeremy merged commit 202c68c into rails:master Mar 12, 2015
@tamird tamird deleted the configurable-job-queue-adapter branch Mar 12, 2015
@cristianbica
Copy link
Member

cristianbica commented Mar 12, 2015

❤️ took a few months to get this merged :)

@trungpham
Copy link

trungpham commented Mar 12, 2015

👍

@rebyn
Copy link
Contributor

rebyn commented Apr 13, 2015

I'm not seeing this feature being mentioned in the Basics guide. Or we have an Advanced guide somewhere that I don't know :)?

I think this is cool and deserves a mention. Should I submit a PR or has anyone worked on this already?

@rafaelfranca rafaelfranca modified the milestones: 5.0.0 [temp], 5.0.0 Dec 30, 2015
@benoitongit
Copy link

benoitongit commented Mar 7, 2016

👍
Really cool, thanks! Would be great to add it in the basic guide, it took some time to know it was possible.

@jrochkind
Copy link
Contributor

jrochkind commented Jul 21, 2017

Is code merged in such that it's possible to have different queue adapters for different jobs? (and if so beginning in what Rails version)

I got here following a trail of links between issues and PRs here and in ActiveJob. And the disussion and statuses here make me think this is now possible. But I have no idea how to do it!

If I'm on the right track, could anyone post an example here? If I get an example, I will try to play with it and understand it and do a doc PR demo'ing it in the right place. (If it's already in the docs but I just haven't found it, would very much appreciate if you wanted to point me there!).

@jrochkind
Copy link
Contributor

jrochkind commented Jul 21, 2017

Nevermind, just found it in the guide! "You can also configure your backend on a per job basis." Okay! Great!

@steveburkett
Copy link

steveburkett commented Oct 4, 2017

does anyone have a "patch" i can use in my rails 4.2 app to get this to work?

sharshenov added a commit to veeqo/advanced-sneakers-activejob that referenced this pull request Apr 6, 2020
Rails 5+ allows to set queue adapter per job class
rails/rails#16992
sharshenov added a commit to veeqo/advanced-sneakers-activejob that referenced this pull request Apr 6, 2020
Rails 5+ allows to set queue adapter per job class
rails/rails#16992
sharshenov added a commit to veeqo/advanced-sneakers-activejob that referenced this pull request Apr 6, 2020
Rails 5+ allows to set queue adapter per job class
rails/rails#16992
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet