Using Includes and Unscoped

[djiang]

class ShipmentItem < ActiveRecord::Base
  belongs_to :item
  ...
end

class Item < ActiveRecord::Base
  default_scope where(deleted: false)
  ...
end

I've been trying to grab a set of ShipmentItem with an includes(:item) but without the scope:

ShipmentItem.includes(:item)
>> SELECT `shipment_items`.* FROM `shipment_items`
>> SELECT `items`.* FROM `items` WHERE `items`.`deleted` = 0 AND `items`.`id` IN (1, 4)

I've also tried this:

Item.unscoped
   ShipmentItem.includes(:item)
end
>> SELECT `shipment_items`.* FROM `shipment_items`
>> SELECT `items`.* FROM `items` WHERE `items`.`deleted` = 0 AND `items`.`id` IN (1, 4)

I've even tried overriding the item method from within ShipmentItem, but it doesn't work either:

class ShipmentItem < ActiveRecord::Base
  belongs_to :item

  def item
    Item.unscoped { super }
  end
end

Is what I'm trying to do possible? Perhaps I need to restructure my query?

[senny]

/cc @jonleighton

[SubaruWRX55]

i'm having this issue with latest version 3.2.17. Preloader class seems to be a work around.

[rafaelfranca]

Is this also failing with Rails 4?

[JoshDoody]

@rafaelfranca I believe it is still an issue in Rails 4. I posted about a related issue on SO earlier:

http://stackoverflow.com/questions/22341279/why-wont-these-nested-unscoped-blocks-work-to-remove-default-scope-in-rails-4-w

That's not the SAME issue, but looking for a solution to that issue caused me to bump into this one. What I really need to do is eager load objects and their associations when all objects/associations have a default_scope that I want to ignore. In Rails 3.2.17, I was able to do this with nested unscoped blocks like this:

Submission.unscoped { Checklist.unscoped { s = Submission.preload(:checklist).find(3269) }}

Both objects's default_scope would be ignored (see SQL in SO post - the default scope is for tenant_id) and I could say "s.checklist" and get the associated checklist. But once I move to Rails 4.1.0.rc1 (and all other versions of 4 that I tried), the same thing would not return the associated checklist because it would fall back on the default_scope when eager loading the checklist (apparently ignoring the nested unscoped block for Checklist).

My next move was to try to create an ":unscoped_checklist" association for Submission like this:

belongs_to :unscoped_checklist, -> { unscope(where: :tenant_id) }, foreign_key: :checklist_id, class_name: "Checklist"

This association does work in general (if I grab the submission and say submission.unscoped_checklist, the resulting SQL doesn't include the tenant_id from the default_scope). But if I try to eager load the associated checklist, the query for "Checklist load" uses the Checklist default_scope and is unable to load the checklist.

Note that when I call s.checklist, the SQL query is looking at tenant_id because that's the Checklist default_scope, but when I call s.unscoped_checklist, it ignores the default scope.

This is Rails 4.1.0.rc1:

1.9.3-p374 :002 > s = Submission.unscoped.find(3269)
  Submission Load (0.4ms)  SELECT  "submissions".* FROM "submissions"  WHERE "submissions"."id" = $1 LIMIT 1  [["id", 3269]]
 => #<Submission id: 3269, user_id: nil, workday_id: 17, checklist_id: 17, note: nil, submitted: false, submitted_at: nil, created_at: "2014-03-12 01:06:03", updated_at: "2014-03-12 01:06:03", for_date: "2014-03-11", tenant_id: 2, position: 1, department_id: nil, status: "blank"> 
1.9.3-p374 :003 > s.checklist
  Checklist Load (0.2ms)  SELECT  "checklists".* FROM "checklists"  WHERE "checklists"."tenant_id" IS NULL AND "checklists"."id" = $1 LIMIT 1  [["id", 17]]
 => nil 
1.9.3-p374 :004 > s.unscoped_checklist
  Checklist Load (0.2ms)  SELECT  "checklists".* FROM "checklists"  WHERE "checklists"."id" = $1 LIMIT 1  [["id", 17]]
 => #<Checklist id: 17, name: "Open", description: "Chores Required To Open Store Front", creator_id: 2, created_at: "2013-09-23 21:55:23", updated_at: "2013-09-23 21:55:23", archived: false, archived_at: nil, ancestry: nil, tenant_id: 2> 

So the "unscoped_checklist" association does work by bypassing the default_scope. But if I try to preload the unscoped_checklist association, it reverts to the default_scope:

1.9.3-p374 :005 > s = Submission.unscoped.preload(:unscoped_checklist).find(3269)
  Submission Load (0.5ms)  SELECT  "submissions".* FROM "submissions"  WHERE "submissions"."id" = $1 LIMIT 1  [["id", 3269]]
  Checklist Load (0.5ms)  SELECT "checklists".* FROM "checklists"  WHERE "checklists"."tenant_id" IS NULL AND "checklists"."id" IN (17)
 => #<Submission id: 3269, user_id: nil, workday_id: 17, checklist_id: 17, note: nil, submitted: false, submitted_at: nil, created_at: "2014-03-12 01:06:03", updated_at: "2014-03-12 01:06:03", for_date: "2014-03-11", tenant_id: 2, position: 1, department_id: nil, status: "blank"> 

Note in the Checklist Load query for s.unscoped_checklist, the Checklist default_scope is ignored, but when I try to preload the unscoped_checklist association, you see:

WHERE "checklists"."tenant_id" IS NULL

as default_scope is invoked for some reason.

I'm happy to help more if I can. I'm new to Rails issues (and Github issues in general) so just tell me what else you need to replicate and I'll do my best.

EDIT: I forgot to mention that the Preloader workaround seemed to work for me in Rails 3, but the class behaves differently now in Rails 4.

When I used it in Rails 3.2.17, I got this:

preloader = ActiveRecord::Associations::Preloader.new(@submissions,[:checklist])
 => #<ActiveRecord::Associations::Preloader:0x007fa6a9c68a20 @records=[#<Submission id: 2097, user_id: nil, workday_id: 32, checklist_id: 65, note: nil, submitted: false, submitted_at: nil, created_at: "2013-12-11 16:32:21", updated_at: "2013-12-11 16:32:21", for_date: "2013-12-11", tenant_id: 4, position: 4, department_id: nil, status: "blank">, #<Submission id: 3288, user_id: nil, workday_id: 4, checklist_id: 48, note: nil, submitted: false, submitted_at: nil, created_at: "2014-03-12 18:28:10", updated_at: "2014-03-12 18:28:10", for_date: "2014-03-12", tenant_id: 1, position: 12, department_id: nil, status: "blank">], @associations=[:checklist], @options={}> 

So the associated checklist was preloaded as expected.

And in Rails 4.1.0.rc1, I get this:

preloader = ActiveRecord::Associations::Preloader.new(@submissions,[:checklist])
ArgumentError: wrong number of arguments(2 for 0)
    from (irb):161:in `initialize'
    from (irb):161:in `new'
    from (irb):161

I assume the Preloader has been changed, but I couldn't figure out how to use it from the docs.

EDIT 2: For completeness, I figured out the Rails 4 preloaded syntax and thought I should share:

Rails 3.2.17

ActiveRecord::Associations::Preloader.new(@submissions,[:checklist]).run()

Rails 4.1.0.rc1

ActiveRecord::Associations::Preloader.new().preload(@submissions,[:checklist])

[rafaelfranca]

Could you create a self executable gist like this to show the issue? https://github.com/rails/rails/blob/master/guides/bug_report_templates/active_record_master.rb

[JoshDoody]

Here you go: https://gist.github.com/JoshDoody/9531033

The description has the lines listed whose assertions should fail:

"Failing tests for Rails 4 unscoped when used with eager loading (includes). Two assertions should fail: Line 88 and Line 110. Line 88 should pass in Rails 3, but won't in Rails 4. Line 110 leverages the 'unscoped association' added in Rails 4, but still fails."

Let me know how else I can help.

[rafaelfranca]

Thanks! ❤️

[chloerei]

I found similar problems, gitst: https://gist.github.com/chloerei/9893738

[steverob]

Would love this to be sorted out 👍

[calvinl]

+1 just ran into this today. kinda stuck on how to resolve it

[JoshDoody]

@calvinl I think the best way to look at it is you can EITHER unscope OR preload associations, but you can't preload unscoped associations. And I believe you can no longer chain unscoped associations either.

I wound up creating unscoped associations, and I just call those directly when I need them. I recommend that you use the Rails Console to test any ideas you have to see if they work before trying to incorporate them into your code.

Example of unscoped association (which must be called when needed, cannot be preloaded):

class Submission < ActiveRecord::Base
  # class Department has this same default_scope  
  default_scope { where(tenant_id: Tenant.current_id) }

  belongs_to :department
  belongs_to :unscoped_department, -> { unscope(where: :tenant_id) }, foreign_key: :department_id, class_name: "Department"
end

It seems like this issue isn't getting much traction, so hopefully we'll get some more +1s.

[calvinl]

@JoshDoody Any clue how I can do that with a polymorphic association?

[JoshDoody]

@calvinl Hm, I don't really know how you would do that. I'm pretty bad at polymorphism. But I'm glad you asked because this would affect something I have on my list to build, so at least I know this issue is looming there, too. Sorry I can't be of more help there.

[momer]

Note: I'm unable to bypass the default scope with the provided work-around, when using a namespaced class. i.e. the unscoped belongs_to association still applies scope to the query for the below convoluted example:

module Employee
    module Portfolio
        module Largecap
            class Stock
                belongs_to :trading_desk, class_name: "TradingDesk"
                belongs_to :unscoped_trading_desk, -> { unscope(where: :stock_id) }, foreign_key: :trading_desk_id, class_name: "TradingDesk"
            end
        end
    end
end

module Employee
    module Portfolio
        module Largecap
            class TradingDesk
                has_many :stocks
                default_scope { joins(:stock).where('stocks.value > 9000') }
            end
        end
    end
end

For my required functionality, I don't really need the association to be 'real'; I just need the object in this scenario. I added a method as follows

module Employee
    module Portfolio
        module Largecap
            class Stock
                belongs_to :trading_desk, class_name: "TradingDesk"

                def unscoped_trading_desk
                    self.class.name.deconstantize.concat('::TradingDesk').constantize.send(:unscoped).find(self.trading_desk_id)
                end
            end
        end
    end
end

Not ideal, but works for my situation around this issue.

Side note: As to why I'm nesting modules so deeply, it's because I'm relying on engines to provide default functionality to one mount point; and, due to namespacing issues when mounting an engine into multiple different namespaces, I've opted to define the single engine at one mount point, and include its functionality elsewhere via concerns a la TaskRabbit. Would have been dope not to do this, but I would have needed to override or write additional functionality for each mount point anyhow (i.e. not avoiding the nested module ugliness in the parent engine), and this avoids that complexity.

[JoshDoody]

@rafaelfranca Seems like this is still open and, as far as I can tell, hasn't really gotten much traction. Any idea if this will ever be resolved? It's making my code very, very ugly :)

[rafaelfranca]

Any idea if this will ever be resolved?

No idea. If someone want to work on it, it will be fixed.

[bronzle]

Think I figured it out, will post in a bit.

[fphilipe]

Just came across this issue. Took me quite a while to figure out why a relation was nil although I unscoped the default scope of the relation target.

[edwardmp]

Why is this issue closed as it's still occurring in Rails 5.0.2 and 5.0.1? Seems @bronzle has provided a fix but it isn't merged in.. @rafaelfranca

[coderifous]

I ran into the same issue on a Spree-based project, and ran through a bunch of failed approaches before implementing this crutch:

https://gist.github.com/coderifous/33e24f7e63800e169b03a16eb7eebb5b

I preferred this to other approaches because it doesn't involved monkey-patching, or any other clever tricks. It simply clears (and later restores) the default_scopes array.

In my project, dropped that file in lib/default_scope_crutch.rb, and then wrapped my code with something like this:

disable_default_scopes(Product, User) {
  @orders = Order.preload(:product, :user).page(1).per(10)
  @orders.load
}

This was on a Rails 4 codebase, but it looks like it may still be relevant for Rails 5. Hopefully this can be useful for others until a proper solution is figured out.

[estani]

The problem is still in 6.0

a work around that seems to work:

User.unscoped { Company.includes(:users).to_a }

The relation is not storing the fact that it was created within a scope, so forcing the resolution of the relation e.g. via to_a will retrieve the objects from the block and thus bypass the scope as expected.

There are many other ways to do this, the important part is that the object needs to be retrieved while in the block, is not enough to create the relation in the block (at least that was not clear to me)