Add `strict_loading` mode to optionally prevent lazy loading #37400

eileencodes · 2019-10-08T18:28:24Z

Add #strict to any record to prevent lazy loading of associations.
strict will cascade down from the parent record to all the
associations to help you catch any places where you may want to use
preload instead of lazy loading.

Co-authored-by: Aaron Patterson aaron.patterson@gmail.com

cc/ @rafaelfranca @tenderlove @jhawthorn @matthewd

kaspth · 2019-10-08T18:33:00Z

activerecord/lib/active_record/relation/query_methods.rb

@@ -805,10 +805,21 @@ def readonly(value = true)
    end

    def readonly!(value = true) # :nodoc:
+      p self.method(:readonly_value).source_location


Haha oops. This is what I get for opening a PR that I'd been sitting on for months.

rafaelfranca

I like the idea. Added some comments about the implementation and it would be important to write documentation as well.

rafaelfranca · 2019-10-08T19:10:53Z

activerecord/lib/active_record/associations/belongs_to_association.rb

@@ -89,6 +89,7 @@ def update_counters_via_scope(klass, foreign_key, by)
        end

        def find_target?
+          raise if @owner.readonly?


Should we raise a specific exception here to tell users what this means? BTW, why readonly? should it not be strict?

I think this line can be removed actually (cc @eileencodes). But yes we should add an exception that is specific to this.

rafaelfranca · 2019-10-08T19:12:45Z

activerecord/lib/active_record/associations/preloader/association.rb

@@ -124,8 +124,12 @@ def build_scope
            end

            scope.merge!(reflection_scope) if reflection.scope
-            scope.merge!(preload_scope) if preload_scope
-            scope
+            scope.merge!(preload_scope) if preload_scope && !preload_scope.empty_scope?


I think a two levels if would be easier to understand.

rafaelfranca · 2019-10-08T19:18:09Z

Also should we add this option also to the association declaration?

belongs_to :foo, strict: true? Right now we kinda of can do that by using a lambda, but maybe it is worth an option.

dylanahsmith · 2019-10-08T20:50:52Z

activerecord/test/cases/strict_test.rb

+    assert_predicate dev, :strict?
+    assert dev.audit_logs.all?(&:strict?), "Expected all audit logs to be strict"
+  end
+end


This could use more tests for relation methods. For example, doing dev.audit_logs.first in test_raises_if_strict_and_lazy_loading instead of dev.audit_logs.to_a will not result in an exception, which I find surprising. If we are considering the relation itself to be strict, then I would expect that at least relation methods that use the records if they are loaded? to raise if the relation hasn't been loaded.

I think you want the following to handle those methods on the collection proxy

diff --git a/activerecord/lib/active_record/associations/collection_association.rb b/activerecord/lib/active_record/associations/collection_association.rb index f1b470f97e..958eba114d 100644 --- a/activerecord/lib/active_record/associations/collection_association.rb +++ b/activerecord/lib/active_record/associations/collection_association.rb @@ -306,6 +306,7 @@ def null_scope? def find_from_target? loaded? || + owner.strict? || owner.new_record? || target.any? { |record| record.new_record? || record.changed? } end diff --git a/activerecord/test/cases/strict_test.rb b/activerecord/test/cases/strict_test.rb index 806810484f..5264521992 100644 --- a/activerecord/test/cases/strict_test.rb +++ b/activerecord/test/cases/strict_test.rb @@ -21,6 +21,15 @@ def test_raises_if_strict_and_lazy_loading end end + def test_raises_on_unloaded_relation_methods_if_strict + dev = Developer.strict.first + assert_predicate dev, :strict? + + assert_raises ActiveRecord::StrictViolationError do + dev.audit_logs.first + end + end + def test_preload_audit_logs_are_strict_because_parent_is_strict developer = Developer.first

matthewd · 2019-10-09T03:00:47Z

Is "strict" a sufficiently communicative name? I worry that User.where(admin: true).strict reads a little too easily as a stricter form of condition checking, or something. Is e.g. "strict_loading" too wordy?

Sure would be nice if I hadn't left #32136 to wilt for [checks calendar] a year. 😐

bogdan · 2019-10-09T14:49:29Z

Can you please make sure that Relation#strict_value is either false or true but never nil. nil creates confusion on other relation values like Relation#lock_value or Relation#readonly_value and I don't think we want to continue that design issue.

rafaelfranca · 2019-10-10T18:40:28Z

nil is falsy so there is no problem with returning nil. It is following the Rails guidelines for boolean logic.

bogdan · 2019-10-11T10:02:59Z

Problems with ‘nil’ start when you start using ‘*_value’ directly in tests for example or advanced activerecord code. It is just an ambiguity no one wants to deal with.

lsylvester · 2019-11-05T23:28:12Z

I don't think that this currently works with records that are eager_loaded in a single query. This test fails for me:

  def test_eager_load_audit_logs_are_strict_because_parent_is_strict
    developer = Developer.first

    3.times do
      AuditLog.create(developer: developer, message: "I am message")
    end

    dev = Developer.eager_load(:audit_logs).strict.first

    assert_predicate dev, :strict?
    assert dev.audit_logs.all?(&:strict?), "Expected all audit logs to be strict"
  end

#=> StrictTest#test_preload_audit_logs_are_strict_because_parent_is_strict [./rails/activerecord/test/cases/strict_test.rb:34]:
    Expected all audit logs to be strict

rails-bot · 2020-02-03T23:47:47Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Thank you for your contributions.

louim · 2020-02-06T18:39:57Z

Hey! This seems like a really useful concept and I would be sad to see this PR disappear in the stale void. 🚀

eileencodes · 2020-02-18T20:24:15Z

Ok we've gotten the extra noted cases working and added some new tests. I cleaned up my messy code. 😬

I like @rafaelfranca's suggestion but that feels like a follow-up PR - maybe even a good first issue for someone to tackle.

I changed the name from strict to strict_loading.

Add `#strict_loading` to any record to prevent lazy loading of associations. `strict_loading` will cascade down from the parent record to all the associations to help you catch any places where you may want to use `preload` instead of lazy loading. This is useful for preventing N+1's. Co-authored-by: Aaron Patterson <aaron.patterson@gmail.com>

…:Base.strict_loading_by_default=`. This will allow to enable/disable strict_loading mode by default for a model. The configuration's value is inheritable by subclasses, but they can override that value and it will not impact parent class: ```ruby class Developer < ApplicationRecord self.strict_loading_by_default = true has_many :projects end dev = Developer.first dev.projects.first \# => ActiveRecord::StrictLoadingViolationError Exception: Developer is marked as strict_loading and Project cannot be lazily loaded. ``` What is great about this feature that it could help users to nip N+1 queries in the bud, especially for fresh applications, by setting `ActiveRecord::Base.strict_loading_by_default = true` / `config.active_record.strict_loading_by_default = true`. That is also a great way to prevent new N+1 queries in the existing applications after all the N+1 queries are eliminated. (See https://guides.rubyonrails.org/v6.0/active_record_querying.html#eager-loading-associations, https://github.com/seejohnrun/prelude for details on how to fight against N+1 queries). Related to rails#37400, rails#38541

@laicuRoot

`strict_loading` was added to [Rails 6.1] to prevent lazy loading of associations. As adding it to an association declaration can have a massive impact on the way the record and its association is treated, it can be useful to ensure in a test suite the presence of this option. This adds support for adding the `strict_loading` option to an association declaration. [Rails 6.1]: rails/rails#37400 Co-authored-by: Jose Blanco <jose.blanco@thoughtbot.com> @laicuRoot

@laicuRoot

`strict_loading` was added to [Rails 6.1] to prevent lazy loading of associations. As adding it to an association declaration can have a massive impact on the way the record and its association is treated, it can be useful to ensure in a test suite the presence of this option. This adds support for adding the `strict_loading` option to an association declaration. [Rails 6.1]: rails/rails#37400 Co-authored-by: Jose Blanco @laicuRoot <jose.blanco@thoughtbot.com>

rails-bot bot added the activerecord label Oct 8, 2019

eileencodes changed the title ~~Add strict mode to prevent N+1's~~ Add strict mode to optionally prevent lazy loading Oct 8, 2019

eileencodes added this to the 6.1.0 milestone Oct 8, 2019

kaspth reviewed Oct 8, 2019

View reviewed changes

rafaelfranca reviewed Oct 8, 2019

View reviewed changes

dylanahsmith reviewed Oct 8, 2019

View reviewed changes

rails-bot bot added the stale label Feb 3, 2020

rails-bot bot removed the stale label Feb 6, 2020

eileencodes force-pushed the add-strict-mode branch 2 times, most recently from de5f6bf to 539ed23 Compare February 18, 2020 20:21

eileencodes changed the title ~~Add strict mode to optionally prevent lazy loading~~ Add strict_loading mode to optionally prevent lazy loading Feb 18, 2020

eileencodes force-pushed the add-strict-mode branch from 539ed23 to dbb92f8 Compare February 20, 2020 13:33

eileencodes merged commit ac9b11a into rails:master Feb 20, 2020

eileencodes deleted the add-strict-mode branch February 20, 2020 13:53

kddnewton mentioned this pull request Feb 21, 2020

Support strict_loading on association declarations #38541

Merged

bogdanvlviv mentioned this pull request May 31, 2020

Allow to enable/disable strict_loading mode by default for a model. #39491

Merged

iHiD mentioned this pull request Dec 14, 2020

Switch to Rails 6.1 exercism/website#365

Merged

paranoicsan mentioned this pull request Jan 21, 2021

Add strict_loading instead of bullet gem learningtapestry/lcms-engine#151

Closed

zzak mentioned this pull request Jun 9, 2021

Strict loading does not cascade down to middle records of through associations #42421

Closed

technicalpickles mentioned this pull request Apr 29, 2023

How does this gem differs from Rails strict_loading? charkost/prosopite#60

Open

uurcank mentioned this pull request Oct 22, 2023

add strict loading config comments to sample app generator #49741

Closed

4 tasks

stevepolitodesign mentioned this pull request Nov 9, 2023

Suspenders 3.0.0 thoughtbot/suspenders#1135

Merged

17 tasks

mike-burns mentioned this pull request Dec 12, 2023

Add suspenders:development:environment generator thoughtbot/suspenders#1149

Merged

rhannequin mentioned this pull request Jan 26, 2024

Support association strict_loading option thoughtbot/shoulda-matchers#1607

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `strict_loading` mode to optionally prevent lazy loading #37400

Add `strict_loading` mode to optionally prevent lazy loading #37400

eileencodes commented Oct 8, 2019

kaspth Oct 8, 2019

eileencodes Oct 8, 2019

rafaelfranca left a comment

rafaelfranca Oct 8, 2019

tenderlove Oct 8, 2019

rafaelfranca Oct 8, 2019

rafaelfranca commented Oct 8, 2019

dylanahsmith Oct 8, 2019

dylanahsmith Oct 8, 2019

matthewd commented Oct 9, 2019

bogdan commented Oct 9, 2019 •

edited

Loading

rafaelfranca commented Oct 10, 2019

bogdan commented Oct 11, 2019

lsylvester commented Nov 5, 2019 •

edited

Loading

rails-bot bot commented Feb 3, 2020

louim commented Feb 6, 2020

eileencodes commented Feb 18, 2020

Add strict_loading mode to optionally prevent lazy loading #37400

Add strict_loading mode to optionally prevent lazy loading #37400

Conversation

eileencodes commented Oct 8, 2019

kaspth Oct 8, 2019

Choose a reason for hiding this comment

eileencodes Oct 8, 2019

Choose a reason for hiding this comment

rafaelfranca left a comment

Choose a reason for hiding this comment

rafaelfranca Oct 8, 2019

Choose a reason for hiding this comment

tenderlove Oct 8, 2019

Choose a reason for hiding this comment

rafaelfranca Oct 8, 2019

Choose a reason for hiding this comment

rafaelfranca commented Oct 8, 2019

dylanahsmith Oct 8, 2019

Choose a reason for hiding this comment

dylanahsmith Oct 8, 2019

Choose a reason for hiding this comment

matthewd commented Oct 9, 2019

bogdan commented Oct 9, 2019 • edited Loading

rafaelfranca commented Oct 10, 2019

bogdan commented Oct 11, 2019

lsylvester commented Nov 5, 2019 • edited Loading

rails-bot bot commented Feb 3, 2020

louim commented Feb 6, 2020

eileencodes commented Feb 18, 2020

Add `strict_loading` mode to optionally prevent lazy loading #37400

Add `strict_loading` mode to optionally prevent lazy loading #37400

bogdan commented Oct 9, 2019 •

edited

Loading

lsylvester commented Nov 5, 2019 •

edited

Loading