Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Create a blacklist to disallow mutator methods to be delegated to Array #13314

Merged
merged 1 commit into from

5 participants

@laurocaetano
Collaborator

This change was necessary because the whitelist wouldn't work.
It would be painful for users trying to update their applications.

Related with: #12129

activerecord/lib/active_record/relation/delegation.rb
((7 lines not shown))
arel.respond_to?(method, include_private)
end
protected
+ def array_delegable?(method)
+ defined = Array.method_defined?(method)
+
+ if defined && !BLACKLISTED_ARRAY_METHODS.include?(method)
+ true
+ else
+ false
+ end
@jeremy Owner
jeremy added a note
def array_delegable?(method)
  Array.method_defined?(method) && BLACKLISTED_ARRAY_METHODS.exclude?(method)
end
@laurocaetano Collaborator

Nice catch :smile:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
activerecord/lib/active_record/relation/delegation.rb
@@ -45,7 +43,13 @@ def inherited(child_class)
:map, :none?, :one?, :partition, :reject, :reverse,
:sample, :second, :sort, :sort_by, :third,
:to_ary, :to_set, :to_xml, :to_yaml
- ]
+ ] # :nodoc:
+
+ BLACKLISTED_ARRAY_METHODS = [
+ :compact!, :flatten!, :reject!, :reverse!, :rotate!,
+ :shuffle!, :slice!, :sort!, :sort_by!, :delete_if,
+ :keep_if, :pop, :shift, :delete_at, :compact
+ ] # :nodoc:
@jeremy Owner
jeremy added a note

Consider making this a Set. Not sure how lookup times would compare, but at 15 elements it's probably quicker than scanning an Array.

@chancancode Owner

Wouldn't this needs require "set"? Or is it implicitly required through one of the AS files you required here? (Either case, seems better to just explicitly require it here in case those changes, wdyt?)

@jeremy Owner
jeremy added a note

Yes, needs the require :+1:

@laurocaetano Collaborator

Thanks @chancancode :thumbsup:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
activerecord/lib/active_record/relation/delegation.rb
@@ -45,7 +43,13 @@ def inherited(child_class)
:map, :none?, :one?, :partition, :reject, :reverse,
:sample, :second, :sort, :sort_by, :third,
:to_ary, :to_set, :to_xml, :to_yaml
- ]
+ ] # :nodoc:
@jeremy Owner
jeremy added a note

Can remove ARRAY_DELEGATES now.

@rafaelfranca Owner

Maybe we should keep this. It will speed up the lookup for these methods that will not need to pass for method_missing

@jeremy Owner
jeremy added a note

If we keep it, we should generate it from Array + Enumerable instance methods - blacklist. Weird to have a hardcoded list.

@rafaelfranca Owner

I have to agree. Before @laurocaetano's path we have a small hardcoded list but I think it was of common methods between Array and the class. @laurocaetano lets revert to the original code. If any of these methods became hot spots in the future we can add to this list.

@laurocaetano Collaborator

:thumbsup:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
activerecord/lib/active_record/relation/delegation.rb
@@ -45,7 +43,13 @@ def inherited(child_class)
:map, :none?, :one?, :partition, :reject, :reverse,
:sample, :second, :sort, :sort_by, :third,
:to_ary, :to_set, :to_xml, :to_yaml
- ]
+ ] # :nodoc:
+
+ BLACKLISTED_ARRAY_METHODS = [
+ :compact!, :flatten!, :reject!, :reverse!, :rotate!,
+ :shuffle!, :slice!, :sort!, :sort_by!, :delete_if,
+ :keep_if, :pop, :shift, :delete_at, :compact
+ ] # :nodoc:
delegate(*ARRAY_DELEGATES, to: :to_a)
@jeremy Owner
jeremy added a note

Ditto, can remove this now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
activerecord/lib/active_record/relation/delegation.rb
@@ -118,15 +122,28 @@ def relation_class_for(klass)
end
def respond_to?(method, include_private = false)
- super || @klass.respond_to?(method, include_private) ||
+ super || array_delegable?(method) ||
+ super || @klass.respond_to?(method, include_private) ||
@jeremy Owner
jeremy added a note

We have two super calls here now. Remove the second one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
activerecord/CHANGELOG.md
((17 lines not shown))
- To use any other method, instead first call `#to_a` on the association.
+ To use those methods, instead first call `#to_a` on the association.
@jeremy Owner
jeremy added a note

This changelog entry describes the change, but doesn't explain why it was made.

Why is there a blacklist? Why are this methods blacklisted? Why do we want to disallow certain methods?

The presence of the blacklist is not important for the changelog. The fact that Relation no longer delegates bang methods to to_a is important, because it resolves odd bugs and confusion in code that attempts to call these methods directly on a Relation.

@rafaelfranca Owner

To use those methods it is needed to call#to_aon the association.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
activerecord/CHANGELOG.md
((18 lines not shown))
- To use any other method, instead first call `#to_a` on the association.
+ To use those methods it is needed to call `#to_a` on the association.
@laurocaetano Collaborator

:thumbsup:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
activerecord/CHANGELOG.md
@@ -1,15 +1,8 @@
-* Create a whitelist of delegable methods to `Array`.
+* `Relation` no longer has mutator methods like `#map!` and `#delete_if`. Convert
+ to an `Array` before using these methods.
@jeremy Owner
jeremy added a note

by calling #to_a.

@chancancode Owner

It might be nice to add a short example here, "instead of this... [code]... you would do this instead... [code]...)", because the average user is probably using Relation without even knowing it :P

Also, since this requires action, it might be worthy of a mention in the upgrade guide.

@jeremy Owner
jeremy added a note

:+1: on both counts.

@chancancode Owner

By the way, sorry for brining this up now since I missed the original PR, but the original commit included a deprecation message for this change, which I find quite nice, because if I'm upgrading, I'd see the message before I get the error, which give me some hints about what I should do to make things work again.

Maybe it's not exactly a "deprecation warning" in the usual Rails sense, because this doesn't work at all anymore, but some sort of warning or error would be quite nice if I am currently using this today.

@chancancode Owner

Actually, looking at the original PR #12129 vs #12590, the behaviour has shifted subtly.

The original PR simply warns about that this will not be supported in the future (array_delegable? still returns true if the method ends with a bang) but doesn't change the behaviour, which is how breaking changes like these are usually handled (deprecated then removed in next version).

In #12590, the behaviour changed to "this would no longer work without warning". Is this shift intentional? I mean if this is rarely used or will cause serious errors, than maybe this is justified, but otherwise, the original behaviour seems more inline with the usual policy. (It should still at least warn though, imo)

Sorry for bombarding you with questions :P

cc @rafaelfranca

@rafaelfranca Owner

The original intention was to deprecate these methods but after discussing with @jeremy we decided to remove support. If these methods are used right now it will cause unexpected behavior so they are not working at all. Deprecating these method will not help since they are still broken

@laurocaetano Collaborator

Done the changes :)

@chancancode Owner

:+1: @rafaelfranca thanks for providing the context

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
activerecord/CHANGELOG.md
((4 lines not shown))
- Currently `Relation` directly delegates methods to `Array`. With this change,
- only the methods present in this whitelist will be delegated.
-
- The whitelist contains:
-
- #&, #+, #[], #all?, #collect, #detect, #each, #each_cons, #each_with_index,
- #flat_map, #group_by, #include?, #length, #map, #none?, :one?, #reverse, #sample,
- #second, #sort, #sort_by, #to_ary, #to_set, #to_xml, #to_yaml
-
- To use any other method, instead first call `#to_a` on the association.
+ It intents to prevent odd bugs and confusion in code that call mutator
+ methods directely on the `Relation`.
@jeremy Owner
jeremy added a note

intents -> intends, directely -> directly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
activerecord/lib/active_record/relation/delegation.rb
((19 lines not shown))
- delegate(*ARRAY_DELEGATES, to: :to_a)
+ delegate :to_xml, :to_yaml, :length, :collect, :map, :each, :all?, :include?, :to_ary, :to => :to_a

1.9 style hash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@carlosantoniodasilva carlosantoniodasilva commented on the diff
activerecord/lib/active_record/relation/delegation.rb
((16 lines not shown))
def method_missing(method, *args, &block)
if @klass.respond_to?(method)
scoping { @klass.public_send(method, *args, &block) }
+ elsif array_delegable?(method)
+ to_a.public_send(method, *args, &block)

It might be good to keep the same call order when checking respond_to? and method_missing.

@laurocaetano Collaborator

:thumbsup: done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@jeremy
Owner

Please rebase :heart:

@laurocaetano
Collaborator

rebased :green_heart:

@laurocaetano laurocaetano Create a blacklist to disallow mutator methods to be delegated to `Ar…
…ray`.

This change was necessary because the whitelist wouldn't work.
It would be painful for users trying to update their applications.

This blacklist intent to prevent odd bugs and confusion in code that call mutator
methods directely on the `Relation`.
d4ee09c
@jeremy jeremy merged commit d4ee09c into from
@laurocaetano
Collaborator

:green_heart: :heart: :blue_heart: :yellow_heart:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Dec 17, 2013
  1. @laurocaetano

    Create a blacklist to disallow mutator methods to be delegated to `Ar…

    laurocaetano authored
    …ray`.
    
    This change was necessary because the whitelist wouldn't work.
    It would be painful for users trying to update their applications.
    
    This blacklist intent to prevent odd bugs and confusion in code that call mutator
    methods directely on the `Relation`.
This page is out of date. Refresh to see the latest.
View
32 activerecord/CHANGELOG.md
@@ -1,3 +1,20 @@
+* `Relation` no longer has mutator methods like `#map!` and `#delete_if`. Convert
+ to an `Array` by calling `#to_a` before using these methods.
+
+ It intends to prevent odd bugs and confusion in code that call mutator
+ methods directly on the `Relation`.
+
+ Example:
+
+ # Instead of this
+ Author.where(name: 'Hank Moody').compact!
+
+ # Now you have to do this
+ authors = Author.where(name: 'Hank Moody').to_a
+ authors.compact!
+
+ *Lauro Caetano*
+
* Better support for `where()` conditions that use a `belongs_to`
association name.
@@ -80,21 +97,6 @@
*arthurnn*
-* Create a whitelist of delegable methods to `Array`.
-
- Currently `Relation` directly delegates methods to `Array`. With this change,
- only the methods present in this whitelist will be delegated.
-
- The whitelist contains:
-
- #&, #+, #[], #all?, #collect, #detect, #each, #each_cons, #each_with_index,
- #flat_map, #group_by, #include?, #length, #map, #none?, :one?, #reverse, #sample,
- #second, #sort, #sort_by, #to_ary, #to_set, #to_xml, #to_yaml
-
- To use any other method, instead first call `#to_a` on the association.
-
- *Lauro Caetano*
-
* Use the right column to type cast grouped calculations with custom expressions.
Fixes #13230.
View
25 activerecord/lib/active_record/relation/delegation.rb
@@ -1,3 +1,4 @@
+require 'set'
require 'active_support/concern'
require 'active_support/deprecation'
@@ -36,18 +37,13 @@ def inherited(child_class)
# may vary depending on the klass of a relation, so we create a subclass of Relation
# for each different klass, and the delegations are compiled into that subclass only.
- # TODO: This is not going to work. Brittle, painful. We'll switch to a blacklist
- # to disallow mutator methods like map!, pop, and delete_if instead.
- ARRAY_DELEGATES = [
- :+, :-, :|, :&, :[],
- :all?, :collect, :detect, :each, :each_cons, :each_with_index,
- :exclude?, :find_all, :flat_map, :group_by, :include?, :length,
- :map, :none?, :one?, :partition, :reject, :reverse,
- :sample, :second, :sort, :sort_by, :third,
- :to_ary, :to_set, :to_xml, :to_yaml
- ]
+ BLACKLISTED_ARRAY_METHODS = [
+ :compact!, :flatten!, :reject!, :reverse!, :rotate!, :map!,
+ :shuffle!, :slice!, :sort!, :sort_by!, :delete_if,
+ :keep_if, :pop, :shift, :delete_at, :compact
+ ].to_set # :nodoc:
- delegate(*ARRAY_DELEGATES, to: :to_a)
+ delegate :to_xml, :to_yaml, :length, :collect, :map, :each, :all?, :include?, :to_ary, to: :to_a
delegate :table_name, :quoted_table_name, :primary_key, :quoted_primary_key,
:connection, :columns_hash, :to => :klass
@@ -119,14 +115,21 @@ def relation_class_for(klass)
def respond_to?(method, include_private = false)
super || @klass.respond_to?(method, include_private) ||
+ array_delegable?(method) ||
arel.respond_to?(method, include_private)
end
protected
+ def array_delegable?(method)
+ Array.method_defined?(method) && BLACKLISTED_ARRAY_METHODS.exclude?(method)
+ end
+
def method_missing(method, *args, &block)
if @klass.respond_to?(method)
scoping { @klass.public_send(method, *args, &block) }
+ elsif array_delegable?(method)
+ to_a.public_send(method, *args, &block)

It might be good to keep the same call order when checking respond_to? and method_missing.

@laurocaetano Collaborator

:thumbsup: done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
elsif arel.respond_to?(method)
arel.public_send(method, *args, &block)
else
View
15 activerecord/test/cases/relation/delegation_test.rb
@@ -26,15 +26,22 @@ def call_method(target, method)
end
module DelegationWhitelistBlacklistTests
- ActiveRecord::Delegation::ARRAY_DELEGATES.each do |method|
+ ARRAY_DELEGATES = [
+ :+, :-, :|, :&, :[],
+ :all?, :collect, :detect, :each, :each_cons, :each_with_index,
+ :exclude?, :find_all, :flat_map, :group_by, :include?, :length,
+ :map, :none?, :one?, :partition, :reject, :reverse,
+ :sample, :second, :sort, :sort_by, :third,
+ :to_ary, :to_set, :to_xml, :to_yaml
+ ]
+
+ ARRAY_DELEGATES.each do |method|
define_method "test_delegates_#{method}_to_Array" do
assert_respond_to target, method
end
end
- [:compact!, :flatten!, :reject!, :reverse!, :rotate!,
- :shuffle!, :slice!, :sort!, :sort_by!, :delete_if,
- :keep_if, :pop, :shift, :delete_at, :compact].each do |method|
+ ActiveRecord::Delegation::BLACKLISTED_ARRAY_METHODS.each do |method|
define_method "test_#{method}_is_not_delegated_to_Array" do
assert_raises(NoMethodError) { call_method(target, method) }
end
View
17 guides/source/upgrading_ruby_on_rails.md
@@ -156,6 +156,23 @@ end
ActiveRecord::FixtureSet.context_class.send :include, FixtureFileHelpers
```
+### Mutator methods called on Relation
+
+`Relation` no longer has mutator methods like `#map!` and `#delete_if`. Convert
+to an `Array` by calling `#to_a` before using these methods.
+
+It intends to prevent odd bugs and confusion in code that call mutator
+methods directly on the `Relation`.
+
+```ruby
+# Instead of this
+Author.where(name: 'Hank Moody').compact!
+
+# Now you have to do this
+authors = Author.where(name: 'Hank Moody').to_a
+authors.compact!
+```
+
Upgrading from Rails 3.2 to Rails 4.0
-------------------------------------
Something went wrong with that request. Please try again.