Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added #or to ActiveRecord::Relation #16052

Closed
wants to merge 1 commit into from
Closed

Conversation

@matthewd
Copy link
Member

matthewd commented Jul 4, 2014

To complement the default and behaviour while building relations, allow two very similar relations to be combined with an or.

Post.where('id = 1').or(Post.where('id = 2'))
# => SELECT * FROM posts WHERE (id = 1) OR (id = 2)

Unlike previous attempts, we avoid any ambiguity about "how much" we're ORing by exclusively accepting a second relation -- thus, we're ORing whatever conditions differ between the two.

This is a much stricter variant of @gaelmuller's #9052; see also @oelmekki's #10891.

/cc @dhh @jeremy @tenderlove @rafaelfranca

    Post.where('id = 1').or(Post.where('id = 2'))
    # => SELECT * FROM posts WHERE (id = 1) OR (id = 2)

[Matthew Draper & Gael Muller]
@dhh
Copy link
Member

dhh commented Jul 4, 2014

API-wise, this seems reasonable to me. Others can have a look at the implementation.

end

def structurally_compatible?(other, allowed_to_vary)
Relation::SINGLE_VALUE_METHODS.all? do |name|

This comment has been minimized.

Copy link
@rafaelfranca

rafaelfranca Jul 4, 2014

Member

I'd extract these conditionals to smaller private methods. I know what they are doing but it would be better if they were easily to nay contributor understand what they are doing.

This comment has been minimized.

Copy link
@matthewd

matthewd Jul 5, 2014

Author Member

Yeah, this ended up a bit uglier than I'd originally anticipated, and I didn't go back and clean it up. Will do.

send("#{name}_values") == other.send("#{name}_values")
end &&
(extending_values - [NullRelation]) == (other.extending_values - [NullRelation]) &&
!limit_value &&

This comment has been minimized.

Copy link
@rafaelfranca

rafaelfranca Jul 4, 2014

Member

Maybe we could put these simple conditional on the begging of the expression since they are cheaper and Ruby would not evaluate the rest of the expression if any of these are true.

This comment has been minimized.

Copy link
@matthewd

matthewd Jul 5, 2014

Author Member

An early exit just means we're heading for an exception, so it probably doesn't gain us much.

@egilburg
Copy link
Contributor

egilburg commented Jul 4, 2014

  1. Is this chainable?

a.or(b).or(c)

  1. If yes, does it combine with and and what's the evaluation order for something like:

a.or(b).and(c).or(d)

@rafaelfranca
Copy link
Member

rafaelfranca commented Jul 4, 2014

@egilburg they only work with relations so they can't be combined with and. We don't even have a and method 😄.

They can be chained but with other relation so b and c need to be something like:

User.where('something').or(User.where('something')).or(User.where('something'))

But I believe something like this is still valid:

User.where('something').or(User.where('something')).where(bar: true)
@oelmekki
Copy link

oelmekki commented Jul 4, 2014

I love this solution, thanks.

I don't see in tests anything specific to handling where.not, does it
works without problems ?

Something like :

Post.where.not('id = 1').or(Post.where.not('id = 2'))

Not that it would be very good code (it's quite hard to understand),
but there probably will be people to try it.

@egilburg
Copy link
Contributor

egilburg commented Jul 4, 2014

Sorry, instead of and I meant just .where().where().where() which of course acts like AND.

In below example what would be evaluation order be?:

User.where(a).where(b).or(User.where(c))

(A && B) || C

or

A && (B || C)

And in below example:

User.where(a).or(User.where(b)).where(c)

Is it:

(A || B) && C

or

A || (B && C)

@gaelmuller
Copy link

gaelmuller commented Jul 4, 2014

Obviously I think this is needed, but I don't see what is wrong with my
proposition.

I personnaly find this notation heavier.

Gael Muller

@rafaelfranca
Copy link
Member

rafaelfranca commented Jul 4, 2014

User.where(a).where(b).or(User.where(c))

(A && B) || C

User.where(a).or(User.where(b)).where(c)

(A || B) && C

@egilburg
Copy link
Contributor

egilburg commented Jul 4, 2014

Ok, so it's always left-to-right. In this case, what ambiguity is solved by using the heavier:

User.where(field1: 'value1').or(User.where(field2: 'value2')).where(field3: 'value3')

As opposed to lighter:

User.where(field1: 'value1').or(field2: 'value2').where(field3: 'value3')

@matthewd
Copy link
Member Author

matthewd commented Jul 4, 2014

and to get the other interpretations:

User.where(a).where(b).or(User.where(a).where(c)))
# or: z = User.where(a); z.where(b).or(z.where(c))

A && (B || C)

User.where(a).or(User.where(b).where(c))

A || (B && C)


In general, I anticipate this being used mostly to combine named scopes, likely often inside another named scope (like this): if you're already dealing with direct where conditions, you're probably still better off using straight SQL.

@dhh
Copy link
Member

dhh commented Jul 4, 2014

User.where(a).or(User.where(b).where(c))
A || (B && C)

On Jul 4, 2014, at 11:05 AM, Eugene Gilburg notifications@github.com wrote:

Ok, so it's always left-to-right. In this case, what ambiguity is solved by using the heavier:

User.where(field1: 'value1').or(User.where(field2: 'value2')).where(field3: 'value3')

As opposed to lighter:

User.where(field1: 'value1').or(field2: 'value2').where(field3: 'value3')


Reply to this email directly or view it on GitHub.

@oelmekki
Copy link

oelmekki commented Jul 4, 2014

@rafaelfranca :

User.where(a).or(User.where(b)).where(c)
(A || B) && C

I think the point is precisely to avoid such order consideration and
only use OR on specific fields :

User.where( foo: 'a' ).or( User.where( foo: 'b' ) ).where( bar: 'c' )

and

User.where( foo: 'a' ).where( bar: 'c' ).or( User.where( foo: 'b' ) )

are both : ( foo = 'a' or foo = 'b' ) and bar = 'c'

Edit : or maybe not, given Matthew answer.

@gaelmuller :

This ordering thing was the problem with your implementation.

@matthewd
Copy link
Member Author

matthewd commented Jul 4, 2014

The main ambiguity in that syntax, I think, is:

User.where(a: 1).where(b: 2).or(c: 3)

That will presumably resolve as "(a = 1 AND b = 2) OR c = 3", as the heavier one would... but how do I express the (arguably) more likely "a = 1 AND (b = 2 OR c = 3)"? (given that I'm in a method on the relation User.where(a: 1))

@egilburg
Copy link
Contributor

egilburg commented Jul 4, 2014

In latter case you'd use:

User.where(a: 1).where(User.where(b: 2).or(c: 3))

So where would allow both a hash syntax or accept a sub-scope inside, allowing combining simple left-to-right interpretations with manual priority order if needed.


common = left_values & right_values
mine = left_values - common
theirs = right_values - common

This comment has been minimized.

Copy link
@sgrif

sgrif Jul 4, 2014

Contributor

Is there a particular reason for this? Shouldn't

WHERE a = 1 AND (b = 2 OR c = 3)

be equivalent to

WHERE ((a = 1 AND b = 2) OR (a = 1 AND c = 3))

This comment has been minimized.

Copy link
@matthewd

matthewd Jul 5, 2014

Author Member
  1. legibility / length of the generated query
  2. keeps the query the same "shape" as the set of relation calls that created it

So, yes.. but it seemed like a nice thing to do, and not unreasonably expensive.

This comment has been minimized.

Copy link
@euwest

euwest Jan 31, 2015

in your example @sgrif if a != 1 wouldn't the first fail after one check whereas the second would still make two checks?

theirs = right_values - common

if mine.any? && theirs.any?
mine = mine.map { |x| String === x ? Arel.sql(x) : x }

This comment has been minimized.

Copy link
@sgrif

sgrif Jul 4, 2014

Contributor

Could this be pulled into a private method?

mine = mine.map { |x| String === x ? Arel.sql(x) : x }
theirs = theirs.map { |x| String === x ? Arel.sql(x) : x }

mine = [Arel::Nodes::And.new(mine)] if mine.size > 1

This comment has been minimized.

Copy link
@sgrif

sgrif Jul 4, 2014

Contributor

Does an And node work with a single child? It feels like it should, the rest of this method could get much simpler if it did.

mine = [Arel::Nodes::And.new(mine)] if mine.size > 1
theirs = [Arel::Nodes::And.new(theirs)] if theirs.size > 1

common << Arel::Nodes::Or.new(mine.first, theirs.first)

This comment has been minimized.

Copy link
@sgrif

sgrif Jul 4, 2014

Contributor

This reads really weird to me, but I don't have a concrete suggestion to improve it.

This comment has been minimized.

Copy link
@sgrif

sgrif Jul 4, 2014

Contributor

Maybe just renaming common to combined or something. This also could become much simpler if an Or node could work when one of its children were empty, or a specialized node for that case.

combining = group_values.any? ? :having : :where

unless structurally_compatible?(other, combining)
raise ArgumentError, 'Relation passed to #or must be structurally compatible'

This comment has been minimized.

Copy link
@sgrif

sgrif Jul 4, 2014

Contributor

Could we provide more guidance as to why they were structurally incompatible? Perhaps structurally_compatible? could become a structural_incompatibilities method, which returns an array of all of the ways in which they are incompatible, the condition could change to structural_incompatibilities.any?, and we could join the result for the error message.

This comment has been minimized.

Copy link
@matthewd

matthewd Jul 5, 2014

Author Member

👍 -- I'd pondered exactly that, so if we've both thought of it, it must be a good idea :)

raise ArgumentError, 'Relation passed to #or must be structurally compatible'
end

unless other.is_a?(NullRelation)

This comment has been minimized.

Copy link
@sgrif

sgrif Jul 5, 2014

Contributor

Would we be able to avoid this is_a? check by defining having_values and where_values on NullRelation?

This comment has been minimized.

Copy link
@matthewd

matthewd Jul 5, 2014

Author Member

👍

@@ -77,5 +77,13 @@ def calculate(operation, _column_name, _options = {})
def exists?(_id = false)
false
end

def or(other)
if other.is_a?(NullRelation)

This comment has been minimized.

Copy link
@sgrif

sgrif Jul 5, 2014

Contributor

Should this just be

def or(other)
  other
end

This comment has been minimized.

Copy link
@matthewd

matthewd Jul 5, 2014

Author Member

My two considerations here were that we should probably return a new relation (even though mutating a relation is not supported, people do it, so I'm hesitant to introduce the first code that actually breaks), and that we should probably still do the compatibility check, lest we mislead someone who's experimenting.

@al2o3cr
Copy link
Contributor

al2o3cr commented Jul 6, 2014

This seems to avoid some of the issues from the previous or attempts (for instance, chaining on an existing where doesn't produce unexpected SQL) but introduces a distinction between bound params and SQL strings:

      # works
      base_rel = Post.where(id: 1)
      base_rel.where("title = 'bar'").or(base_rel.where("title = 'baz'")).to_a

      # fails with "ArgumentError: Relation passed to #or must be structurally compatible"
      base_rel = Post.where(id: 1)
      base_rel.where(title: 'bar').or(base_rel.where(title: 'baz')).to_a

This appears to be caused by the two relations being ord having different bind_values in the second case. I'm not sure how to deal with merging bind_values, but this should work or it's going to be very hard to reliably or relations.

@ghost
Copy link

ghost commented Jul 7, 2014

Maybe you can use a code block as argument to make it simpeler in use:

Post.where(id: 1).or({ where(id: 2) })
@oelmekki
Copy link

oelmekki commented Jul 20, 2014

Btw @matthewd , by far the biggest problem I had with
activerecord_any_of was the lost of atypical information (not WHERE,
HAVING or JOIN) from more elaborated queries (that's why I have scoped
it behind WhereChain to make clear you're supposed to use where like
relations, and not just any kind of relation you want).

I've toyed for a time with the idea of using UNION instead of OR for
those queries (which remain fairly rare compared to usual queries):
better dropping perfs (provided everyone is aware of it) than failing.

I've hit two limitations making this a hard task :

  • arel implementation of UNION is tricky and probably needs refactoring
    for it to become a first class keyword
  • sqlite's support of UNION is problematic, for example you can't use a
    dedicated LIMIT for each statement union'd

In case it helps.

@sgrif
Copy link
Contributor

sgrif commented Nov 5, 2014

It looks like this will break on queries which use bound parameters, we should probably merge the bind_values and add a test for hash style where, as well.

@lucascaton
Copy link
Contributor

lucascaton commented Jan 31, 2015

Nice one! 👍

@duduribeiro
Copy link
Contributor

duduribeiro commented Jan 31, 2015

👍

@brunojabs
Copy link

brunojabs commented Feb 2, 2015

Great! 👍 🎉

@flowerett
Copy link

flowerett commented Feb 3, 2015

🎉

@deniskorobicyn
Copy link

deniskorobicyn commented Feb 3, 2015

👍

@nurey
Copy link

nurey commented Feb 3, 2015

👍
any idea which version of Rails this will land in?

@dhh
Copy link
Member

dhh commented Feb 3, 2015

Rails 5.0.

On Feb 3, 2015, at 11:35, Ilia Lobsanov notifications@github.com wrote:

any idea which version of Rails this will land in?


Reply to this email directly or view it on GitHub.

@EppO
Copy link

EppO commented Feb 6, 2015

Awesome ! 👏

@dalpo
Copy link

dalpo commented Feb 16, 2015

❤️

@dgilperez
Copy link
Contributor

dgilperez commented Mar 20, 2015

Bravo! ❤️

@pinak1180
Copy link

pinak1180 commented Jun 11, 2015

Great..!!! 👍

@sergiotapia
Copy link

sergiotapia commented Jun 11, 2015

Fanastic!

@davidgeere
Copy link

davidgeere commented Jul 26, 2015

It's wonderful but I feel like I didn't see an example above that allows for the following example:

 @person = Person.where("(first_name = :forename OR middle_name = :forename OR nickname = :forename) AND last_name = :last_name AND gender = :gender AND birthday = :birthday", forename: first_name, last_name: last_name, gender: gender, birthday: birthday).first

So I would reuse the first name passed in to try an prepare for the chance that someone could have used the persons middle or nickname instead of their full name. Stupid example but on contracts I am David (first name), to the general public I am Mitchell (middle) and to friends I am Mitch (nickname) and I have been searched for on all of them.

Looking forward to getting the or relation!

send("#{name}_value") == other.send("#{name}_value")
end &&
(Relation::MULTI_VALUE_METHODS - [allowed_to_vary, :extending]).all? do |name|
send("#{name}_values") == other.send("#{name}_values")

This comment has been minimized.

Copy link
@albertossilva

albertossilva Aug 28, 2015

Could you create a method like this?

def validate_strutucture_for(attr)
  lambda { |name| send("#{name}_#{attr}") == other.send("#{name}_#{attr}") }
end

And with that method, you can write

Relation::SINGLE_VALUE_METHODS.all? &validate_strutucture_for("value")

and

(Relation::MULTI_VALUE_METHODS - [allowed_to_vary, :extending]).all? &validate_strutucture_for("values")

What do you think?

This comment has been minimized.

Copy link
@sgrif

sgrif Aug 28, 2015

Contributor

I think that's a bit harder to read in Ruby than what we have now (though it could be broken up)

assert_equal p.loaded?, true
assert_equal expected, p.or(Post.where('id = 2')).to_a
end

This comment has been minimized.

Copy link
@albertossilva

albertossilva Aug 28, 2015

Is this necessary line?

This comment has been minimized.

Copy link
@sgrif

sgrif Aug 28, 2015

Contributor

The code in this PR differs from what was merged. See
9e42cf0

On Fri, Aug 28, 2015, 12:21 PM Alberto notifications@github.com wrote:

In activerecord/test/cases/relation/or_test.rb
#16052 (comment):

  •  assert_equal expected, Post.where('id = 1').or(Post.containing_the_letter_a)
    
  • end
  • def test_or_inside_named_scope
  •  expected = Post.where("body LIKE '\%a\%' OR title LIKE ?", "%'%").order('id DESC').to_a
    
  •  assert_equal expected, Post.order(id: :desc).typographically_interesting
    
  • end
  • def test_or_on_loaded_relation
  •  expected = Post.where('id = 1 or id = 2').to_a
    
  •  p = Post.where('id = 1')
    
  •  p.load
    
  •  assert_equal p.loaded?, true
    
  •  assert_equal expected, p.or(Post.where('id = 2')).to_a
    
  • end

Is this line necessary?


Reply to this email directly or view it on GitHub
https://github.com/rails/rails/pull/16052/files#r38228595.

@bf4
Copy link
Contributor

bf4 commented Sep 1, 2015

FWIW, I've written up a backport of ActiveRelation#or to Rails 4.2.3

@Eric-Guo
Copy link
Contributor

Eric-Guo commented Sep 16, 2015

I just copy @bf4 file to a new gems called where-or

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

You can’t perform that action at this time.