Join GitHub today
The construct "where.not" doesn't respect boolean algebra #31209
Steps to reproduce
When two predicates A and B are given to a
Model.where(col1: 1, col2: 2).to_sql # => SELECT "models".* FROM "models" WHERE "models"."col1" = 1 AND "models"."col2" = 2
When the clause is negated (using
Model.where.not(col1: 1, col2: 2).to_sql # => SELECT "models".* FROM "models" WHERE ("models"."col1" != 1) AND ("models"."col2" != 2)
In my example, the query should look like:
SELECT "models".* FROM "models" WHERE ("models"."col1" != 1) OR ("models"."col2" != 2)
It's worth noting that the correct behavior can still be obtained by writing SQL directly:
Model.where.not('"models"."col1" = 1 AND "models"."col2" = 2').to_sql # => SELECT "models".* FROM "models" WHERE (NOT ("models"."col1" = 1 AND "models"."col2" = 2))
It makes the construct
The culprit seems to be located here :
Rails version: 5.1.4
Ruby version: MRI 2.4.2
There's an example in the API documentation which shows this behaviour, so it's definitely "we can't silently change this" documented. (Plus I'm sure people are depending on it, and we reallllly don't want to change how queries evaluate.)
I do wish this didn't work the way it does, though. ISTM the best path forward would be a new
We can do all that while only disrupting people who are passing multiple values, so it seems viable on the benefit-vs-inconvenience front.
@jeremy does that plan sound reasonable? I'm not certain about the history... was this meaning a deliberate choice that we don't want to change, or more of an edge case that "just happened"? Do you recall?
For me it is fine the way it is.
A different topic is if we want API to express an OR of negations.
I guess the main reason I'm inclined to change it, separate from any expectation/intuition, is that the alternative seems far more useful to me.
It would be perfectly defensible, designed from scratch, for multiple keys to
We also already have another syntax to briefly AND two not-conditions together:
(Likewise, anyone currently encountering this and just wanting to produce the right expression can use
Yes, agree that ORing would have been just as defensible from scratch, and to choose the behavior based on what is the most common need (I would not be able to say which that is though).
Also, this issue means it is probably a good idea to mention the AND explicitly in the docs, which right now kinda say: "basically same as
I strongly believe the current behavior is the right one. The OP is based on the interpretation that the string
is just a shortcut form for:
I think it's highly desirable then if:
were also just a shortcut for:
So to give it the semantics in the OP, you'd have to give the second form the same semantics, which I think is highly problematic. You'd have to explain that
That's not what's proposed here. The proposal is that
i.e., that "where.not" is a negation of the "where" (taking its arguments as a whole), instead of inverting each argument individually.
Moreover, as I suggested in #31209 (comment), this isn't about what's "right", because whatever we document is right by definition. The question is which is useful. I am contending that the useful opposite of "where these conditions are all true" is "where these conditions are not all true", and not "where these conditions are all false" -- especially when the latter has quite variable interpretation as to the boundary between each condition.
Yes but what's proposed here breaks the equivalence of
The problem comes from interpreting
Of course the whole issue comes from
All predicates in where clause are combined using a logical AND, so this is exactly the same thing.
Most people I talked with are understanding this as
where('NOT(col1 = 1 AND col2 = 2)') = where.not(col1: 1, col2: 2)
The choice of distributing the NOT operator to every argument is arbitrary. Sure, the documented behavior is right by definition, but it would be nicer and less surprising to be compliant with basic boolean algebra. Another consequence, the following assertion is also incorrect if there are more than one condition.
all = where.not(conds) + where(conds)
If you're not aware of this behavior you would probably assume it to be true. The documentation doesn't make it obvious today.
@fterrazzoni A minor quibble:
does not yield any rows with
SQL logic != boolean logic.
Hum, you're totally right about NULLs and indeed, my comment is only valid when all columns are NOT NULL. I should have mentioned that!
However, even in SQL three-valued logic, these statements are always equivalent:
NOT(col1 = 1 AND col2 = 2)
NOT(col1 = 1) OR NOT(col2 = 2)
The NOT never distributes like it does in Rails.
Let me try to rephrase this. What we're talking about here is what's the "right" scope of the NOT in
The current semantics is 1. and in my opinion it's the simplest and most consistent with the ruby language semantics, but you're right it's arbitrary. You prefer 2. and you have your arguments, but you cannot say it's less arbitrary. It's just a matter of preference.
More important from my point of view is the question of why we have to deal with these 'pseudo-logical' expressions instead of being able to write exactly the logical formulas we want. As I was saying before, squeel showed us what it could look like, and the author was saying there were discussions to incorporate it into Rails core. Does anybody know or can point to some discussion on why this never happened?
I know of no such discussion; it might've been before my time. Personally, I find that style of extreme pseudo-ruby DSL to be too fragile for core... but that's what gems are for.
Well, they do generate 1 and respectively 2 SQL conditions. How many conditions does something like
This is an interesting case. I'd have thought that this is exactly the expansion you'd want under your preferred scope for
I was trying to look through my code for anything like this, to see what would be most useful to me. I couldn't find anything exactly like it, but I have a couple of cases where I have a negative condition on some other attribute of a polymorphic association, and in those cases what I needed is an extra condition that checks that the type is still the same as for the elements with the negated condition. So by analogy it seems that what would be most useful to me under option 1 was if it expanded to:
Well, I wouldn't call redefining existing methods for new types of objects extreme pseudo-ruby, I think this is pretty standard technique in object-oriented programming. And actually Arel is already using it, but it's defining them as bitwise operators. Which makes sense, and maybe it would be a little too much to overload them also as regular boolean operators, but it's still pretty bad that we don't have something similar for
The main issue with having this as a separate gem is that it depends a lot on internal Rails stuff, so it's hard to maintain by somebody not part of Rails core team. If Rails core team were willing to maintain it that would be great.
Is this related? I have scopes producing incorrect results on polymorphic "where not":
which returns too few records, rather than the correct
which just excludes one record.
This issue has been automatically marked as stale because it has not been commented on for at least three months.