Support `where` with comparison operators (`>`, `>=`, `<`, and `<=`) Take 2 #39863

kamipo · 2020-07-17T14:24:10Z

Revert "Revert "Merge pull request #39613 from kamipo/where_with_custom_operator""

This reverts commit da02291.

posts = Post.order(:id)

posts.where("id >": 9).pluck(:id)  # => [10, 11]
posts.where("id >=": 9).pluck(:id) # => [9, 10, 11]
posts.where("id <": 3).pluck(:id)  # => [1, 2]
posts.where("id <=": 3).pluck(:id) # => [1, 2, 3]

From type casting and table/column name resolution's point of view,
where("created_at >=": time) is better alternative than where("created_at >= ?", time).

class Post < ActiveRecord::Base
  attribute :created_at, :datetime, precision: 3
end

time = Time.now.utc # => 2020-06-24 10:11:12.123456 UTC

Post.create!(created_at: time) # => #<Post id: 1, created_at: "2020-06-24 10:11:12.123000">

# SELECT `posts`.* FROM `posts` WHERE (created_at >= '2020-06-24 10:11:12.123456')
Post.where("created_at >= ?", time) # => []

# SELECT `posts`.* FROM `posts` WHERE `posts`.`created_at` >= '2020-06-24 10:11:12.123000'
Post.where("created_at >=": time) # => [#<Post id: 1, created_at: "2020-06-24 10:11:12.123000">]

As a main contributor of the predicate builder area, I'd recommend to
people use the hash syntax, the hash syntax also have other useful
effects (making boundable queries, unscopeable queries, hash-like
relation merging friendly, automatic other table references detection).

Making boundable queries

While working on #23461, I realized that Active Record doesn't generate
boundable queries perfectly, so I've been improving generated queries to
be boundable for a long time.

e.g.

#26117
7d53993
#39219

Now, where with the hash syntax will generate boundable queries
perfectly.

I also want to generate boundable queries with a comparison operator in
a third party gem, but currently there is no other way than calling
predicate_builder directly.

kufu/activerecord-bitemporal#62

Unscopeable queries, Hash-like relation merging friendly

Unscopeable, and Hash-like merging friendly queries are relying on where
clause is an array of attr with value, and attr name is normalized as a
string (i.e. using User.arel_table[:name] is not preferable for
unscope and merge).

Example:

id = User.arel_table[:id]

users = User.where(id.gt(1).and(id.lteq(10)))

# no-op due to `id.gt(1).and(id.lteq(10))` is not an attr with value
users.unscope(:id)

Automatic other table references detection

It works only for the hash syntax.

ee7f666

pirj · 2020-07-18T10:23:15Z

Won't the following provide roughly the same functionality while being less intrusive?

module WhereComparison
  refine Symbol do
    def >(value)
      return "#{self} > ?", value
    end
  end
end

include WhereComparison
posts = Post.order(:id)

posts.where(:id > 9).count #   (5.2ms)  SELECT COUNT(*) FROM `posts` WHERE (id > 9)

eugeneius · 2020-07-18T22:08:48Z

That refinement provides a roughly-equivalent syntax, but still generates an SQL fragment, which loses the other benefits:

boundable queries, unscopeable queries, hash-like relation merging friendly, automatic other table references detection.

Those benefits are the point of this change, rather than the particular user-facing API. If you don't care about those things, where("id > ?", 9) is already good enough.

This pull request was opened to solicit feedback from the core team (see #39613 (comment)); let's wait for that feedback. 🙂

pirj · 2020-07-19T05:23:23Z

Sure, I understand.
Just a note that the monkey-patched Symbol doesn't necessarily have to return a String with SQL.
It can equally return a lambda, that would call ->(model_or_relation) { model_or_relation.arel_table[:id].gt(9) }, or an object.

In Active Record internal, `arel_table` is not directly used but `arel_attribute` is used, since `arel_table` doesn't normalize an attribute name as a string, and doesn't resolve attribute aliases. For the above reason, `arel_attribute` should be used rather than `arel_table`, but most people directly use `arel_table`, both `arel_table` and `arel_attribute` are private API though. Although I'd not recommend using private API, `arel_table` is actually widely used, and it is also problematic for unscopeable queries and hash-like relation merging friendly, as I explained at rails#39863. To resolve the issue, this change moves Arel attribute normalization (attribute name as a string, and attribute alias resolution) into `arel_table`.

inopinatus · 2020-09-04T07:01:14Z

I've been using the isomorphism of intervals and inequalities for some time, i.e. passing a range to the predicate builder:

posts.where(id: 9..).ids # => [9, 10, 11]
posts.where(id: ...3).ids  # => [1, 2]
posts.where(id: ..3).ids # => [1, 2, 3]

The only missing inequality is greater-than (>), I think that is an omission from Ruby itself that could be fixed.

Edit/addendum: greater-than is the logical inverse of less-than-or-equal. So in a pinch, one may already write posts.where.not(id: ..9).ids # => [10,11]. This still correctly omits rows with nulls, if present. Unfortunately, this inverted form is harder to instantly comprehend, and that "pass around any interval-equivalent range" narrative remains just out of reach.

raghubetina · 2020-11-18T18:18:25Z

@inopinatus I think > is missing because the beginning of a Range is always included in it; it's only the ending that is included or excluded. So 9.. (i.e. 9..Float::INFINITY) is the same, technically, as 9... (i.e. 9...Float::INFINITY); they both include 9 and fail to exclude infinity.

Still, I also was surprised by the missing > when I tried an implicit endless exclusive range with where; so I wrote up #40628 to add it. It doesn't require changes to Ruby itself.

marian13 · 2020-12-22T00:08:18Z

Hi @kamipo.

I have used your suggestion of where with comparison operators for my answers to some old Stack Overflow questions about SQL comparison in Rails (here and here) and it looks like folks like this idea.

Could you tell me, please, do you have any plans to continue its development?

Thanks in advance.

Revert "Revert "Merge pull request rails#39613 from kamipo/where_with_custom_operator"" This reverts commit da02291. ```ruby posts = Post.order(:id) posts.where("id >": 9).pluck(:id) # => [10, 11] posts.where("id >=": 9).pluck(:id) # => [9, 10, 11] posts.where("id <": 3).pluck(:id) # => [1, 2] posts.where("id <=": 3).pluck(:id) # => [1, 2, 3] ``` From type casting and table/column name resolution's point of view, `where("created_at >=": time)` is better alternative than `where("created_at >= ?", time)`. ```ruby class Post < ActiveRecord::Base attribute :created_at, :datetime, precision: 3 end time = Time.now.utc # => 2020-06-24 10:11:12.123456 UTC Post.create!(created_at: time) # => #<Post id: 1, created_at: "2020-06-24 10:11:12.123000"> # SELECT `posts`.* FROM `posts` WHERE (created_at >= '2020-06-24 10:11:12.123456') Post.where("created_at >= ?", time) # => [] # SELECT `posts`.* FROM `posts` WHERE `posts`.`created_at` >= '2020-06-24 10:11:12.123000' Post.where("created_at >=": time) # => [#<Post id: 1, created_at: "2020-06-24 10:11:12.123000">] ``` As a main contributor of the predicate builder area, I'd recommend to people use the hash syntax, the hash syntax also have other useful effects (making boundable queries, unscopeable queries, hash-like relation merging friendly, automatic other table references detection). * Making boundable queries While working on rails#23461, I realized that Active Record doesn't generate boundable queries perfectly, so I've been improving generated queries to be boundable for a long time. e.g. rails#26117 7d53993 rails#39219 Now, `where` with the hash syntax will generate boundable queries perfectly. I also want to generate boundable queries with a comparison operator in a third party gem, but currently there is no other way than calling `predicate_builder` directly. kufu/activerecord-bitemporal#62 * Unscopeable queries, Hash-like relation merging friendly Unscopeable, and Hash-like merging friendly queries are relying on where clause is an array of attr with value, and attr name is normalized as a string (i.e. using `User.arel_table[:name]` is not preferable for `unscope` and `merge`). Example: ```ruby id = User.arel_table[:id] users = User.where(id.gt(1).and(id.lteq(10))) # no-op due to `id.gt(1).and(id.lteq(10))` is not an attr with value users.unscope(:id) ``` * Automatic other table references detection It works only for the hash syntax. ee7f666

jon-sully · 2021-04-02T17:19:31Z

Hoping to not add unneeded noise here but would love to see this go in 👍🏻

For others in the meantime, especially those that want this because (I believe) it allows the query fragment to adhere to the join-table-aliasing introduced in #40106, I'll be using the hash syntax by logically flipping the filter and using an infinite range.

E.g. where (with this PR) I would use

Model.where('created_at <=': Date.today)

I'll flip the where, use an infinite date starting at tomorrow, and use the hash syntax:

Model.where.not(created_at: Date.tomorrow..)

Which does adhere to the join table aliasing (calling out your Model.arel_table[:column] would break it) 👍🏻

p8 · 2021-04-23T13:43:02Z

What about:

Post.where(updated_at: Arel.gt(1.day.ago))
Post.where(author: { name: Arel.like("Ryuta %") })
Post.where(author: { id: Arel.not(42) })

rails-bot · 2021-07-22T15:08:44Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Thank you for your contributions.

jon-sully · 2021-07-22T16:28:00Z

Would still love to see this go in 👍🏻

rails-bot · 2021-10-20T17:09:29Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Thank you for your contributions.

rails-bot · 2022-01-24T03:35:45Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Thank you for your contributions.

jon-sully · 2022-02-26T02:53:00Z

Hoping this can be reopened and re-looked at. The value of having inequality operators with the hash syntax that respects table aliases is worth a serious look IMO. 🙏🏻

jon-sully · 2022-03-03T22:53:01Z

I've obviously thrown my support behind this above but I had to write this not + 'backwards range' again today and hope this PR can get a review/consensus in time 🙂

UserDeal.joins(:deal).where.not(deal: { closes_on: Date.tomorrow... })

While the syntax (stringed-symbol) isn't the cleanest, the quick understandability of this alternative feels really valuable

UserDeal.joins(:deal).where(deal: { "closes_on <=": Date.today })

jon-sully · 2022-11-06T11:58:36Z

Coming back to post here several months later since I've been the squeaky cog in this thread 😆. Just want to add my 2c.

I'm not as sold on the necessity of this particular PR the more and more I use ranges in queries (which already work great). I thought this particular PR would be a great value-add previously but as I've used ranges more in the last year, I've grown to like them. They also bring the benefit of using standard symbol key syntax rather than string symbols.

E.g. where this PR would enable:

Purchase.where("bought_at <=": 5.days.ago)

I've found the range version to be pretty easy to grok too:

# bought before 5 days ago
Purchase.where(bought_at: (...5.days.ago)) # endless range in 'beginless' format requires parens
# or
# bought since 5 days ago
Purchase.where(bought_at: 5.days.ago..) # normal endless range requires no parens — clean!

pirj · 2022-11-06T13:42:38Z

Some gotchas regarding the use of beginless/endless ranges in the community Rails style guide, Where with Ranges, and a related discussion.

abdelbk · 2025-03-17T15:06:59Z

I see that this PR is mentioned here. Is there a way to force aliasing the joined association in a where clause that cannot use the hash syntax ?
All I can think of is creating a scope that adds an always-truthy condition :

scope :joins_my_association, -> { joins(:my_association).where.not(my_association: { id: nil })

scope :other_scope, -> { joins_my_association.where("my_association.created_at > ?", 1.day.ago) }

kamipo · 2025-03-17T16:58:58Z

I see that this PR is mentioned here. Is there a way to force aliasing the joined association in a where clause that cannot use the hash syntax ?
All I can think of is creating a scope that adds an always-truthy condition :
scope :joins_my_association, -> { joins(:my_association).where.not(my_association: { id: nil })

scope :other_scope, -> { joins_my_association.where("my_association.created_at > ?", 1.day.ago) }

There is no official way to do this, but force aliasing as introduced in #40106 is achieved by hacking the automatic references detection in hash syntax, so we can also achieve the same effect by explicitly calling references without using hash syntax.

scope :joins_my_association, -> { joins(:my_association).references(Arel.sql("my_association")) }

scope :other_scope, -> { joins_my_association.where("my_association.created_at > ?", 1.day.ago) }

Alternatively, we can use hash syntax for created_at > by using activerecord-pretty-comparator.

scope :joins_my_association, -> { joins(:my_association) }

scope :other_scope, -> { joins_my_association.where("my_association.created_at >": 1.day.ago) }

jon-sully · 2025-03-17T17:04:13Z

Whoa, I didn't realize activerecord-pretty-comparator was made for all of this 😆 ~~@kamipo didn't you have a PR at one point that would've main-lined that functionality?~~ 🤦‍♂️ this is that PR 🤦‍♂️

Also 👏 that force-the-alias hack is rad. Thanks for that!

abdelbk · 2025-03-17T17:23:29Z

Thank you @kamipo ! The references workaround is neat

rails-bot bot added the activerecord label Jul 17, 2020

rails locked and limited conversation to collaborators Jul 18, 2020

rails unlocked this conversation Jul 18, 2020

kamipo mentioned this pull request Jul 19, 2020

Move Arel attribute normalization into arel_table #39881

Merged

raghubetina mentioned this pull request Nov 18, 2020

Make exclusive endless ranges produce gt queries #40628

Closed

kamipo force-pushed the where_with_comparison_operator branch 2 times, most recently from 72bbb89 to 0f5f596 Compare January 14, 2021 08:36

Base automatically changed from master to main January 14, 2021 17:01

kamipo force-pushed the where_with_comparison_operator branch from 0f5f596 to 8827baa Compare March 11, 2021 02:19

rails-bot bot added the stale label Jul 22, 2021

rails-bot bot removed the stale label Jul 22, 2021

rails-bot bot added the stale label Oct 20, 2021

kamipo removed the stale label Oct 26, 2021

rails-bot bot added the stale label Jan 24, 2022

rails-bot bot closed this Jan 31, 2022

kamipo reopened this Feb 26, 2022

rails-bot bot removed the stale label Feb 26, 2022

kamipo mentioned this pull request May 18, 2022

Statements with timestamp should not be prepared even if prepared_statements is enabled in mysql2 adapter #43005

Open

Support where with comparison operators (>, >=, <, and <=) Take 2 #39863

Are you sure you want to change the base?

Support where with comparison operators (>, >=, <, and <=) Take 2 #39863

Uh oh!

Conversation

kamipo commented Jul 17, 2020

Uh oh!

pirj commented Jul 18, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eugeneius commented Jul 18, 2020

Uh oh!

pirj commented Jul 19, 2020

Uh oh!

inopinatus commented Sep 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

raghubetina commented Nov 18, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

marian13 commented Dec 22, 2020

Uh oh!

jon-sully commented Apr 2, 2021

Uh oh!

p8 commented Apr 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rails-bot bot commented Jul 22, 2021

Uh oh!

jon-sully commented Jul 22, 2021

Uh oh!

rails-bot bot commented Oct 20, 2021

Uh oh!

rails-bot bot commented Jan 24, 2022

Uh oh!

jon-sully commented Feb 26, 2022

Uh oh!

jon-sully commented Mar 3, 2022

Uh oh!

jon-sully commented Nov 6, 2022

Uh oh!

pirj commented Nov 6, 2022

Uh oh!

abdelbk commented Mar 17, 2025

Uh oh!

kamipo commented Mar 17, 2025

Uh oh!

jon-sully commented Mar 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

abdelbk commented Mar 17, 2025

Uh oh!

Uh oh!

Support `where` with comparison operators (`>`, `>=`, `<`, and `<=`) Take 2 #39863

Support `where` with comparison operators (`>`, `>=`, `<`, and `<=`) Take 2 #39863

pirj commented Jul 18, 2020 •

edited

Loading

inopinatus commented Sep 4, 2020 •

edited

Loading

raghubetina commented Nov 18, 2020 •

edited

Loading

p8 commented Apr 23, 2021 •

edited

Loading

jon-sully commented Mar 17, 2025 •

edited

Loading