New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All of queries should return correct result even if including large number #30000

Open
wants to merge 2 commits into
from

Conversation

Projects
None yet
4 participants
@kamipo
Member

kamipo commented Jul 30, 2017

Currently several queries cannot return correct result due to incorrect
RangeError handling.

First example:

assert_equal true, Topic.where(id: [1, 9223372036854775808]).exists?
assert_equal true, Topic.where.not(id: 9223372036854775808).exists?

The first example is obviously to be true, but currently it returns
false.

Second example:

assert_equal topics(:first), Topic.where(id: 1..9223372036854775808).find(1)

The second example also should return the object, but currently it
raises RecordNotFound.

It can be seen from the examples, the queries including large number
assuming empty result is not always correct.

Therefore, This change handles RangeError to generate executable SQL
instead of raising RangeError to users to always return correct
result. By this change, it is no longer raised RangeError to users.

@matthewd

This comment has been minimized.

Show comment
Hide comment
@matthewd

matthewd Jul 30, 2017

Member

Ugh... I'm sure I was worried about these sorts of cases when we added that rescue 😕

I don't think replacing binds with inline values is the solution, though. It provides a trivial means of blowing out the prepared statement cache, and even in the best case gives a surprise inefficient query.. what was an index lookup is now a full table scan.

If we can't pick out just the known cases, like find(hugenum), I think we have to just let the exception raise.

Member

matthewd commented Jul 30, 2017

Ugh... I'm sure I was worried about these sorts of cases when we added that rescue 😕

I don't think replacing binds with inline values is the solution, though. It provides a trivial means of blowing out the prepared statement cache, and even in the best case gives a surprise inefficient query.. what was an index lookup is now a full table scan.

If we can't pick out just the known cases, like find(hugenum), I think we have to just let the exception raise.

@kamipo

This comment has been minimized.

Show comment
Hide comment
@kamipo

kamipo Jul 31, 2017

Member

I changed the implementation to build "1=0" predicate rather than replacing binds with inline values. I think it is no longer given inefficient query.

Member

kamipo commented Jul 31, 2017

I changed the implementation to build "1=0" predicate rather than replacing binds with inline values. I think it is no longer given inefficient query.

@kamipo kamipo added the activerecord label Aug 20, 2017

@rafaelfranca

This comment has been minimized.

Show comment
Hide comment
@rafaelfranca

rafaelfranca Jan 17, 2018

Member

I'm thing this new implementation is good. @matthewd do you still have concerns?

Member

rafaelfranca commented Jan 17, 2018

I'm thing this new implementation is good. @matthewd do you still have concerns?

Show outdated Hide outdated activerecord/lib/active_record/relation/predicate_builder.rb
@@ -55,6 +55,13 @@ def build_bind_attribute(column_name, value)
Arel::Nodes::BindParam.new(attr)
end
def build_query_attribute(column_name, value)
bind = build_bind_attribute(column_name, value)
bind.nil?

This comment has been minimized.

@sgrif

sgrif Jan 17, 2018

Member

Why do we have the random nil? check?

@sgrif

sgrif Jan 17, 2018

Member

Why do we have the random nil? check?

This comment has been minimized.

@kamipo

kamipo Jan 17, 2018

Member

This intend to call QueryAttribute#value_for_database in QueryAttribute#nil? in BindParam to check Type::Integer#ensure_in_range before building arel predicate.

def nil?
!value_before_type_cast.is_a?(StatementCache::Substitute) &&
(value_before_type_cast.nil? || value_for_database.nil?)
end

@kamipo

kamipo Jan 17, 2018

Member

This intend to call QueryAttribute#value_for_database in QueryAttribute#nil? in BindParam to check Type::Integer#ensure_in_range before building arel predicate.

def nil?
!value_before_type_cast.is_a?(StatementCache::Substitute) &&
(value_before_type_cast.nil? || value_for_database.nil?)
end

This comment has been minimized.

@sgrif

sgrif Jan 17, 2018

Member

See my second comment. I'd like to see this made more explicit in the code. This is too spooky-action-at-a-distance for my taste.

@sgrif

sgrif Jan 17, 2018

Member

See my second comment. I'd like to see this made more explicit in the code. This is too spooky-action-at-a-distance for my taste.

This comment has been minimized.

@sgrif

sgrif Jan 17, 2018

Member

Especially since nil? there is basically just a hack to satisfy Arel that I'd like to remove at some point.

@sgrif

sgrif Jan 17, 2018

Member

Especially since nil? there is basically just a hack to satisfy Arel that I'd like to remove at some point.

This comment has been minimized.

@kamipo

kamipo Jan 17, 2018

Member

hm... how about here?

diff --git a/activerecord/lib/active_record/relation/predicate_builder.rb b/activerecord/lib/active_record/relation/predicate_builder.rb
index c69a238eff..56e632f0d7 100644
--- a/activerecord/lib/active_record/relation/predicate_builder.rb
+++ b/activerecord/lib/active_record/relation/predicate_builder.rb
@@ -58,7 +58,13 @@ def build_bind_attribute(column_name, value)
 
     def build_query_attribute(column_name, value)
       bind = build_bind_attribute(column_name, value)
-      bind.nil?
+      attr = bind.value
+
+      # Checking whether an integer value is within the range
+      if attr.type.type == :integer && attr.value_before_type_cast
+        attr.value_for_database
+      end
+
       bind
     rescue ::RangeError
     end
@kamipo

kamipo Jan 17, 2018

Member

hm... how about here?

diff --git a/activerecord/lib/active_record/relation/predicate_builder.rb b/activerecord/lib/active_record/relation/predicate_builder.rb
index c69a238eff..56e632f0d7 100644
--- a/activerecord/lib/active_record/relation/predicate_builder.rb
+++ b/activerecord/lib/active_record/relation/predicate_builder.rb
@@ -58,7 +58,13 @@ def build_bind_attribute(column_name, value)
 
     def build_query_attribute(column_name, value)
       bind = build_bind_attribute(column_name, value)
-      bind.nil?
+      attr = bind.value
+
+      # Checking whether an integer value is within the range
+      if attr.type.type == :integer && attr.value_before_type_cast
+        attr.value_for_database
+      end
+
       bind
     rescue ::RangeError
     end

This comment has been minimized.

@matthewd

matthewd Jan 18, 2018

Member

That's slightly more obvious about what it's doing, but still feels both over-specialised and too vague at the same time. My first thought was maybe we want a bind.value.impossible? or similar, but that isn't actually sufficient for our needs: see the new tests I proposed.

I'm not excited about this method name, either... particularly the fact we're losing "bind", which seems important to understand while reading callers. But that's better revisited after the changes for the new tests too.

@matthewd

matthewd Jan 18, 2018

Member

That's slightly more obvious about what it's doing, but still feels both over-specialised and too vague at the same time. My first thought was maybe we want a bind.value.impossible? or similar, but that isn't actually sufficient for our needs: see the new tests I proposed.

I'm not excited about this method name, either... particularly the fact we're losing "bind", which seems important to understand while reading callers. But that's better revisited after the changes for the new tests too.

@sgrif

sgrif approved these changes Jan 17, 2018

build_query_attribute appears to be doing something really implicit. I'd like to see it be made much more explicit, but this seems fine to me overall. The additional test cases make sense.

I would like to see the intent of build_query_attribute made more clear before this is merged though

@matthewd

This still opens the possibility of 2**N query cache entries for a single query with N integer parameters, but that's far more bounded than infinity. And it avoids wasting database effort on the known-false condition, so that's good.

I agree the current rescues are catching too much, so something must change. If this can fix it while avoiding even more exceptions, so much the better.

@@ -215,6 +215,13 @@ def test_exists_with_order
assert_equal true, Topic.order("invalid sql here").exists?
end
def test_exists_with_large_number
assert_equal true, Topic.where(id: [1, 9223372036854775808]).exists?
assert_equal true, Topic.where(id: 1..9223372036854775808).exists?

This comment has been minimized.

@matthewd

matthewd Jan 18, 2018

Member

Some more:

assert_equal false, Topic.where(id: 9223372036854775808..9223372036854775809).exists?
assert_equal true, Topic.where(id: -9223372036854775809..9223372036854775808).exists?

(I'm pretty sure these would both currently be true?)

@matthewd

matthewd Jan 18, 2018

Member

Some more:

assert_equal false, Topic.where(id: 9223372036854775808..9223372036854775809).exists?
assert_equal true, Topic.where(id: -9223372036854775809..9223372036854775808).exists?

(I'm pretty sure these would both currently be true?)

Show outdated Hide outdated ...ord/lib/active_record/relation/predicate_builder/basic_object_handler.rb
bind = predicate_builder.build_bind_attribute(attribute.name, value)
attribute.eq(bind)
bind = predicate_builder.build_query_attribute(attribute.name, value)
bind ? attribute.eq(bind) : "1=0"

This comment has been minimized.

@matthewd

matthewd Jan 18, 2018

Member

I think attribute.in([]) would be more consistent with the range handler, as a more abstract false.

@matthewd

matthewd Jan 18, 2018

Member

I think attribute.in([]) would be more consistent with the range handler, as a more abstract false.

This comment has been minimized.

@kamipo

kamipo Jan 18, 2018

Member

Yeah, changed to attribute.in([]).

@kamipo

kamipo Jan 18, 2018

Member

Yeah, changed to attribute.in([]).

Show outdated Hide outdated ...erecord/lib/active_record/relation/predicate_builder/relation_handler.rb
@@ -5,7 +5,7 @@ class PredicateBuilder
class RelationHandler # :nodoc:
def call(attribute, value)
if value.select_values.empty?
value = value.select(value.arel_attribute(value.klass.primary_key))
value = value.select(value.klass.primary_key)

This comment has been minimized.

@matthewd

matthewd Jan 18, 2018

Member

This doesn't seem related?

@matthewd

matthewd Jan 18, 2018

Member

This doesn't seem related?

This comment has been minimized.

@kamipo

kamipo Jan 18, 2018

Member

Right, unrelated. I removed the change.

@kamipo

kamipo Jan 18, 2018

Member

Right, unrelated. I removed the change.

Show outdated Hide outdated activerecord/lib/active_record/relation/predicate_builder.rb
@@ -55,6 +55,13 @@ def build_bind_attribute(column_name, value)
Arel::Nodes::BindParam.new(attr)
end
def build_query_attribute(column_name, value)
bind = build_bind_attribute(column_name, value)
bind.nil?

This comment has been minimized.

@matthewd

matthewd Jan 18, 2018

Member

That's slightly more obvious about what it's doing, but still feels both over-specialised and too vague at the same time. My first thought was maybe we want a bind.value.impossible? or similar, but that isn't actually sufficient for our needs: see the new tests I proposed.

I'm not excited about this method name, either... particularly the fact we're losing "bind", which seems important to understand while reading callers. But that's better revisited after the changes for the new tests too.

@matthewd

matthewd Jan 18, 2018

Member

That's slightly more obvious about what it's doing, but still feels both over-specialised and too vague at the same time. My first thought was maybe we want a bind.value.impossible? or similar, but that isn't actually sufficient for our needs: see the new tests I proposed.

I'm not excited about this method name, either... particularly the fact we're losing "bind", which seems important to understand while reading callers. But that's better revisited after the changes for the new tests too.

assert_equal true, Topic.where(id: 1..9223372036854775808).exists?
assert_equal true, Topic.where(id: -9223372036854775809..9223372036854775808).exists?
assert_equal false, Topic.where(id: 9223372036854775808..9223372036854775809).exists?
assert_equal false, Topic.where(id: -9223372036854775810..-9223372036854775809).exists?

This comment has been minimized.

@kamipo

kamipo Jan 18, 2018

Member

(I'm pretty sure these would both currently be true?)

Good point. I've fixed and added the tests.

@kamipo

kamipo Jan 18, 2018

Member

(I'm pretty sure these would both currently be true?)

Good point. I've fixed and added the tests.

Show outdated Hide outdated activerecord/lib/active_record/relation/predicate_builder.rb
# Checking whether an integer value is within the range
if attr.type.type == :integer && attr.value_before_type_cast
attr.value_for_database

This comment has been minimized.

@kamipo

kamipo Jan 18, 2018

Member

I've changed it from bind.nil? for now.

@kamipo

kamipo Jan 18, 2018

Member

I've changed it from bind.nil? for now.

Show outdated Hide outdated activerecord/lib/active_record/relation/predicate_builder/range_handler.rb
private
def build_bind_with_value(attribute, value)
bind = predicate_builder.build_query_attribute(attribute.name, value)
value = Float::INFINITY unless bind

This comment has been minimized.

@matthewd

matthewd Jan 18, 2018

Member

I think converting all out-of-range values to infinity is still problematic. Try 9223372036854775808..10.

I'm also worried that this class really shouldn't be doing any integer/integer-like operations. Just because something is infinite doesn't mean it's comparable with zero.

How about we push the other direction, and instead take out the concept of infinity? The question this code needs to ask is whether the value is above- or below range: Infinity is both of those (because we give it special treatment), but over-sized integer values are one or the other.

After thinking through a couple of possible APIs, the least bad one I can come up with is to add an impossible_database_value? method to AMo::Attribute, with documented return values of false (value is normal), true (value is impossible, no more information available), :above, and :below (value is greater/less than all possible values). Then values where value_for_database will raise RangeError should return one of the truthy values from that method.

@sgrif this is extending your nice simple low level interface in way I'm not particularly proud of; opinions / better ideas very welcome.

@matthewd

matthewd Jan 18, 2018

Member

I think converting all out-of-range values to infinity is still problematic. Try 9223372036854775808..10.

I'm also worried that this class really shouldn't be doing any integer/integer-like operations. Just because something is infinite doesn't mean it's comparable with zero.

How about we push the other direction, and instead take out the concept of infinity? The question this code needs to ask is whether the value is above- or below range: Infinity is both of those (because we give it special treatment), but over-sized integer values are one or the other.

After thinking through a couple of possible APIs, the least bad one I can come up with is to add an impossible_database_value? method to AMo::Attribute, with documented return values of false (value is normal), true (value is impossible, no more information available), :above, and :below (value is greater/less than all possible values). Then values where value_for_database will raise RangeError should return one of the truthy values from that method.

@sgrif this is extending your nice simple low level interface in way I'm not particularly proud of; opinions / better ideas very welcome.

This comment has been minimized.

@kamipo

kamipo Jan 18, 2018

Member

I think converting all out-of-range values to infinity is still problematic. Try 9223372036854775808..10.

ah... I've addressed it in e80e677 anyway.

Originally this PR was written as not to add any APIs as much as possible (the converting to infinity, and bind.nil?).
But I agree that adding an API for this issue will make it easier to understand what problems are addressed.
So I'll try another direction to make more explicit in the code.

@kamipo

kamipo Jan 18, 2018

Member

I think converting all out-of-range values to infinity is still problematic. Try 9223372036854775808..10.

ah... I've addressed it in e80e677 anyway.

Originally this PR was written as not to add any APIs as much as possible (the converting to infinity, and bind.nil?).
But I agree that adding an API for this issue will make it easier to understand what problems are addressed.
So I'll try another direction to make more explicit in the code.

This comment has been minimized.

@matthewd

matthewd Jan 18, 2018

Member

I think that condition would now break on Float::INFINITY..3. That's definitely an empty range, but we've historically (I believe deliberately) treated it as open-ended -- pretending it is -Float::INFINITY..3 instead.

This one isn't even related to the limited-integer stuff, so I'm surprised there isn't already a test for it 😕

Another value we should be aware of is Date::Infinity. It's a very limited object, and officially only exists for historical Marshal data... but I've seen it used in this context in the wild, in a belief that it sounds more correct. If I'm right that it currently works, we should be aware if a change now breaks it. (Notable because while it can be compared to zero, it cannot be compared to a Date [which is likely to be at the other end of the range].)

Anyway, yeah, see what you come up with while changing internal APIs more freely.

@matthewd

matthewd Jan 18, 2018

Member

I think that condition would now break on Float::INFINITY..3. That's definitely an empty range, but we've historically (I believe deliberately) treated it as open-ended -- pretending it is -Float::INFINITY..3 instead.

This one isn't even related to the limited-integer stuff, so I'm surprised there isn't already a test for it 😕

Another value we should be aware of is Date::Infinity. It's a very limited object, and officially only exists for historical Marshal data... but I've seen it used in this context in the wild, in a belief that it sounds more correct. If I'm right that it currently works, we should be aware if a change now breaks it. (Notable because while it can be compared to zero, it cannot be compared to a Date [which is likely to be at the other end of the range].)

Anyway, yeah, see what you come up with while changing internal APIs more freely.

@rafaelfranca

This comment has been minimized.

Show comment
Hide comment
@rafaelfranca

rafaelfranca Jan 18, 2018

Member

What Topic.where(id: [1, 9223372036854775808]).to_a would do? Raise an exception or return an empty array? I feel if it return an empty array it will be really hard to track down that your query is not returning any object because one or more of the ids you passed are too big.

Member

rafaelfranca commented Jan 18, 2018

What Topic.where(id: [1, 9223372036854775808]).to_a would do? Raise an exception or return an empty array? I feel if it return an empty array it will be really hard to track down that your query is not returning any object because one or more of the ids you passed are too big.

@@ -359,6 +359,16 @@ def permit!
assert_equal author, Author.where(params.permit!).first
end
def test_where_with_large_number
assert_equal [authors(:bob)], Author.where(id: [3, 9223372036854775808])

This comment has been minimized.

@kamipo

kamipo Jan 18, 2018

Member

What Topic.where(id: [1, 9223372036854775808]).to_a would do?

Before: Raise RangeError.

After: return [an object(id: 1)].

@kamipo

kamipo Jan 18, 2018

Member

What Topic.where(id: [1, 9223372036854775808]).to_a would do?

Before: Raise RangeError.

After: return [an object(id: 1)].

@rafaelfranca

Sounds good to me, but let's wait @matthewd's approval

Show outdated Hide outdated activerecord/test/cases/finder_test.rb
@@ -812,6 +832,10 @@ def test_find_on_hash_conditions_with_array_of_ranges
assert_equal [1, 2, 6, 7, 8], Comment.where(id: [1..2, 6..8]).to_a.map(&:id).sort
end
def test_find_on_hash_conditions_with_open_ended_range
assert_equal [1, 2, 3], Comment.where(id: Float::INFINITY..3).to_a.map(&:id).sort

This comment has been minimized.

@kamipo

kamipo Jan 18, 2018

Member

Added test case for open-ended range.

@kamipo

kamipo Jan 18, 2018

Member

Added test case for open-ended range.

Show outdated Hide outdated activerecord/lib/active_record/relation/predicate_builder/range_handler.rb
end_bind = predicate_builder.build_boundable_attribute(attribute.name, value.end)
if infinity?(value.begin) || begin_bind.nil?
if infinity?(value.end) || end_bind.nil?
if begin_bind.nil? && value.begin > 0 || end_bind.nil? && value.end < 0

This comment has been minimized.

@kamipo

kamipo Jan 18, 2018

Member

I've removed converting out-of-range values to infinity.
So this line will check out-of-range values only, doesn't affect open-ended ranges.

@kamipo

kamipo Jan 18, 2018

Member

I've removed converting out-of-range values to infinity.
So this line will check out-of-range values only, doesn't affect open-ended ranges.

Show outdated Hide outdated activerecord/lib/active_record/relation/predicate_builder/range_handler.rb
if value.begin.respond_to?(:infinite?) && value.begin.infinite?
if value.end.respond_to?(:infinite?) && value.end.infinite?
attribute.not_in([])
if !infinity?(value.begin) && !infinity?(value.end) && value.begin > value.end

This comment has been minimized.

@matthewd

matthewd Jan 24, 2018

Member

This really feels like it's taking more effort to work around the problem than giving the attribute responsibility for ranges.

My examples are getting more obscure, and the gaps are getting smaller... but the fact there are any gaps highlights that it feels like a piecemeal solution, with lots of conditionals, instead of few wide-reaching rules.

x = Author.create! name: "12 Monkeys"
assert_equal [x], Author.where(name: 10..2)

Maybe that's silly enough that we don't care.. but it is a change in behaviour.

(Or maybe I'm misreading and the value's already been cast before this? But I think that only happens below.)

I'll defer to @sgrif, but I still feel like we want something like #30000 (comment) instead of having this class expect more from all the value types it might encounter.

@matthewd

matthewd Jan 24, 2018

Member

This really feels like it's taking more effort to work around the problem than giving the attribute responsibility for ranges.

My examples are getting more obscure, and the gaps are getting smaller... but the fact there are any gaps highlights that it feels like a piecemeal solution, with lots of conditionals, instead of few wide-reaching rules.

x = Author.create! name: "12 Monkeys"
assert_equal [x], Author.where(name: 10..2)

Maybe that's silly enough that we don't care.. but it is a change in behaviour.

(Or maybe I'm misreading and the value's already been cast before this? But I think that only happens below.)

I'll defer to @sgrif, but I still feel like we want something like #30000 (comment) instead of having this class expect more from all the value types it might encounter.

Show outdated Hide outdated activerecord/lib/active_record/relation/predicate_builder/range_handler.rb
if impossible?(begin_bind) && value.begin > 0 || impossible?(end_bind) && value.end < 0
attribute.in([])
elsif infinity?(value.begin) || !boundable?(begin_bind)

This comment has been minimized.

@kamipo

kamipo Jan 24, 2018

Member

Thanks for the feedback. I've added boundable? and impossible? helper methods for the range handler to make more explicit in the code.

@kamipo

kamipo Jan 24, 2018

Member

Thanks for the feedback. I've added boundable? and impossible? helper methods for the range handler to make more explicit in the code.

Show outdated Hide outdated activerecord/test/cases/finder_test.rb
def test_find_on_hash_conditions_with_numeric_range_for_string
topic = Topic.create!(title: "12 Factor App")
assert_equal [topic], Topic.where(title: 10..2).to_a

This comment has been minimized.

@kamipo

kamipo Jan 24, 2018

Member

Added test case for numeric range for string column.

@kamipo

kamipo Jan 24, 2018

Member

Added test case for numeric range for string column.

@kamipo

This comment has been minimized.

Show comment
Hide comment
@kamipo

kamipo Jan 30, 2018

Member

In this PR, finder methods no longer raise RangeError.
So StatementCache#execute is the only place to raise the exception for finder queries.

#29988 closely related with this PR to avoid catching the exception in much places.
Thus I cherry-picked #29988 in this PR.

Member

kamipo commented Jan 30, 2018

In this PR, finder methods no longer raise RangeError.
So StatementCache#execute is the only place to raise the exception for finder queries.

#29988 closely related with this PR to avoid catching the exception in much places.
Thus I cherry-picked #29988 in this PR.

Show outdated Hide outdated activerecord/lib/active_record/statement_cache.rb
@@ -106,6 +106,8 @@ def execute(params, connection, &block)
sql = query_builder.sql_for bind_values, connection
klass.find_by_sql(sql, bind_values, preparable: true, &block)
rescue ::RangeError
[]

This comment has been minimized.

@matthewd

matthewd Jan 30, 2018

Member

This rescue makes me nervous, just because it sounds far-reaching.

I agree it's fine to put it here in practice, because of the very limited set of statements that are eligible for this cache, as you noted in the commit message. I think that's worth a comment here too, though: both to hopefully draw attention if the cache ever gets used for other queries, and just to reassure future readers me next time I read through this code 😅

@matthewd

matthewd Jan 30, 2018

Member

This rescue makes me nervous, just because it sounds far-reaching.

I agree it's fine to put it here in practice, because of the very limited set of statements that are eligible for this cache, as you noted in the commit message. I think that's worth a comment here too, though: both to hopefully draw attention if the cache ever gets used for other queries, and just to reassure future readers me next time I read through this code 😅

# since it is not executable (raise a range error exception).
#
# Note that empty result is not always correct unless cached
# queries are simple finder queries.

This comment has been minimized.

@kamipo

kamipo Feb 2, 2018

Member

Added the comment for returning empty result.
@matthewd Is it enough for you?

@kamipo

kamipo Feb 2, 2018

Member

Added the comment for returning empty result.
@matthewd Is it enough for you?

Show outdated Hide outdated activerecord/lib/active_record/relation/predicate_builder/range_handler.rb
if impossible?(begin_bind) && value.begin > 0 || impossible?(end_bind) && value.end < 0
attribute.in([])
elsif infinity?(value.begin) || !begin_bind.value.boundable?

This comment has been minimized.

@kamipo

kamipo Feb 2, 2018

Member

TBH I'd like to postpone implementing like impossible_database_value? API (#30000 (comment)) in this PR.

The API is only useful for building > or < with boundable queries (like #31219), so range predicate handler is the only place where it will be used for now.

But the range predicate handler need to treat infinity value as open-ended regardless of whether + or - value. Even if impossible_database_value? is implemented, it does not make the code easy.

@matthewd @sgrif What do you think about that?

diff --git a/activerecord/lib/active_record/relation/predicate_builder/range_handler.rb b/activerecord/lib/active_record/relation/predicate_builder/range_handler.rb
index 7ea9b46ab6..4497d9e296 100644
--- a/activerecord/lib/active_record/relation/predicate_builder/range_handler.rb
+++ b/activerecord/lib/active_record/relation/predicate_builder/range_handler.rb
@@ -17,7 +17,7 @@ def call(attribute, value)
         begin_bind = predicate_builder.build_bind_attribute(attribute.name, value.begin)
         end_bind = predicate_builder.build_bind_attribute(attribute.name, value.end)
 
-        if impossible?(begin_bind) && value.begin > 0 || impossible?(end_bind) && value.end < 0
+        if impossible?(begin_bind) && begin_bind.value.impossible_database_value? == :above || impossible?(end_bind) && end_bind.value.impossible_database_value? == :below
           attribute.in([])
         elsif infinity?(value.begin) || !begin_bind.value.boundable?
           if infinity?(value.end) || !end_bind.value.boundable?
diff --git a/activerecord/lib/active_record/relation/query_attribute.rb b/activerecord/lib/active_record/relation/query_attribute.rb
index 6b96cf6745..f63846815b 100644
--- a/activerecord/lib/active_record/relation/query_attribute.rb
+++ b/activerecord/lib/active_record/relation/query_attribute.rb
@@ -29,6 +29,16 @@ def boundable?
       rescue ::RangeError
         @_boundable = false
       end
+
+      def impossible_database_value?
+        if boundable?
+          false
+        elsif value_before_type_cast > 0
+          :above
+        else
+          :below
+        end
+      end
     end
   end
 end
@kamipo

kamipo Feb 2, 2018

Member

TBH I'd like to postpone implementing like impossible_database_value? API (#30000 (comment)) in this PR.

The API is only useful for building > or < with boundable queries (like #31219), so range predicate handler is the only place where it will be used for now.

But the range predicate handler need to treat infinity value as open-ended regardless of whether + or - value. Even if impossible_database_value? is implemented, it does not make the code easy.

@matthewd @sgrif What do you think about that?

diff --git a/activerecord/lib/active_record/relation/predicate_builder/range_handler.rb b/activerecord/lib/active_record/relation/predicate_builder/range_handler.rb
index 7ea9b46ab6..4497d9e296 100644
--- a/activerecord/lib/active_record/relation/predicate_builder/range_handler.rb
+++ b/activerecord/lib/active_record/relation/predicate_builder/range_handler.rb
@@ -17,7 +17,7 @@ def call(attribute, value)
         begin_bind = predicate_builder.build_bind_attribute(attribute.name, value.begin)
         end_bind = predicate_builder.build_bind_attribute(attribute.name, value.end)
 
-        if impossible?(begin_bind) && value.begin > 0 || impossible?(end_bind) && value.end < 0
+        if impossible?(begin_bind) && begin_bind.value.impossible_database_value? == :above || impossible?(end_bind) && end_bind.value.impossible_database_value? == :below
           attribute.in([])
         elsif infinity?(value.begin) || !begin_bind.value.boundable?
           if infinity?(value.end) || !end_bind.value.boundable?
diff --git a/activerecord/lib/active_record/relation/query_attribute.rb b/activerecord/lib/active_record/relation/query_attribute.rb
index 6b96cf6745..f63846815b 100644
--- a/activerecord/lib/active_record/relation/query_attribute.rb
+++ b/activerecord/lib/active_record/relation/query_attribute.rb
@@ -29,6 +29,16 @@ def boundable?
       rescue ::RangeError
         @_boundable = false
       end
+
+      def impossible_database_value?
+        if boundable?
+          false
+        elsif value_before_type_cast > 0
+          :above
+        else
+          :below
+        end
+      end
     end
   end
 end

kamipo added some commits Jul 21, 2017

Ensure `StatementCache#execute` never raises `RangeError`
Since bc805d0, finder methods no longer raise `RangeError`. So
`StatementCache#execute` is the only place to raise the exception for
finder queries.

`StatementCache` is used for simple equality queries. This means that if
`StatementCache#execute` raises `RangeError`, the result is always
empty. So `StatementCache#execute` simply return empty result in that
case, and we can avoid catching the exception in much places.
All of queries should return correct result even if including large n…
…umber

Currently several queries cannot return correct result due to incorrect
`RangeError` handling.

First example:

```ruby
assert_equal true, Topic.where(id: [1, 9223372036854775808]).exists?
assert_equal true, Topic.where.not(id: 9223372036854775808).exists?
```

The first example is obviously to be true, but currently it returns
false.

Second example:

```ruby
assert_equal topics(:first), Topic.where(id: 1..9223372036854775808).find(1)
```

The second example also should return the object, but currently it
raises `RecordNotFound`.

It can be seen from the examples, the queries including large number
assuming empty result is not always correct.

Therefore, This change handles `RangeError` to generate executable SQL
instead of raising `RangeError` to users to always return correct
result. By this change, it is no longer raised `RangeError` to users.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment