st.integers() never generates examples close to the max_value

[ckp95]

Suppose I have some function that takes numbers and has a large, but finite max value for which it is valid. Numbers higher than this value cause it to throw an exception. Numbers less than or equal to to this value are valid, but the function might exhibit weird/pathological behavior near to the boundary, which would require careful testing (e.g. numerical instability, or a discontinuity at the boundary, or something). My expectation was that if I use given(integers(max_value=upper_bound)), then hypothesis would duly generate at least some numbers near that upper bound to see what happens there. But that isn't true:

lower_bound = 15
upper_bound = 123456789

@given(foo=integers(min_value=lower_bound, max_value=upper_bound))
@settings(max_examples=5000)
def test_foo(foo):
    if abs(foo - upper_bound) < 10:
        assert False, "hit near to upper bound"
    
    if abs(foo - lower_bound) < 10:
        print("hit near to lower bound")
        
    assert True

This prints out lots of "hit near to lower bound", but it never triggers the "hit near to upper bound" exception. If there is some implementation failure near the upper bound, hypothesis would not find it. Is there a way to make it give a bit of extra attention to the upper boundary? For a while I assumed that hypothesis did this already and was surprised that it didn't.

I am using hypothesis version 6.10.0.

[Zac-HD]

This is... harder than it sounds, at least while maintaining performance, though it would probably be useful too.

FWIW you're probably seeing only exactly the lower bound, and not anything close to it - we generate minimal parts much more often than chance, but not near-minimal. (a heuristic which helps us to generate a variety of 'tree depths' without exceeding sensible 'tree sizes')

[Zac-HD]

Related to #2671 and #2907.

[rodrigogiraoserrao]

There is something I am missing. For things like integers, why isn't the upper bound itself tested for explicitly?

Changing the example above, it is fairly easy to see that the upper bound isn't picked as a special case:

from hypothesis import strategies as st, given, settings


lower_bound = 0
upper_bound = 100

@given(foo=st.integers(min_value=lower_bound, max_value=upper_bound))
@settings(max_examples=10)
def test_foo(foo):
    if foo == upper_bound:
        assert False, "hit the upper bound"

[Zac-HD]

Neither bound is special - only zero, as the minimal example! (or the bound closest to zero, if zero is disallowed).

The solution would be to expand IntegerStrategy.do_draw() to do something fancier in the bounded case - e.g. "if the bounds are not together, pick a number in [0, 127], and map the high options to (min, min, max, max, min+1, max-1); otherwise generate as usual". See FloatStrategy.do_draw() for an example of this style (it's tricky!); note that we don't want to push that down further where it would distort e.g. characters().

[rodrigogiraoserrao]

Neither bound is special

If there are bounds, they are very likely to be special! I would even argue that zero is quite uninteresting in a case that is bounded between -5 and 10, for example.

[Zac-HD]

Oops, unclear phrasing! I agree that we should treat bounds (and off-by-one!) specially; I meant that we currently don't do that.