Supports... Protocol strategies bugs & suggested improvements.

[leaprovenzano]

Supports... Protocol strategies bugs & suggested improvements.

---

hypothesis: 4.56.1
python : 3.7.4

---

The following are suggested improvements for the typing.Supports... protocol types. The general expectation is that if i do from_type on a one of these protocol types i will get at minimum all strategies which generate values who's types support the protocol. At the moment many of them are looking pretty bare and so wouldn't be generating the range of values a user would expect.

For each protocol below I've provided the underlying current strategy drawn by from_type and a suggested strategy.
I gathered the suggested strategies with a little help from hypothesis.find and have also tried to optimize the order of the statigies with a little stupid heruristic funtion to support good shrinking. I then did some basic tests with the example function to make sure they hold.
There are also a couple which contain bugs. I will try and clarify where that is the case in each example:

---

typing.SupportsInt

current strategy:

floats()

example:

import typing
from hypothesis.strategies import *
from hypothesis import given


@given(from_type(typing.SupportsInt))
def test_parse_int(inp):
    int(inp)

BUGS: The above currently fails it's not protected from infinites and nans.

suggested strategy:

supportsint = (booleans() |
               integers() |
               floats(allow_infinity=False, allow_nan=False) |
               uuids() |
               decimals(allow_infinity=False, allow_nan=False) 
               )

really really nice to have:

another nice touch would be to add a stringified version of int particularly since casting from string is such a common use case

supportsint |= integers().map(str)

---

typing.SupportsAbs

current strategy:

complex_numbers()

example:

import typing
from hypothesis.strategies import *
from hypothesis import given


@given(from_type(typing.SupportsAbs))
def test_abs(inp):
    abs(inp)

BUGS:

we encounter an overflow error on on an huge complex number

inp = (1.2711610061536462e+308+1.2711610061536464e+308j)

    @given(from_type(typing.SupportsAbs))
    def test_parse_int(inp):
>       abs(inp)
E       OverflowError: absolute value too large

suggested strategy:

we need to limit the mangitude on complex numbers to avoid overflow. The threshold below is just a guess...

# this is just a guess...
MAX_MAGNITUDE_FOR_ABS = 308

supportsabs = (
    booleans() |
    integers() |
    floats(allow_nan=False, allow_infinity=False) |
    complex_numbers(max_magnitude=MAX_MAGNITUDE_FOR_ABS) |
    fractions() |
    decimals() |
    timedeltas()
)

nice to have:

support for numpy.extra where applicable since most numeric arrays support abs.

---

typing.SupportsRound

current strategy:

complex_numbers()

example:

import typing
from hypothesis.strategies import *
from hypothesis import given


@given(from_type(typing.SupportsRound))
def test_round(inp):
    round(inp)

BUGS:

We quickly get a TypeError since complex numbers don't define __round__ method !!!

suggested strategy:

supports_round = (
    booleans() |
    integers() |
    floats(allow_nan=False, allow_infinity=False) |
    decimals(allow_nan=False, allow_infinity=False) |
    fractions()
)

---

typing.SupportsFloat

current strategy:

floats()

example:

import typing
from hypothesis.strategies import *
from hypothesis import given


@given(from_type(typing.SupportsFloat))
def test_cast_to_float(inp):
    float(inp)

The current strategy doesn't have any overt bugs it's just too permissive.

suggested strategy:

supports_float = (
    booleans() |
    floats() |
    integers() |
    decimals(allow_nan=False) |
    fractions()
)

really really nice to have:

as with SupportsInt would be really nice to include stringified floats in the strategy since it's a common use case.

supportsint |= floats().map(str)

---

I haven't covered SupportsBytes and SupportsComplex ( sorry i got a bit knackered of writing..) but they also look too narrow. So maybe worth having a look into them as well.

---

I'll file another issue for improvements to abc type protocols when in a bit. If this format is overwhelming let me know and i'll do issue per strategy instead of grouping together.

[leaprovenzano]

also most of these protocols are direct mappings in the _global_type_lookup and so would not involve too much faffing in types .

I'd be happy to open a PR for the changes if maintainers don't see any problems with these suggestions.

[Zac-HD]

Ooh, thanks for looking into this! Short on time now, but I'll share more thoughts tomorrow about what would be helpful in a PR 😁

[Zac-HD]

• I would be very happy to have a single PR with these changes 🎉
• I'd actually prefer to keep generating values which can overflow (etc) - the SupportsOp classes indicate that the type of some value supports an operation, not that the operation will succeed on that specific value.
• integers().map(str) seems fairer than text(), but from_regex(r"-?\d+", fullmatch=True).filter(can_cast_to_int) could be even more fun...
• Note that for collection types, I don't want to infer a strategy for nested collections - while they would be valid and perhaps find bugs, it's also a significant performance drain.

[leaprovenzano]

• I would be very happy to have a single PR with these changes 🎉

cool i will have a go at it then!

• I'd actually prefer to keep generating values which can overflow (etc) - the SupportsOp classes indicate that the type of some value supports an operation, not that the operation will succeed on that specific value.

yeah this makes sense and i don't suppose it's much work for users to filter downstream if this is what they're after... It's just me being lazy and (and possibly wanting SupportOps to be way more powerful than they are/should be).

• integers().map(str) seems fairer than text(), but from_regex(r"-?\d+", fullmatch=True).filter(can_cast_to_int) could be even more fun...

yeah i was looking at the regex for floats but really that requires lookaheads to do properly and is more complex. int should be simpler... something like :

r'^-?([1-9]\d*)|0$'

and we could avoid filter altogether.

• Note that for collection types, I don't want to infer a strategy for nested collections - while they would be valid and perhaps find bugs, it's also a significant performance drain.

As to collection stuff I will make another issue so we can discuss if that's cool... for SupportsOp stuff it's pretty much just numeric so we shouldn't be an problem.

[leaprovenzano]

thinking about it another option might be to make a general supports strategy that accepts an abc and returns a union of strategies whos value types implement all of its abstractmethods? ... just a thought...

longterm could extend to stuff like error filtering (ie for overflow error) at users discretion....
signature checking on the abstract methods might even be useful? again... just a potential avenue.

[Zac-HD]

• I'd stick with the filtering int regex, because the first digit need not be ASCII 😉
• for floats, we could generate a float and a format spec to apply - there are many!
• strategies inferred from types aren't that fancy - if you want control of overflow, proper signature checks, etc, I think that belongs in your downstream project. If it's widely useful downstream, we're often happy to have it added upstream too!

[leaprovenzano]

• I'd stick with the filtering int regex, because the first digit need not be ASCII 😉
• for floats, we could generate a float and a format spec to apply - there are many!
• strategies inferred from types aren't that fancy - if you want control of overflow, proper signature checks, etc, I think that belongs in your downstream project. If it's widely useful downstream, we're often happy to have it added upstream too!

fair nuff I may have gotten a bit carried away! I'll do a pr patching to patch the updates into _global_type_lookup in a bit...

[Zac-HD]

Closed by #2297 🚀