Support for hypothesis

[apirogov]

I currently wanted to add hypothesis to my tests and I use phantom types in my pydantic models. Hypothesis has built-in support for pydantic, but it turns out it does not work with phantom types.

from phantom.interval import Open
from hypothesis import given, strategies as st
from pydantic import BaseModel


class Int0To10(int, Open, low=0, high=10):
    ...

# st.register_type_strategy(Int0To10, st.integers(0, 10))

class MySchema(BaseModel):
    val: Int0To10

@given(st.builds(MySchema))
def test_schema(instance):
    print(instance.json(indent=2))

test_schema()

results in

Traceback (most recent call last):
  File "/local/home/a.pirogov/.cache/pypoetry/virtualenvs/metador-core-AXME58BY-py3.8/lib/python3.8/site-packages/hypothesis/strategies/_internal/core.py", line 836, in do_draw
    return self.target(
TypeError: __call__() missing 1 required positional argument: 'instance'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "scratch_hypo.py", line 24, in <module>
    test_schema()
  File "scratch_hypo.py", line 20, in test_schema
    def test_schema(instance):
  File "/local/home/a.pirogov/.cache/pypoetry/virtualenvs/metador-core-AXME58BY-py3.8/lib/python3.8/site-packages/hypothesis/core.py", line 1325, in wrapped_test
    raise the_error_hypothesis_found
  File "/local/home/a.pirogov/.cache/pypoetry/virtualenvs/metador-core-AXME58BY-py3.8/lib/python3.8/site-packages/hypothesis/strategies/_internal/core.py", line 855, in do_draw
    raise InvalidArgument(
hypothesis.errors.InvalidArgument: Calling <class '__main__.Int0To10'> with no arguments raised an error - try using from_type(<class '__main__.Int0To10'>) instead of builds(<class '__main__.Int0To10'>)

but works when registering the type strategy (line in example is uncommented)

Would it be possible to register the corresponding generating strategies for all built-in types and provide a hook like __hypothesis_strategy__ where an automatic strategy is not feasible / to override the default one?

I think that probably should be done in the metaclass, so the strategy is registered whenever a type is defined and hypothesis is installed in the environment.

[antonagestam]

This would be really cool to have support for. I have no experience from registering custom hypothesis strategies, but my first instinct is I suspect they will need to be quite bespoke per type.

We also need to consider how we can introduce this without requiring hypothesis to be installed, but that should be doable.

[apirogov]

I also am not very experienced with constructing hypothesis strategies, but its mostly looking like generator chaining. I guess looking at the default strategies for number types and container types should be quite helpful, and for general predicates the user will need to provide the strategy using the hook.

Checking if it is installed is easy - something like

try:
   import hypothesis
   _HAS_HYPOTHESIS = True
except ImportError:
   _HAS_HYPOTHESIS = False

which is done only once say in _base.py, and then this flag can be used in __init_subclass__ or wherever this is needed.

[rsokl]

Hello! I'm a maintainer of Hypothesis (and a fan of phantom types!) and I happened to come across this.

You might be interested in using an entry point to automatically register strategies only when hypothesis is imported.

[antonagestam]

@rsokl Thanks for chiming in, that looks great and like exactly what we need! 🙏

I just ran into a case where I'd want Hypothesis + phantom-types for a work thing, so I might take a stab at this soonish.

[antonagestam]

I have WIP going, but I haven't yet fully convinced myself I want to do it this way, with inheritance, or whether strategy registration should be more separated.

#262

[antonagestam]

#262 is now merged.