When st.from_type() falls back to st.builds(), try passing infer for arguments with default values

[lsorber]

For functions, builds uses a just(default_value) strategy for kwargs with a default_value, but it also allows you to use infer to instead use something to the effect of just(default_value) | from_type(type_annotation) [1].

For Pydantic models, Hypothesis inspects the __signature__ [2] and will also use the equivalent of a just(default_value) strategy for fields with a default_value (cf. example below). It would be nice if we could also choose to use an infer-like strategy in this scenario. What would be the recommended workaround for that?

Side question: why was the default strategy for kwargs/fields with default values chosen to be just(default_value) instead of just(default_value) | from_type(type_annotation)?

from typing import Optional
import hypothesis.strategies as st
from pydantic import BaseModel

class Foo(BaseModel):
    bar: int
    baz: Optional[str] = None

st.from_type(Foo).example()  # `baz` will always be `None`, even though you might expect to see some `str`s

[1] https://hypothesis.readthedocs.io/en/latest/details.html#inferred-strategies
[2] #2388

[Zac-HD]

Why was the default strategy for kwargs/fields with default values chosen to be just(default_value) instead of just(default_value) | from_type(type_annotation)?

I think this, and the difference between builds(Foo) and from_type(Foo), is actually the heart of the matter.

builds(Foo, ...) calls Foo with arguments drawn from the provided strategies, and we later added support for inferring strategies for missing required arguments - which was previously an error. There's not actually a just(default_value) involved here; we're simply not providing any argument for parameters which have a default value unless explicitly told to do so via baz=infer.

On the other hand, from_type(Foo) does not have any particular meaning beyond "somehow generates instances of Foo - as implemented, we try looking up registered strategies for the exact type, then subtypes, and finally use builds() if we couldn't find anything. The 'working as intended' workaround is to st.register_type_strategy(Foo, st.builds(Foo, baz=infer) before st.from_type(Foo).

---

However, I think it would make sense to have from_type(Foo) call builds(Foo, baz=infer) internally - i.e. pass infer for args with default values - because the current conservative behaviour of builds() is an implementation detail of from_type() rather than part of the contract.

We would need to be careful not to break backwards compatibility by raising errors on arg: UnresolvableType = ...; this is certainly possible with some more introspection and a call to try: inferred_strategy.validate(); except ....

[mristin]

@Zac-HD This also has implications when registering types with contracts in Hypothesis. When you register a strategy for Foo, from_type works correctly, where builds(Foo) does not look up the registered strategy for the type.

Is that behavior intented?

[Zac-HD]

Yes, the current behaviour is intended.

For any registered type, from_type() will return whatever you registered, while builds() will call the constructor - which is the intended behaviour. The idea for this issue is that if nothing is registered and from_type() delegates to builds(), it would be appropriate to infer strategies for all arguments - after all, if that's not what the user wants they can always use builds() directly and/or register a strategy!

[mristin]

@Zac-HD thanks for the clarification. I'll document that the types with contracts in constructor should always use from_type and never builds. As these classes with contracts are automatically registered, this should work as expected.