Improve reporting of failing examples #3529
Labels
enhancement
it's not broken, but we want it to be better
legibility
make errors helpful and Hypothesis grokable
It's been a while since we revisited the way that Hypothesis reports failing examples, but I think it's time:
@example()
decorator, perhaps with.via()
on Python 3.9+, and reporting failures in that format seems like an effective way to do so - and makes reproducing a mere matter of copy-pasting when the database doesn't make it fully automatic.For both of these reasons, using the
__repr__
of custom objects is unsatisfying, because they often - and by default - don't consist of executable code which would return an equivalent object. Happily, our friends over at CrossHair recently solved this problem in a way I think we can imitate: represent custom objects by (recursively) representing the call that we executed in order to create them!You can see their implementation here; in short we can store a tree of fragments (via
get_pretty_function_description()
etc.) in a dict keyed off the ID of the object, and spanning the lifetime of that example. For anything not in the dict, i.e. where the function call was not executed by Hypothesisbuilds()
or.map()
(or flatmap, or recursive, or etc.), we'll fall back to our existing pretty-printing code. One notable divergence: I'll want to store the start and end span of the underlying buffer that generated each fragment, since that's a key component of #3411.The text was updated successfully, but these errors were encountered: