Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add helpful error message for st.sampled_from(EmptyEnumWithAnnotations) #2923

Closed
BrendanJM opened this issue Apr 6, 2021 · 3 comments · Fixed by #2924
Closed

Add helpful error message for st.sampled_from(EmptyEnumWithAnnotations) #2923

BrendanJM opened this issue Apr 6, 2021 · 3 comments · Fixed by #2924
Labels
legibility make errors helpful and Hypothesis grokable

Comments

@BrendanJM
Copy link

BrendanJM commented Apr 6, 2021 •
edited by Zac-HD
Loading

Following the discussion at #2693, the documented method of sampling from an Enum does not appear to work (or the documentation could use clarification and/or an example). The approach I tried is:

from enum import Enum
from hypothesis import given, strategies as st

class MyEnum(Enum):
    a: 'a'
    b: 'b'

@given(st.sampled_from(MyEnum))
def test_example(instance):
    assert True

This raises hypothesis.errors.InvalidArgument: Cannot sample from a length-zero sequence. Additionally, sampled_from(list(MyEnum)) raises the same issue. Hypothesis version is 6.8.5, python is 3.9.2
@BrendanJM
Copy link
Author

For context, the end goal I'm looking to achieve is generating Pydantic model, with some attributes being enums: https://pydantic-docs.helpmanual.io/hypothesis_plugin/

@Zac-HD Zac-HD added the legibility make errors helpful and Hypothesis grokable label Apr 7, 2021
@Zac-HD
Copy link
Member

Zac-HD commented Apr 7, 2021 •
edited
Loading

Your problem here is that MyEnum doesn't have any elements, because you've used annotations instead of assigning them - Enums are different to dataclasses or Pydantic like that. Replace : with =, and it'll work 🙂

This is a weird enough edge case that I didn't spot it immediately though, so I think we should upgrade st.sampled_from() to check vars(MyEnum).get('__annotations__') if list(MyEnum) == [] and raise an error that can explain what's happening.

@Zac-HD Zac-HD changed the title sampled_from(Enum) documentation and usage unclear Add helpful error message for st.sampled_from(EmptyEnumWithAnnotations) Apr 7, 2021
@Zac-HD Zac-HD mentioned this issue Apr 7, 2021
Merged
@BrendanJM
Copy link
Author

Well that's embarrassing - good catch, and thanks for the quick response!

@ThatXliner ThatXliner mentioned this issue Apr 21, 2021
Closed
bafranco10 added a commit to bafranco10/ivy that referenced this issue Dec 30, 2022
@bafranco10
Update fFix sorting.test_numpy_sort_complex ivy-llc#9123
2f3adde 
I have fixed the errors that are present in the existing issue, but I believe there is an issue with another part of the framework. It had said that a was missing and then that they had different data types so I fixed it. Now this error is present ../ivy_tests/test_ivy/test_frontends/test_numpy/test_sorting_searching_counting/test_sorting.py::test_numpy_sort_complex[cpu-ivy.functional.backends.numpy-False-False] FAILED [100%]
ivy_tests/test_ivy/test_frontends/test_numpy/test_sorting_searching_counting/test_sorting.py:112 (test_numpy_sort_complex[cpu-ivy.functional.backends.numpy-False-False])
@handle_frontend_test(
>       fn_tree="numpy.sort_complex",
        dtype_x_axis=helpers.dtype_values_axis(
            available_dtypes=helpers.get_dtypes("complex"),
            min_num_dims=1,
            min_dim_size=1,
            min_axis=-1,
            max_axis=0,
        ),
    )

../ivy_tests/test_ivy/test_frontends/test_numpy/test_sorting_searching_counting/test_sorting.py:114: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../ivy_tests/test_ivy/helpers/hypothesis_helpers/array_helpers.py:354: in dtype_values_axis
    results = draw(
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/internal/conjecture/data.py:941: in draw
    return strategy.do_draw(self)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/strategies/_internal/lazy.py:156: in do_draw
    return data.draw(self.wrapped_strategy)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/internal/conjecture/data.py:941: in draw
    return strategy.do_draw(self)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/strategies/_internal/core.py:1506: in do_draw
    return self.definition(data.draw, *self.args, **self.kwargs)
../ivy_tests/test_ivy/helpers/hypothesis_helpers/array_helpers.py:191: in dtype_and_values
    dtype = draw(
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/internal/conjecture/data.py:941: in draw
    return strategy.do_draw(self)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/strategies/_internal/lazy.py:156: in do_draw
    return data.draw(self.wrapped_strategy)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/internal/conjecture/data.py:941: in draw
    return strategy.do_draw(self)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/strategies/_internal/core.py:1506: in do_draw
    return self.definition(data.draw, *self.args, **self.kwargs)
../ivy_tests/test_ivy/helpers/hypothesis_helpers/dtype_helpers.py:142: in array_dtypes
    dtypes = draw(ah.list_of_length(x=st.sampled_from(available_dtypes), length=1))
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/internal/conjecture/data.py:927: in draw
    strategy.validate()
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/strategies/_internal/strategies.py:417: in validate
    self.do_validate()
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/strategies/_internal/lazy.py:129: in do_validate
    w = self.wrapped_strategy
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/strategies/_internal/lazy.py:103: in wrapped_strategy
    unwrapped_args = tuple(unwrap_strategies(s) for s in self.__args)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/strategies/_internal/lazy.py:103: in <genexpr>
    unwrapped_args = tuple(unwrap_strategies(s) for s in self.__args)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/strategies/_internal/lazy.py:42: in unwrap_strategies
    result = unwrap_strategies(s.wrapped_strategy)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/strategies/_internal/lazy.py:108: in wrapped_strategy
    base = self.function(*self.__args, **self.__kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

elements = []

    @defines_strategy(try_non_lazy=True)  # noqa: F811
    def sampled_from(
        elements: Union[Type[enum.Enum], Sequence[Any]]
    ) -> SearchStrategy[Any]:
        """Returns a strategy which generates any value present in ``elements``.
    
        Note that as with :func:`~hypothesis.strategies.just`, values will not be
        copied and thus you should be careful of using mutable data.
    
        ``sampled_from`` supports ordered collections, as well as
        :class:`~python:enum.Enum` objects.  :class:`~python:enum.Flag` objects
        may also generate any combination of their members.
    
        Examples from this strategy shrink by replacing them with values earlier in
        the list. So e.g. ``sampled_from([10, 1])`` will shrink by trying to replace
        1 values with 10, and ``sampled_from([1, 10])`` will shrink by trying to
        replace 10 values with 1.
    
        It is an error to sample from an empty sequence, because returning :func:`nothing`
        makes it too easy to silently drop parts of compound strategies.  If you need
        that behaviour, use ``sampled_from(seq) if seq else nothing()``.
        """
        values = check_sample(elements, "sampled_from")
        if not values:
            if (
                isinstance(elements, type)
                and issubclass(elements, enum.Enum)
                and vars(elements).get("__annotations__")
            ):
                # See HypothesisWorks/hypothesis#2923
                raise InvalidArgument(
                    f"Cannot sample from {elements.__module__}.{elements.__name__} "
                    "because it contains no elements.  It does however have annotations, "
                    "so maybe you tried to write an enum as if it was a dataclass?"
                )
>           raise InvalidArgument("Cannot sample from a length-zero sequence.")
E           hypothesis.errors.InvalidArgument: Cannot sample from a length-zero sequence.
E           
E           You can reproduce this example by temporarily adding @reproduce_failure('6.61.0', b'AA==') as a decorator on your test case

/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/hypothesis/strategies/_internal/core.py:200: InvalidArgument
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
legibility make errors helpful and Hypothesis grokable
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Improve error message for certain empty enums Zac-HD/hypothesis
2 participants
@Zac-HD @BrendanJM