Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AssertionError when starting pytest session #3265

Closed
greyblue9 opened this issue Mar 24, 2022 · 8 comments
Closed

AssertionError when starting pytest session #3265

greyblue9 opened this issue Mar 24, 2022 · 8 comments
Labels
legibility make errors helpful and Hypothesis grokable

Comments

@greyblue9
Copy link

greyblue9 commented Mar 24, 2022

When starting a pytest session in a brand-new python 3.10 venv running on Debian Linux (sid), and running pytest via ./venv/bin/python3.10 -m pytest, an assertion is encountered during the plugin gathering (I assume?) process in which the hypothesis pytest plugin is initialized.

At this point it looks like it is initializing some of the strategy types, and hits an assert while calling a function that looks related to numeric strategy:

AssertionError: assert (False)
  +  where False = is_negative(0.0)

Full traceback and pytest log (happy to provide -vv and/or trace/debug logs from pytest if it would help):

=========================== test session starts ============================
platform linux -- Python 3.10.0rc2+, pytest-7.1.1, pluggy-1.0.0 -- /data/media/0/src/use/src/venv/bin/python3.10
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/_pytest/main.py", line 266, in wrap_session
INTERNALERROR>     config.hook.pytest_sessionstart(session=session)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/_pytest/terminal.py", line 710, in pytest_sessionstart
INTERNALERROR>     lines = self.config.hook.pytest_report_header(
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/_pytest/config/compat.py", line 67, in fixed_hook
INTERNALERROR>     return hook(**kw)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/_hypothesis_pytestplugin.py", line 131, in pytest_report_header
INTERNALERROR>     from hypothesis import Verbosity, settings
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 168, in exec_module
INTERNALERROR>     exec(co, module.__dict__)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/hypothesis/__init__.py", line 56, in <module>
INTERNALERROR>     run()
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/hypothesis/entry_points.py", line 63, in run
INTERNALERROR>     hook = entry.load()
INTERNALERROR>   File "/opt/python310/lib/python3.10/importlib/metadata/__init__.py", line 162, in load
INTERNALERROR>     module = import_module(match.group('module'))
INTERNALERROR>   File "/opt/python310/lib/python3.10/importlib/__init__.py", line 126, in import_module
INTERNALERROR>     return _bootstrap._gcd_import(name[level:], package, level)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/pydantic/_hypothesis_plugin.py", line 362, in <module>
INTERNALERROR>     _registered(typ)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/pydantic/_hypothesis_plugin.py", line 196, in _registered
INTERNALERROR>     st.register_type_strategy(typ, resolver(typ))  # type: ignore
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/hypothesis/strategies/_internal/core.py", line 1785, in register_type_strategy
INTERNALERROR>     elif isinstance(strategy, SearchStrategy) and strategy.is_empty:
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/hypothesis/strategies/_internal/strategies.py", line 134, in accept
INTERNALERROR>     recur(self)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/hypothesis/strategies/_internal/strategies.py", line 130, in recur
INTERNALERROR>     mapping[strat] = getattr(strat, calculation)(recur)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/hypothesis/strategies/_internal/lazy.py", line 88, in calc_is_empty
INTERNALERROR>     return recur(self.wrapped_strategy)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/hypothesis/strategies/_internal/lazy.py", line 108, in wrapped_strategy
INTERNALERROR>     base = self.function(*self.__args, **self.__kwargs)
INTERNALERROR>   File "/data/media/0/src/use/src/venv/lib/python3.10/site-packages/hypothesis/strategies/_internal/numbers.py", line 401, in floats
INTERNALERROR>     assert is_negative(max_value) and not is_negative(max_arg)
INTERNALERROR> AssertionError: assert (False)
INTERNALERROR>  +  where False = is_negative(0.0)

The process exit status is 3.

Full list of installed packages:

./venv/bin/python3.10 -m pip list
Package            Version
------------------ ---------
asttokens          2.0.5
attrs              21.4.0
beartype           0.10.4
certifi            2021.10.8
charset-normalizer 2.0.12
furl               2.1.3
hypothesis         6.39.4
icontract          2.6.1
idna               3.3
iniconfig          1.1.1
Jinja2             3.0.3
justuse            0.7.1
MarkupSafe         2.1.1
orderedmultidict   1.0.1
packaging          21.0
pip                21.2.3
pluggy             1.0.0
py                 1.11.0
pydantic           1.9.0
pyparsing          3.0.7
pytest             7.1.1
requests           2.27.1
setuptools         57.4.0
six                1.16.0
sortedcontainers   2.4.0
toml               0.10.2
tomli              2.0.1
typing_extensions  4.1.1
urllib3            1.26.9
wheel              0.37.1

Thanks for looking at this issue.

@Zac-HD
Copy link
Member

Zac-HD commented Mar 24, 2022

It looks like your environment is running without signed zero? This is a standards-violating compiler optimisation, people sometimes use in hopes of better performance (e.g. -ffast-math, https://simonbyrne.github.io/notes/fastmath/), but honestly I don't think it's worth it.

To check: can your Python represent -0.0?

@Zac-HD Zac-HD added the legibility make errors helpful and Hypothesis grokable label Mar 26, 2022
@Zac-HD
Copy link
Member

Zac-HD commented Mar 26, 2022

(whatever the cause, it would be nice to give a high-diagnostic-value error message in these cases, rather than an assertion error. I'm coming around to the idea of leading every call to floats() with a quick fp-conformance check, and if we're careful it shouldn't even have much performance overhead...)

@Zac-HD
Copy link
Member

Zac-HD commented Mar 26, 2022

Oh, I just noticed that you're using Python 3.10.0rc2+; you should update to a stable Python 3.10.

Given that I'm closing this issue as out of scope, although note that we do welcome bug reports about upcoming pre-releases - e.g. we currently have Python 3.11 alpha in our CI system, and aspire to fix the known issues before the stable release in October.

@Zac-HD Zac-HD closed this as completed Mar 26, 2022
@greyblue9
Copy link
Author

@Zac-HD --

I'm really sorry, I meant to reply tp your original follow-up, but I guess I forgot to hit 'Comment'. Yes I think that was the issue! I had compiled with -funsafe-math-optimizations 😂🤣 Something I'll never do again.

Thank you so much for the quick turnaround amd for making great suggestions and comments. I learned quite a bit and came away feeling positive (no pun intended).

@greyblue9
Copy link
Author

greyblue9 commented Mar 26, 2022

Also, here's my output, attempting to check if the python can represent signed zero. I'm happy to do more if you're interested:

/data/media/0/src/python310/build/python.exe
Python 3.10.0rc2+ (heads/branch-v3.10.0-dirty:e0b61b2, Dec 20 2021, 08:28:23) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> -0.0
-0.0
>>> 0.0
0.0
>>> (-0.1) * (0/0.1)
-0.0

@greyblue9
Copy link
Author

Oh, I just noticed that you're using Python 3.10.0rc2+; you should update to a stable Python 3.10.

Given that I'm closing this issue as out of scope, although note that we do welcome bug reports about upcoming pre-releases - e.g. we currently have Python 3.11 alpha in our CI system, and aspire to fix the known issues before the stable release in October.

Oh that's awesome that you're using 3.11. I am using it off and on as well. I use it as my libpython for gdb. The error messages are truly next level.

(Side note, I just saw the video by (core dev) Raymond Hettinger yesterday where he talks about testing tools. He has some glowing words to say about hypothesis. You've probably already heard about but if not, it's worth a watch: https://m.youtube.com/watch?v=ARKbfWk4Xyw)

Keep up the awesome work! ✌💥

@Zac-HD
Copy link
Member

Zac-HD commented Mar 26, 2022

-funsafe-math-optimizations

Who wouldn't want fun, safe math optimizations?? Unfortunately "If fast-math was to give always the correct results, it wouldn’t be fast-math, it would be the standard way of doing math." 😅

The canonical way of checking is the math.copysign() function, though I always have to check the order of arguments... got a PR up with that right now.

3.11's error messages and handling are so exciting, I can't wait for the tracebacks and the ExceptionGroup type. And PEP-678 notes are also going to be pretty nice, if I say so myself 😉

@greyblue9
Copy link
Author

The canonical way of checking is the math.copysign() function, though I always have to check the order of arguments... got a PR up with that right now.

Ah, cool, I learned something new!

And with respect to the error messages (and new caret(s)) , sometimes I run 3.11 just to benefit from its more specific error messages 🤣

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
legibility make errors helpful and Hypothesis grokable
Projects
None yet
Development

No branches or pull requests

2 participants