Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

24.1.0: pytest fails in 3 units and some pytest warnings #1298

Closed
kloczek opened this issue Mar 14, 2024 · 12 comments
Closed

24.1.0: pytest fails in 3 units and some pytest warnings #1298

kloczek opened this issue Mar 14, 2024 · 12 comments

Comments

@kloczek
Copy link

kloczek commented Mar 14, 2024

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

  • python3 -sBm build -w --no-isolation
  • because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
  • install .whl file in </install/prefix> using installer module
  • run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
  • build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")
Here is pytest output: 
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pyOpenSSL-24.1.0-2.fc36.x86_64/usr/lib64/python3.9/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pyOpenSSL-24.1.0-2.fc36.x86_64/usr/lib/python3.9/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.9.18, pytest-8.1.1, pluggy-1.4.0
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
OpenSSL: b'OpenSSL 3.2.1 30 Jan 2024'
cryptography: 42.0.5
rootdir: /home/tkloczko/rpmbuild/BUILD/pyopenssl-24.1.0
configfile: pyproject.toml
testpaths: tests
plugins: hypothesis-6.99.5, benchmark-4.0.0, flaky-3.8.1
collected 504 items

tests/test_crypto.py .........................................................................................E.E.................................................................... [ 31%]
.............................................................................................                                                                                         [ 50%]
tests/test_debug.py .                                                                                                                                                                 [ 50%]
tests/test_rand.py ....                                                                                                                                                               [ 51%]
tests/test_ssl.py .....E............................................................................................................................................................. [ 83%]
..................................................................................                                                                                                    [ 99%]
tests/test_util.py .                                                                                                                                                                  [100%]

========================================================================================== ERRORS ===========================================================================================
___________________________________________________________________ ERROR at setup of TestX509.test_gmtime_adj_notBefore ____________________________________________________________________

>       lambda: ihook(item=item, **kwds), when=when, reraise=reraise
    )

/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <flaky.flaky_pytest_plugin.FlakyPlugin object at 0x7f12783c5850>, item = <Function test_gmtime_adj_notBefore>

    def pytest_runtest_setup(self, item):
        """
        Pytest hook to modify the test before it's run.

        :param item:
            The test item.
        """
        if not self._has_flaky_attributes(item):
            if hasattr(item, 'iter_markers'):
                for marker in item.iter_markers(name='flaky'):
>                   self._make_test_flaky(item, *marker.args, **marker.kwargs)
E                   TypeError: _make_test_flaky() got an unexpected keyword argument 'reruns'

/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:255: TypeError
____________________________________________________________________ ERROR at setup of TestX509.test_gmtime_adj_notAfter ____________________________________________________________________

>       lambda: ihook(item=item, **kwds), when=when, reraise=reraise
    )

/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <flaky.flaky_pytest_plugin.FlakyPlugin object at 0x7f12783c5850>, item = <Function test_gmtime_adj_notAfter>

    def pytest_runtest_setup(self, item):
        """
        Pytest hook to modify the test before it's run.

        :param item:
            The test item.
        """
        if not self._has_flaky_attributes(item):
            if hasattr(item, 'iter_markers'):
                for marker in item.iter_markers(name='flaky'):
>                   self._make_test_flaky(item, *marker.args, **marker.kwargs)
E                   TypeError: _make_test_flaky() got an unexpected keyword argument 'reruns'

/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:255: TypeError
____________________________________________________________ ERROR at setup of TestContext.test_set_cipher_list_no_cipher_match _____________________________________________________________

>       lambda: ihook(item=item, **kwds), when=when, reraise=reraise
    )

/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <flaky.flaky_pytest_plugin.FlakyPlugin object at 0x7f12783c5850>, item = <Function test_set_cipher_list_no_cipher_match>

    def pytest_runtest_setup(self, item):
        """
        Pytest hook to modify the test before it's run.

        :param item:
            The test item.
        """
        if not self._has_flaky_attributes(item):
            if hasattr(item, 'iter_markers'):
                for marker in item.iter_markers(name='flaky'):
>                   self._make_test_flaky(item, *marker.args, **marker.kwargs)
E                   TypeError: _make_test_flaky() got an unexpected keyword argument 'reruns'

/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:255: TypeError
===================================================================================== warnings summary ======================================================================================
tests/test_crypto.py:21
  /home/tkloczko/rpmbuild/BUILD/pyopenssl-24.1.0/tests/test_crypto.py:21: DeprecationWarning: sign() is deprecated. Use the equivalent APIs in cryptography.
    from OpenSSL.crypto import (

tests/test_crypto.py:21
  /home/tkloczko/rpmbuild/BUILD/pyopenssl-24.1.0/tests/test_crypto.py:21: DeprecationWarning: verify() is deprecated. Use the equivalent APIs in cryptography.
    from OpenSSL.crypto import (

tests/test_ssl.py::TestContext::test_set_cipher_list[hello world:AES128-SHA1]
  /home/tkloczko/rpmbuild/BUILD/pyopenssl-24.1.0/tests/test_ssl.py:499: DeprecationWarning: str for cipher_list is no longer accepted, use bytes
    context.set_cipher_list(cipher_string)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================== short test summary info ==================================================================================
ERROR tests/test_crypto.py::TestX509::test_gmtime_adj_notBefore - TypeError: _make_test_flaky() got an unexpected keyword argument 'reruns'
ERROR tests/test_crypto.py::TestX509::test_gmtime_adj_notAfter - TypeError: _make_test_flaky() got an unexpected keyword argument 'reruns'
ERROR tests/test_ssl.py::TestContext::test_set_cipher_list_no_cipher_match - TypeError: _make_test_flaky() got an unexpected keyword argument 'reruns'
========================================================================= 501 passed, 3 warnings, 3 errors in 8.16s =========================================================================
List of installed modules in build env: 
Package                       Version
----------------------------- -----------
alabaster                     0.7.16
Babel                         2.14.0
build                         1.1.1
cffi                          1.16.0
charset-normalizer            3.3.2
cryptography                  42.0.5
distro                        1.9.0
dnf                           4.19.0
docutils                      0.20.1
exceptiongroup                1.1.3
flaky                         3.8.1
gpg                           1.23.2
idna                          3.6
imagesize                     1.4.1
importlib_metadata            7.0.1
iniconfig                     2.0.0
installer                     0.7.0
Jinja2                        3.1.3
libdnf                        0.73.0
MarkupSafe                    2.1.3
packaging                     24.0
pluggy                        1.4.0
ply                           3.11
pretend                       1.0.9
pycparser                     2.21
Pygments                      2.17.2
pyproject_hooks               1.0.0
pytest                        8.1.1
python-dateutil               2.9.0.post0
requests                      2.31.0
setuptools                    69.1.1
snowballstemmer               2.2.0
Sphinx                        7.2.6
sphinx_rtd_theme              2.0.0
sphinxcontrib-applehelp       1.0.8
sphinxcontrib-devhelp         1.0.5
sphinxcontrib-htmlhelp        2.0.5
sphinxcontrib-jquery          4.1
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.7
sphinxcontrib-serializinghtml 1.1.10
tokenize_rt                   5.2.0
tomli                         2.0.1
urllib3                       1.26.18
wheel                         0.43.0
zipp                          3.17.0

Please let me know if you need more details or want me to perform some diagnostics.
@alex
Copy link
Member

alex commented Mar 14, 2024

It looks like you're still installing flaky, even though it was dropped as a dependency in 510b0e5

@kloczek
Copy link
Author

kloczek commented Mar 14, 2024

One sec let me check 😋

@kloczek
Copy link
Author

kloczek commented Mar 14, 2024

Hmm ..

  • that commit is not part of the master 24.1.0...main
  • I cannot apply patch from that commit on top of 24.1.0
warning: Downloading https://github.com/pyca/pyopenssl//archive/24.1.0/python-pyOpenSSL-24.1.0.tar.gz to /home/tkloczko/rpmbuild/SOURCES/python-pyOpenSSL-24.1.0.tar.gz
warning: Downloading https://github.com/pyca/pyopenssl//commit/510b0e57.patch#/python-pyOpenSSL-Stop-using-flaky-lib-use-pytest-rerunfailures-instead.patch to /home/tkloczko/rpmbuild/SOURCES/python-pyOpenSSL-Stop-using-flaky-lib-use-pytest-rerunfailures-instead.patch
warning: source_date_epoch_from_changelog set but %changelog is missing
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.4zzikU
+ umask 022
+ cd /home/tkloczko/rpmbuild/BUILD
+ cd /home/tkloczko/rpmbuild/BUILD
+ rm -rf pyopenssl-24.1.0
+ /usr/lib/rpm/rpmuncompress -x /home/tkloczko/rpmbuild/SOURCES/python-pyOpenSSL-24.1.0.tar.gz
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd pyopenssl-24.1.0
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ /usr/lib/rpm/rpmuncompress /home/tkloczko/rpmbuild/SOURCES/python-pyOpenSSL-Stop-using-flaky-lib-use-pytest-rerunfailures-instead.patch
+ /usr/bin/patch -p1 -s --fuzz=0 --no-backup-if-mismatch -f
1 out of 1 hunk FAILED -- saving rejects to file setup.py.rej
3 out of 3 hunks FAILED -- saving rejects to file tests/test_crypto.py.rej
2 out of 2 hunks FAILED -- saving rejects to file tests/test_ssl.py.rej
1 out of 1 hunk FAILED -- saving rejects to file tests/util.py.rej
error: Bad exit status from /var/tmp/rpm-tmp.4zzikU (%prep)

🤔

@kloczek
Copy link
Author

kloczek commented Mar 14, 2024

flaky still is used in pytest marks

[tkloczko@pers-jacek pyopenssl-24.1.0]$ grep -r flaky
tests/test_ssl.py:    @pytest.mark.flaky(reruns=2)
tests/test_crypto.py:    @pytest.mark.flaky(reruns=2)
tests/test_crypto.py:    @pytest.mark.flaky(reruns=2)

Without installed flaky pytest fails on those marks

==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.9.18, pytest-8.1.1, pluggy-1.4.0
OpenSSL: b'OpenSSL 3.2.1 30 Jan 2024'
cryptography: 42.0.5
rootdir: /home/tkloczko/rpmbuild/BUILD/pyopenssl-24.1.0
configfile: pyproject.toml
testpaths: tests
collected 6 items / 2 errors

========================================================================================== ERRORS ===========================================================================================
___________________________________________________________________________ ERROR collecting tests/test_crypto.py ___________________________________________________________________________
'flaky' not found in `markers` configuration option
____________________________________________________________________________ ERROR collecting tests/test_ssl.py _____________________________________________________________________________
'flaky' not found in `markers` configuration option
===================================================================================== warnings summary ======================================================================================
tests/test_crypto.py:21
tests/test_crypto.py:21
  /home/tkloczko/rpmbuild/BUILD/pyopenssl-24.1.0/tests/test_crypto.py:21: DeprecationWarning: sign() is deprecated. Use the equivalent APIs in cryptography.
    from OpenSSL.crypto import (

tests/test_crypto.py:21
tests/test_crypto.py:21
  /home/tkloczko/rpmbuild/BUILD/pyopenssl-24.1.0/tests/test_crypto.py:21: DeprecationWarning: verify() is deprecated. Use the equivalent APIs in cryptography.
    from OpenSSL.crypto import (

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================== short test summary info ==================================================================================
ERROR tests/test_crypto.py - Failed: 'flaky' not found in `markers` configuration option
ERROR tests/test_ssl.py - Failed: 'flaky' not found in `markers` configuration option
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=============================================================================== 4 warnings, 2 errors in 0.89s ===============================================================================

@kloczek
Copy link
Author

kloczek commented Mar 14, 2024

Ok so looks like remove using flaky is not fully finished. Am I right? 😋

@alex
Copy link
Member

alex commented Mar 14, 2024 via email

@kloczek
Copy link
Author

kloczek commented Mar 14, 2024

No, those marks are from pytest-rerunfailures, not flaky

One sec .. 😋

@kloczek
Copy link
Author

kloczek commented Mar 14, 2024

OK so pytest is OK however I see some warnings

Here is pytest output: 
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pyOpenSSL-24.1.0-2.fc36.x86_64/usr/lib64/python3.9/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pyOpenSSL-24.1.0-2.fc36.x86_64/usr/lib/python3.9/site-packages
+ /usr/bin/pytest -ra -m 'not network'
============================= test session starts ==============================
platform linux -- Python 3.9.18, pytest-8.1.1, pluggy-1.4.0
OpenSSL: b'OpenSSL 3.2.1 30 Jan 2024'
cryptography: 42.0.5
rootdir: /home/tkloczko/rpmbuild/BUILD/pyopenssl-24.1.0
configfile: pyproject.toml
testpaths: tests
plugins: rerunfailures-12.0
collected 504 items

tests/test_crypto.py ................................................... [ 10%]
........................................................................ [ 24%]
........................................................................ [ 38%]
..........................................................               [ 50%]
tests/test_debug.py .                                                    [ 50%]
tests/test_rand.py ....                                                  [ 51%]
tests/test_ssl.py ...................................................... [ 61%]
........................................................................ [ 76%]
........................................................................ [ 90%]
...............................................                          [ 99%]
tests/test_util.py .                                                     [100%]

=============================== warnings summary ===============================
tests/test_crypto.py:21
  /home/tkloczko/rpmbuild/BUILD/pyopenssl-24.1.0/tests/test_crypto.py:21: DeprecationWarning: sign() is deprecated. Use the equivalent APIs in cryptography.
    from OpenSSL.crypto import (

tests/test_crypto.py:21
  /home/tkloczko/rpmbuild/BUILD/pyopenssl-24.1.0/tests/test_crypto.py:21: DeprecationWarning: verify() is deprecated. Use the equivalent APIs in cryptography.
    from OpenSSL.crypto import (

tests/test_ssl.py::TestContext::test_set_cipher_list[hello world:AES128-SHA1]
  /home/tkloczko/rpmbuild/BUILD/pyopenssl-24.1.0/tests/test_ssl.py:499: DeprecationWarning: str for cipher_list is no longer accepted, use bytes
    context.set_cipher_list(cipher_string)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================= 504 passed, 3 warnings in 7.45s ========================

@alex
Copy link
Member

alex commented Mar 14, 2024 via email

@kloczek
Copy link
Author

kloczek commented Mar 14, 2024

Yes, those warnings are expected at this time

Thank you very much for your time 👍 😄
Closing

PS. Would you accept PR with filtering all pyopenssl code over pyupgrade --py38 and bumping minimum python version to 3.8 ?🤔
(3.7 has been EOSed +halt year ago)

@kloczek kloczek closed this as completed Mar 14, 2024
@alex
Copy link
Member

alex commented Mar 14, 2024 via email

@kloczek
Copy link
Author

kloczek commented Mar 15, 2024

No, we won't drop 37 until our users stop using it.

OK 👍

@alex alex mentioned this issue Mar 21, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@alex @kloczek