[BUG] 5.8.0: pytest is failing in random units whem pytest-randomly is usied

[kloczek]

Summary

• OS: Linux
• Architecture: x86_64
• Psutil version: 5.8.0
• Python version: 3.8.11
• Type: tests

Description

I'm trying to package your module as rpm packag. So I'm using typical in such case build, install and test cycle used on building package from non-root account:

• "setup.py build"
• "setup.py install --root </install/prefix>"
• "pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

May I ask for help because few units are failing:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-psutil-5.8.0-12.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-psutil-5.8.0-12.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --import-mode=importlib
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=80250483
rootdir: /home/tkloczko/rpmbuild/BUILD/psutil-release-5.8.0
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, pyfakefs-4.5.0, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, xprocess-0.18.1, black-0.3.12, anyio-3.3.0, Faker-8.11.0, asyncio-0.15.1, trio-0.7.0, httpbin-1.0.0, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, randomly-3.8.0, checkdocs-2.7.1
collected 642 items

psutil/tests/test_memleaks.py ....................s.......s....s..ss.....s..................s.......s................s..........                                     [ 15%]
psutil/tests/test_process.py ...F.................s...............................s..........................                                                        [ 27%]
psutil/tests/test_sunos.py ss                                                                                                                                        [ 28%]
psutil/tests/test_unicode.py ...................                                                                                                                     [ 30%]
psutil/tests/test_testutils.py .................s...............                                                                                                     [ 36%]
psutil/tests/test_connections.py .................                                                                                                                   [ 38%]
psutil/tests/test_misc.py ssssssssssssssssssssssss.........................s.......                                                                                  [ 47%]
psutil/tests/test_bsd.py sssssssssssssssssssssssssssssssssssssssssssssss                                                                                             [ 54%]
psutil/tests/test_contracts.py ........................................                                                                                              [ 61%]
psutil/tests/test_system.py .....................sss........ss..........                                                                                             [ 68%]
psutil/tests/test_osx.py ssssssssssssss                                                                                                                              [ 70%]
psutil/tests/test_posix.py ........................                                                                                                                  [ 73%]
psutil/tests/test_linux.py ........................s..s..sssssssss............s..s.....sss...s.s..Fss.............................                                   [ 90%]
psutil/tests/test_windows.py ssssssssssssssssssssssssssssssssssssssssssssssssssssssssss                                                                              [ 99%]
psutil/tests/test_aix.py s                                                                                                                                           [ 99%]
. .                                                                                                                                                                  [ 99%]
psutil/tests/test_aix.py ssss                                                                                                                                        [100%]

================================================================================= FAILURES =================================================================================
________________________________________________________________________ TestProcess.test_terminal _________________________________________________________________________

self = <test_process.TestProcess testMethod=test_terminal>

    @unittest.skipIf(not POSIX, 'POSIX only')
    def test_terminal(self):
        terminal = psutil.Process().terminal()
        if terminal is not None:
>           tty = os.path.realpath(sh('tty'))

psutil/tests/test_process.py:299:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../BUILDROOT/python-psutil-5.8.0-12.fc35.x86_64/usr/lib64/python3.8/site-packages/psutil/tests/__init__.py:296: in wrapper
    return fun(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cmd = 'tty', kwds = {'creationflags': 0, 'shell': True, 'stderr': -1, 'stdout': -1, ...}, shell = True, flags = 0, p = <subprocess.Popen object at 0x7fccab8c4940>
stdout = 'not a tty\n', stderr = ''

    @_reap_children_on_err
    def sh(cmd, **kwds):
        """run cmd in a subprocess and return its output.
        raises RuntimeError on error.
        """
        shell = True if isinstance(cmd, (str, unicode)) else False
        # Prevents subprocess to open error dialogs in case of error.
        flags = 0x8000000 if WINDOWS and shell else 0
        kwds.setdefault("shell", shell)
        kwds.setdefault("stdout", subprocess.PIPE)
        kwds.setdefault("stderr", subprocess.PIPE)
        kwds.setdefault("universal_newlines", True)
        kwds.setdefault("creationflags", flags)
        p = subprocess.Popen(cmd, **kwds)
        _subprocesses_started.add(p)
        if PY3:
            stdout, stderr = p.communicate(timeout=GLOBAL_TIMEOUT)
        else:
            stdout, stderr = p.communicate()
        if p.returncode != 0:
>           raise RuntimeError(stderr)
E           RuntimeError

../../BUILDROOT/python-psutil-5.8.0-12.fc35.x86_64/usr/lib64/python3.8/site-packages/psutil/tests/__init__.py:467: RuntimeError
______________________________________________________________________ TestSystemNetIfAddrs.test_ips _______________________________________________________________________

self = <test_linux.TestSystemNetIfAddrs testMethod=test_ips>

    def test_ips(self):
        for name, addrs in psutil.net_if_addrs().items():
            for addr in addrs:
                if addr.family == psutil.AF_LINK:
                    self.assertEqual(addr.address, get_mac_address(name))
                elif addr.family == socket.AF_INET:
                    self.assertEqual(addr.address, get_ipv4_address(name))
                    self.assertEqual(addr.netmask, get_ipv4_netmask(name))
                    if addr.broadcast is not None:
                        self.assertEqual(addr.broadcast,
                                         get_ipv4_broadcast(name))
                    else:
                        self.assertEqual(get_ipv4_broadcast(name), '0.0.0.0')
                elif addr.family == socket.AF_INET6:
                    # IPv6 addresses can have a percent symbol at the end.
                    # E.g. these 2 are equivalent:
                    # "fe80::1ff:fe23:4567:890a"
                    # "fe80::1ff:fe23:4567:890a%eth0"
                    # That is the "zone id" portion, which usually is the name
                    # of the network interface.
                    address = addr.address.split('%')[0]
>                   self.assertEqual(address, get_ipv6_address(name))
E                   AssertionError: 'fe80::bd93:94d1:e85b:9aa1' != 'fdaa:bbcc:ddee:0:caa4:d494:16e7:697a'
E                   - fe80::bd93:94d1:e85b:9aa1
E                   + fdaa:bbcc:ddee:0:caa4:d494:16e7:697a

psutil/tests/test_linux.py:949: AssertionError
========================================================================= short test summary info ==========================================================================
SKIPPED [2] psutil/tests/test_memleaks.py:262: WINDOWS only
SKIPPED [2] psutil/tests/test_memleaks.py:150: worthless on POSIX
SKIPPED [2] psutil/tests/test_memleaks.py:165: WINDOWS only
SKIPPED [60] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: WINDOWS only
SKIPPED [1] psutil/tests/test_memleaks.py:370: WINDOWS only
SKIPPED [1] psutil/tests/test_process.py:518: WINDOWS only
SKIPPED [1] psutil/tests/test_process.py:389: not supported on this win version
SKIPPED [2] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: SUNOS only
SKIPPED [1] psutil/tests/test_testutils.py:367: Unreliable in mock
SKIPPED [24] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: can't locate scripts directory
SKIPPED [1] psutil/tests/test_misc.py:357: can't find setup.py
SKIPPED [5] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: BSD only
SKIPPED [32] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: FREEBSD only
SKIPPED [1] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: OPENBSD only
SKIPPED [9] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: NETBSD only
SKIPPED [1] psutil/tests/test_system.py:514: Unreliable in mock
SKIPPED [1] psutil/tests/test_system.py:859: no battery
SKIPPED [1] psutil/tests/test_system.py:833: Unreliable in mock
SKIPPED [1] psutil/tests/test_system.py:583: unreliable on CI
SKIPPED [1] psutil/tests/test_system.py:574: not an ASCII fs
SKIPPED [14] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: MACOS only
SKIPPED [1] psutil/tests/test_linux.py:763: Unreliable in mock
SKIPPED [1] psutil/tests/test_linux.py:749: Unreliable in mock
SKIPPED [9] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: no battery
SKIPPED [1] psutil/tests/test_linux.py:2130: Unreliable in mock
SKIPPED [1] psutil/tests/test_linux.py:2137: Unreliable in mock
SKIPPED [5] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: Unreliable in mock
SKIPPED [1] psutil/tests/test_linux.py:666: Unreliable on mock
SKIPPED [1] psutil/tests/test_linux.py:672: Unreliable on mock
SKIPPED [1] psutil/tests/test_aix.py:23: AIX only
SKIPPED [1] psutil/tests/test_aix.py:107: AIX only
SKIPPED [1] psutil/tests/test_aix.py:74: AIX only
SKIPPED [1] psutil/tests/test_aix.py:113: AIX only
SKIPPED [1] psutil/tests/test_aix.py:53: AIX only
FAILED psutil/tests/test_process.py::TestProcess::test_terminal - RuntimeError
FAILED psutil/tests/test_linux.py::TestSystemNetIfAddrs::test_ips - AssertionError: 'fe80::bd93:94d1:e85b:9aa1' != 'fdaa:bbcc:ddee:0:caa4:d494:16e7:697a'
=============================================================== 2 failed, 452 passed, 188 skipped in 33.29s ================================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.

[kloczek]

After deselecting those two units another two popped out

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-psutil-5.8.0-12.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-psutil-5.8.0-12.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --import-mode=importlib --deselect psutil/tests/test_linux.py::TestSystemNetIfAddrs::test_ips --deselect psutil/tests/test_process.py::TestProcess::test_terminal
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=1805326212
rootdir: /home/tkloczko/rpmbuild/BUILD/psutil-release-5.8.0
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, pyfakefs-4.5.0, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, xprocess-0.18.1, black-0.3.12, anyio-3.3.0, Faker-8.11.0, asyncio-0.15.1, trio-0.7.0, httpbin-1.0.0, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, randomly-3.8.0, checkdocs-2.7.1
collected 642 items / 2 deselected / 640 selected

psutil/tests/test_process.py .........s....................................s................................                                                         [ 12%]
psutil/tests/test_memleaks.py ss.....s........ss..............................s.....................s................s.........s                                     [ 27%]
psutil/tests/test_unicode.py ...................                                                                                                                     [ 30%]
psutil/tests/test_aix.py s                                                                                                                                           [ 30%]
. .                                                                                                                                                                  [ 30%]
psutil/tests/test_aix.py ssss                                                                                                                                        [ 31%]
psutil/tests/test_osx.py ssssssssssssss                                                                                                                              [ 33%]
psutil/tests/test_sunos.py ss                                                                                                                                        [ 34%]
psutil/tests/test_system.py ..........s....s.ss..................s......                                                                                             [ 40%]
psutil/tests/test_contracts.py ........................................                                                                                              [ 47%]
psutil/tests/test_bsd.py sssssssssssssssssssssssssssssssssssssssssssssss                                                                                             [ 54%]
psutil/tests/test_misc.py ssssssssssssssssssssssss..s................F.............                                                                                  [ 63%]
psutil/tests/test_posix.py ........................                                                                                                                  [ 67%]
psutil/tests/test_testutils.py ...s.............................                                                                                                     [ 72%]
psutil/tests/test_connections.py .................                                                                                                                   [ 75%]
psutil/tests/test_linux.py .............sssssssss................ss....sssss.ss........ss...............F........................                                    [ 90%]
psutil/tests/test_windows.py ssssssssssssssssssssssssssssssssssssssssssssssssssssssssss                                                                              [100%]

================================================================================= FAILURES =================================================================================
___________________________________________________________________________ TestMisc.test__all__ ___________________________________________________________________________

self = <test_misc.TestMisc testMethod=test__all__>

    def test__all__(self):
        dir_psutil = dir(psutil)
        for name in dir_psutil:
            if name in ('long', 'tests', 'test', 'PermissionError',
                        'ProcessLookupError'):
                continue
            if not name.startswith('_'):
                try:
>                   __import__(name)
E                   ModuleNotFoundError: No module named 'cpu_freq'

psutil/tests/test_misc.py:172: ModuleNotFoundError

During handling of the above exception, another exception occurred:

self = <test_misc.TestMisc testMethod=test__all__>

    def test__all__(self):
        dir_psutil = dir(psutil)
        for name in dir_psutil:
            if name in ('long', 'tests', 'test', 'PermissionError',
                        'ProcessLookupError'):
                continue
            if not name.startswith('_'):
                try:
                    __import__(name)
                except ImportError:
                    if name not in psutil.__all__:
                        fun = getattr(psutil, name)
                        if fun is None:
                            continue
                        if (fun.__doc__ is not None and
                                'deprecated' not in fun.__doc__.lower()):
>                           self.fail('%r not in psutil.__all__' % name)
E                           AssertionError: 'cpu_freq' not in psutil.__all__

psutil/tests/test_misc.py:180: AssertionError
_________________________________________________________________________ TestMisc.test_issue_687 __________________________________________________________________________

self = <test_linux.TestMisc testMethod=test_issue_687>

    @retry_on_failure()
    def test_issue_687(self):
        # In case of thread ID:
        # - pid_exists() is supposed to return False
        # - Process(tid) is supposed to work
        # - pids() should not return the TID
        # See: https://github.com/giampaolo/psutil/issues/687
        t = ThreadTask()
        t.start()
        try:
            p = psutil.Process()
            threads = p.threads()
>           self.assertEqual(len(threads), 2)
E           AssertionError: 6 != 2

psutil/tests/test_linux.py:1439: AssertionError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
AssertionError('6 != 2'), retrying
AssertionError('6 != 2'), retrying
AssertionError('6 != 2'), retrying
AssertionError('6 != 2'), retrying
AssertionError('6 != 2'), retrying
AssertionError('6 != 2'), retrying
AssertionError('6 != 2'), retrying
AssertionError('6 != 2'), retrying
AssertionError('6 != 2'), retrying
AssertionError('6 != 2'), retrying
========================================================================= short test summary info ==========================================================================
SKIPPED [1] psutil/tests/test_process.py:389: not supported on this win version
SKIPPED [1] psutil/tests/test_process.py:518: WINDOWS only
SKIPPED [60] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: WINDOWS only
SKIPPED [2] psutil/tests/test_memleaks.py:150: worthless on POSIX
SKIPPED [2] psutil/tests/test_memleaks.py:262: WINDOWS only
SKIPPED [2] psutil/tests/test_memleaks.py:165: WINDOWS only
SKIPPED [1] psutil/tests/test_memleaks.py:370: WINDOWS only
SKIPPED [1] psutil/tests/test_aix.py:23: AIX only
SKIPPED [1] psutil/tests/test_aix.py:113: AIX only
SKIPPED [1] psutil/tests/test_aix.py:74: AIX only
SKIPPED [1] psutil/tests/test_aix.py:53: AIX only
SKIPPED [1] psutil/tests/test_aix.py:107: AIX only
SKIPPED [14] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: MACOS only
SKIPPED [2] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: SUNOS only
SKIPPED [1] psutil/tests/test_system.py:583: unreliable on CI
SKIPPED [1] psutil/tests/test_system.py:574: not an ASCII fs
SKIPPED [1] psutil/tests/test_system.py:833: Unreliable in mock
SKIPPED [1] psutil/tests/test_system.py:859: no battery
SKIPPED [1] psutil/tests/test_system.py:514: Unreliable in mock
SKIPPED [1] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: OPENBSD only
SKIPPED [5] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: BSD only
SKIPPED [32] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: FREEBSD only
SKIPPED [9] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: NETBSD only
SKIPPED [24] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: can't locate scripts directory
SKIPPED [1] psutil/tests/test_misc.py:357: can't find setup.py
SKIPPED [1] psutil/tests/test_testutils.py:367: Unreliable in mock
SKIPPED [9] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: no battery
SKIPPED [1] psutil/tests/test_linux.py:672: Unreliable on mock
SKIPPED [1] psutil/tests/test_linux.py:666: Unreliable on mock
SKIPPED [5] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: Unreliable in mock
SKIPPED [1] psutil/tests/test_linux.py:749: Unreliable in mock
SKIPPED [1] psutil/tests/test_linux.py:763: Unreliable in mock
SKIPPED [1] psutil/tests/test_linux.py:2130: Unreliable in mock
SKIPPED [1] psutil/tests/test_linux.py:2137: Unreliable in mock
FAILED psutil/tests/test_misc.py::TestMisc::test__all__ - AssertionError: 'cpu_freq' not in psutil.__all__
FAILED psutil/tests/test_linux.py::TestMisc::test_issue_687 - AssertionError: 6 != 2
======================================================== 2 failed, 450 passed, 188 skipped, 2 deselected in 33.91s =========================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.

3rd test without deselecting those two was OK:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-psutil-5.8.0-12.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-psutil-5.8.0-12.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --import-mode=importlib --deselect psutil/tests/test_linux.py::TestSystemNetIfAddrs::test_ips --deselect psutil/tests/test_process.py::TestProcess::test_terminal
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=168735774
rootdir: /home/tkloczko/rpmbuild/BUILD/psutil-release-5.8.0
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, pyfakefs-4.5.0, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, xprocess-0.18.1, black-0.3.12, anyio-3.3.0, Faker-8.11.0, asyncio-0.15.1, trio-0.7.0, httpbin-1.0.0, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, randomly-3.8.0, checkdocs-2.7.1
collected 642 items / 2 deselected / 640 selected

psutil/tests/test_testutils.py ......s..........................                                                                                                     [  5%]
psutil/tests/test_connections.py .................                                                                                                                   [  7%]
psutil/tests/test_contracts.py ........................................                                                                                              [ 14%]
psutil/tests/test_osx.py ssssssssssssss                                                                                                                              [ 16%]
psutil/tests/test_unicode.py ...................                                                                                                                     [ 19%]
psutil/tests/test_sunos.py ss                                                                                                                                        [ 19%]
psutil/tests/test_process.py .........................s..........................s..........................                                                         [ 31%]
psutil/tests/test_linux.py ...................ss.....................sssssssss..........ss...................s..s.sss....ss......                                    [ 47%]
psutil/tests/test_memleaks.py ss..s........................................s.s...........s....s...................s..s..........                                     [ 63%]
psutil/tests/test_aix.py ss                                                                                                                                          [ 63%]
. .                                                                                                                                                                  [ 63%]
psutil/tests/test_aix.py sss                                                                                                                                         [ 64%]
psutil/tests/test_posix.py ........................                                                                                                                  [ 67%]
psutil/tests/test_system.py .....s....................s.s.....ss........                                                                                             [ 74%]
psutil/tests/test_misc.py ssssssssssssssssssssssss............................s....                                                                                  [ 83%]
psutil/tests/test_windows.py ssssssssssssssssssssssssssssssssssssssssssssssssssssssssss                                                                              [ 92%]
psutil/tests/test_bsd.py sssssssssssssssssssssssssssssssssssssssssssssss                                                                                             [100%]

========================================================================= short test summary info ==========================================================================
SKIPPED [1] psutil/tests/test_testutils.py:367: Unreliable in mock
SKIPPED [14] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: MACOS only
SKIPPED [2] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: SUNOS only
SKIPPED [1] psutil/tests/test_process.py:389: not supported on this win version
SKIPPED [1] psutil/tests/test_process.py:518: WINDOWS only
SKIPPED [1] psutil/tests/test_linux.py:749: Unreliable in mock
SKIPPED [1] psutil/tests/test_linux.py:763: Unreliable in mock
SKIPPED [9] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: no battery
SKIPPED [5] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: Unreliable in mock
SKIPPED [1] psutil/tests/test_linux.py:672: Unreliable on mock
SKIPPED [1] psutil/tests/test_linux.py:666: Unreliable on mock
SKIPPED [1] psutil/tests/test_linux.py:2137: Unreliable in mock
SKIPPED [1] psutil/tests/test_linux.py:2130: Unreliable in mock
SKIPPED [60] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: WINDOWS only
SKIPPED [1] psutil/tests/test_memleaks.py:370: WINDOWS only
SKIPPED [2] psutil/tests/test_memleaks.py:262: WINDOWS only
SKIPPED [2] psutil/tests/test_memleaks.py:165: WINDOWS only
SKIPPED [2] psutil/tests/test_memleaks.py:150: worthless on POSIX
SKIPPED [1] psutil/tests/test_aix.py:107: AIX only
SKIPPED [1] psutil/tests/test_aix.py:23: AIX only
SKIPPED [1] psutil/tests/test_aix.py:53: AIX only
SKIPPED [1] psutil/tests/test_aix.py:74: AIX only
SKIPPED [1] psutil/tests/test_aix.py:113: AIX only
SKIPPED [1] psutil/tests/test_system.py:514: Unreliable in mock
SKIPPED [1] psutil/tests/test_system.py:583: unreliable on CI
SKIPPED [1] psutil/tests/test_system.py:574: not an ASCII fs
SKIPPED [1] psutil/tests/test_system.py:859: no battery
SKIPPED [1] psutil/tests/test_system.py:833: Unreliable in mock
SKIPPED [24] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: can't locate scripts directory
SKIPPED [1] psutil/tests/test_misc.py:357: can't find setup.py
SKIPPED [32] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: FREEBSD only
SKIPPED [5] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: BSD only
SKIPPED [9] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: NETBSD only
SKIPPED [1] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: OPENBSD only
============================================================= 452 passed, 188 skipped, 2 deselected in 33.14s ==============================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.

I've checked and I have installed pytest-randomly extension and looks like it messes in test suite.
Looksl like psutil test suite has some dependencies between units and random ordrt of the pytest units execution is breaking those dependencies.
Possible temporary work around: run pytest with -p no:randomly .
Do diagnose those issues can be used https://github.com/mrbean-bremen/pytest-find-dependencies/

[kloczek]

Strange .. looks like even with disabled pytest-randomly pytest execution produces random results

FAILED psutil/tests/test_contracts.py::TestFetchAllProcesses::test_all - AssertionError: 'kworker/u97:4-events_unbound' != 'kworker/u97:4+events_unbound'
FAILED psutil/tests/test_linux.py::TestSystemNetIfAddrs::test_ips - AssertionError: 'fe80::bd93:94d1:e85b:9aa1' != 'fdaa:bbcc:ddee:0:caa4:d494:16e7:697a'
FAILED psutil/tests/test_linux.py::TestMisc::test_issue_687 - AssertionError: 6 != 2
FAILED psutil/tests/test_process.py::TestProcess::test_terminal - RuntimeError

FAILED psutil/tests/test_linux.py::TestSystemNetIfAddrs::test_ips - AssertionError: 'fe80::bd93:94d1:e85b:9aa1' != 'fdaa:bbcc:ddee:0:caa4:d494:16e7:697a'
FAILED psutil/tests/test_linux.py::TestMisc::test_issue_687 - AssertionError: 6 != 2
FAILED psutil/tests/test_process.py::TestProcess::test_terminal - RuntimeError

FAILED psutil/tests/test_linux.py::TestSystemNetIfAddrs::test_ips - AssertionError: 'fe80::bd93:94d1:e85b:9aa1' != 'fdaa:bbcc:ddee:0:caa4:d494:16e7:697a'
FAILED psutil/tests/test_linux.py::TestMisc::test_issue_687 - AssertionError: 6 != 2
FAILED psutil/tests/test_process.py::TestProcess::test_terminal - RuntimeError

FAILED psutil/tests/test_contracts.py::TestFetchAllProcesses::test_all - AssertionError: 'kworker/11:0-events' != 'kworker/11:0-mm_percpu_wq'
FAILED psutil/tests/test_linux.py::TestSystemNetIfAddrs::test_ips - AssertionError: 'fe80::bd93:94d1:e85b:9aa1' != 'fdaa:bbcc:ddee:0:caa4:d494:16e7:697a'
FAILED psutil/tests/test_linux.py::TestMisc::test_issue_687 - AssertionError: 6 != 2
FAILED psutil/tests/test_process.py::TestProcess::test_terminal - RuntimeError