24.3.3: pytest fails in pydoctor/test/test_cyclic_imports_base_classes.py::test_cyclic_imports_base_classes unit

[kloczek]

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix> using installer module
• run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
• build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pydoctor-24.3.3-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pydoctor-24.3.3-2.fc37.x86_64/usr/lib/python3.10/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.10.14, pytest-8.2.1, pluggy-1.5.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pydoctor-24.3.3
configfile: setup.cfg
testpaths: pydoctor/test
plugins: hypothesis-6.100.0, subtests-0.12.1
collected 1341 items

pydoctor/test/epydoc/epytext.doctest .                                                                                                                                                [  0%]
pydoctor/test/epydoc/restructuredtext.doctest .                                                                                                                                       [  0%]
pydoctor/test/epydoc/test_epytext.py ....                                                                                                                                             [  0%]
pydoctor/test/epydoc/test_epytext2html.py ...........                                                                                                                                 [  1%]
pydoctor/test/epydoc/test_epytext2node.py .                                                                                                                                           [  1%]
pydoctor/test/epydoc/test_google_numpy.py .....                                                                                                                                       [  1%]
pydoctor/test/epydoc/test_parsed_docstrings.py .                                                                                                                                      [  1%]
pydoctor/test/epydoc/test_pyval_repr.py ....................................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,.                                           [  5%]
pydoctor/test/epydoc/test_restructuredtext.py ..................                                                                                                                      [  7%]
pydoctor/test/test_astbuilder.py .................................................................................................................................................... [ 18%]
..................................................................................................................................................................................... [ 31%]
..................................................................................................................................................................................... [ 45%]
..................................................................................................................................................................................... [ 58%]
..................................................................................................................                                                                    [ 67%]
pydoctor/test/test_attrs.py ..........                                                                                                                                                [ 67%]
pydoctor/test/test_colorize.py .....                                                                                                                                                  [ 68%]
pydoctor/test/test_commandline.py ...................                                                                                                                                 [ 69%]
pydoctor/test/test_configparser.py ......                                                                                                                                             [ 70%]
pydoctor/test/test_cyclic_imports_base_classes.py F                                                                                                                                   [ 70%]
pydoctor/test/test_epydoc2stan.py ..........xx................................................................................                                                        [ 77%]
pydoctor/test/test_model.py ...............s........                                                                                                                                  [ 78%]
pydoctor/test/test_mro.py ..........                                                                                                                                                  [ 79%]
pydoctor/test/test_napoleon_docstring.py .......................................................                                                                                      [ 83%]
pydoctor/test/test_napoleon_iterators.py .................                                                                                                                            [ 84%]
pydoctor/test/test_node2stan.py ..                                                                                                                                                    [ 85%]
pydoctor/test/test_options.py ..........                                                                                                                                              [ 85%]
pydoctor/test/test_packages.py ............                                                                                                                                           [ 86%]
pydoctor/test/test_pydantic_fields.py ..                                                                                                                                              [ 86%]
pydoctor/test/test_qnmatch.py ......                                                                                                                                                  [ 87%]
pydoctor/test/test_sphinx.py .............................                                                                                                                            [ 89%]
pydoctor/test/test_templatewriter.py .................................................                                                                                                [ 93%]
pydoctor/test/test_twisted_python_deprecate.py ........                                                                                                                               [ 93%]
pydoctor/test/test_type_fields.py ...........                                                                                                                                         [ 94%]
pydoctor/test/test_utils.py ...............                                                                                                                                           [ 95%]
pydoctor/test/test_visitor.py ..                                                                                                                                                      [ 95%]
pydoctor/test/test_zopeinterface.py ........................................................                                                                                          [100%]

========================================================================================= FAILURES ==========================================================================================
_____________________________________________________________________________ test_cyclic_imports_base_classes ______________________________________________________________________________

    def test_cyclic_imports_base_classes() -> None:
        if sys.platform == 'win32':
            # Running this script with the following subprocess call fails on Windows
            # with an ImportError that isn't actually related to what we want to test.
            # So we just skip for Windows.
            return

        process = subprocess.Popen(
            [sys.executable, os.path.basename(__file__)],
            env={'PYTHONHASHSEED': '0'},
            cwd=os.path.dirname(__file__),
        )
>       assert process.wait() == 0
E       AssertionError: assert 1 == 0
E        +  where 1 = <bound method Popen.wait of <Popen: returncode: 1 args: ['/usr/bin/python3', 'test_cyclic_imports_base_c...>>()
E        +    where <bound method Popen.wait of <Popen: returncode: 1 args: ['/usr/bin/python3', 'test_cyclic_imports_base_c...>> = <Popen: returncode: 1 args: ['/usr/bin/python3', 'test_cyclic_imports_base_c...>.wait

pydoctor/test/test_cyclic_imports_base_classes.py:22: AssertionError
----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tkloczko/rpmbuild/BUILD/pydoctor-24.3.3/pydoctor/test/test_cyclic_imports_base_classes.py", line 26, in <module>
    from test_packages import processPackage, model # type: ignore
  File "/home/tkloczko/rpmbuild/BUILD/pydoctor-24.3.3/pydoctor/test/test_packages.py", line 5, in <module>
    from pydoctor import model
ModuleNotFoundError: No module named 'pydoctor'
========================================================================================= XFAILURES =========================================================================================
_____________________________________________________________________________ test_func_arg_and_ret_annotation ______________________________________________________________________________

    @pytest.mark.xfail
    def test_func_arg_and_ret_annotation() -> None:
        annotation_mod = fromText('''
        def f(a: List[str], b: "List[str]") -> bool:
            """
            @param a: an arg, a the best of args
            @param b: a param to follow a
            @return: the best that we can do
            """
        ''')
        classic_mod = fromText('''
        def f(a, b):
            """
            @param a: an arg, a the best of args
            @type a: C{List[str]}
            @param b: a param to follow a
            @type b: C{List[str]}
            @return: the best that we can do
            @rtype: C{bool}
            """
        ''')
        annotation_fmt = docstring2html(annotation_mod.contents['f'])
        classic_fmt = docstring2html(classic_mod.contents['f'])
>       assert annotation_fmt == classic_fmt
E       assert '<div>\n<tabl...able>\n</div>' == '<div>\n<tabl...able>\n</div>'
E
E         Skipping 173 identical leading characters in diff, use -v to show
E           :</span>
E         - <tt class="rst-docutils rst-literal">List[str]</tt>
E         + <code>List[str]</code>
E           </td>
E           <td class="fieldArgDesc">an arg, a the best of args</td>...
E
E         ...Full output truncated (21 lines hidden), use '-vv' to show

pydoctor/test/test_epydoc2stan.py:283: AssertionError
______________________________________________________________________ test_func_arg_and_ret_annotation_with_override _______________________________________________________________________

    @pytest.mark.xfail
    def test_func_arg_and_ret_annotation_with_override() -> None:
        annotation_mod = fromText('''
        def f(a: List[str], b: List[str]) -> bool:
            """
            @param a: an arg, a the best of args
            @param b: a param to follow a
            @type b: C{List[awesome]}
            @return: the best that we can do
            """
        ''')
        classic_mod = fromText('''
        def f(a, b):
            """
            @param a: an arg, a the best of args
            @type a: C{List[str]}
            @param b: a param to follow a
            @type b: C{List[awesome]}
            @return: the best that we can do
            @rtype: C{bool}
            """
        ''')
        annotation_fmt = docstring2html(annotation_mod.contents['f'])
        classic_fmt = docstring2html(classic_mod.contents['f'])
>       assert annotation_fmt == classic_fmt
E       assert '<div>\n<tabl...able>\n</div>' == '<div>\n<tabl...able>\n</div>'
E
E         Skipping 173 identical leading characters in diff, use -v to show
E           :</span>
E         - <tt class="rst-docutils rst-literal">List[str]</tt>
E         + <code>List[str]</code>
E           </td>
E           <td class="fieldArgDesc">an arg, a the best of args</td>...
E
E         ...Full output truncated (20 lines hidden), use '-vv' to show

pydoctor/test/test_epydoc2stan.py:309: AssertionError
================================================================================== short test summary info ==================================================================================
SKIPPED [1] pydoctor/test/test_model.py:311: cython_test_exception_raiser not installed
XFAIL pydoctor/test/test_epydoc2stan.py::test_func_arg_and_ret_annotation
XFAIL pydoctor/test/test_epydoc2stan.py::test_func_arg_and_ret_annotation_with_override
FAILED pydoctor/test/test_cyclic_imports_base_classes.py::test_cyclic_imports_base_classes - AssertionError: assert 1 == 0
========================================================= 1 failed, 1337 passed, 1 skipped, 2 xfailed, 46 subtests passed in 22.54s =========================================================

List of installed modules in build env:

Package                       Version
----------------------------- -----------
alabaster                     0.7.16
appdirs                       1.4.4
astor                         0.8.1
attrs                         23.2.0
Automat                       22.10.0
Babel                         2.15.0
beautifulsoup4                4.12.3
build                         1.2.1
CacheControl                  0.14.0
charset-normalizer            3.3.2
ConfigArgParse                1.7
constantly                    23.10.4
defusedxml                    0.7.1
docutils                      0.20.1
exceptiongroup                1.1.3
filelock                      3.14.0
hyperlink                     21.0.0
hypothesis                    6.100.0
idna                          3.7
imagesize                     1.4.1
importlib_metadata            7.1.0
incremental                   22.10.0
iniconfig                     2.0.0
installer                     0.7.0
Jinja2                        3.1.4
lunr                          0.7.0.post1
MarkupSafe                    2.1.5
msgpack                       1.0.8
packaging                     24.0
pluggy                        1.5.0
pyenchant                     3.2.2
Pygments                      2.18.0
pyproject_hooks               1.0.0
pytest                        8.2.1
pytest-subtests               0.12.1
python-dateutil               2.9.0.post0
requests                      2.32.2
setuptools                    69.4.0
snowballstemmer               2.2.0
sortedcontainers              2.4.0
soupsieve                     2.5
Sphinx                        7.3.7
sphinx-argparse               0.4.0
sphinx_rtd_theme              2.0.0
sphinxcontrib-applehelp       1.0.8
sphinxcontrib-devhelp         1.0.6
sphinxcontrib-htmlhelp        2.0.5
sphinxcontrib-jquery          4.1
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.7
sphinxcontrib-serializinghtml 1.1.10
sphinxcontrib-spelling        8.0.0
tokenize_rt                   5.2.0
toml                          0.10.2
tomli                         2.0.1
Twisted                       24.3.0
typing_extensions             4.12.0
urllib3                       2.2.1
wheel                         0.43.0
zipp                          3.19.0
zope.event                    5.0
zope.interface                6.4.post2

Please let me know if you need more details or want me to perform some diagnostics.

[tristanlatr]

Hello @kloczek,

Please do a quick search before you open a ticket: You 'll see that we already discuss these kind of test failure in #670.
The resolution was: as long as it works on the github CI, I'm not personnally going to work on this issue.

Now, if you want to submit a patch to fix this situation in your environment, you can and I'll be glad to review your PR.

Thanks,