v3.1.2 - test suite "frozen" after it finishes

[sandrotosi]

Hello,
i'm preparing 3.1.2 for upload to Debian but i noticed something unusual: the test suite just stops when it finishes, and that blocks the execution of further build steps for Debian.

we run the tests like:

/usr/bin/xvfb-run -a python3.7 tests.py --no-network --verbose --full-trace

(exporting PYTHONPATH and MATPLOTLIBDATA accordingly)

the suite runs:

...
build/lib.linux-x86_64-3.7/matplotlib/tests/test_tightlayout.py::test_collapsed
  /usr/lib/python3/dist-packages/_pytest/python.py:174: UserWarning: Tight layout not applied. The left and right margins cannot be made large enough to accommodate all axes decorations. 
    testfunction(**testargs)

build/lib.linux-x86_64-3.7/matplotlib/tests/test_ttconv.py::test_truetype_conversion[pdf]
  /home/morph/deb/build-area/matplotlib-3.1.2/build/lib.linux-x86_64-3.7/matplotlib/backends/_backend_pdf_ps.py:62: RuntimeWarning: Glyph 108 missing from current font.
    font.set_text(s, 0.0, flags=ft2font.LOAD_NO_HINTING)

build/lib.linux-x86_64-3.7/matplotlib/tests/test_ttconv.py::test_truetype_conversion[pdf]
  /home/morph/deb/build-area/matplotlib-3.1.2/build/lib.linux-x86_64-3.7/matplotlib/backends/_backend_pdf_ps.py:62: RuntimeWarning: Glyph 112 missing from current font.
    font.set_text(s, 0.0, flags=ft2font.LOAD_NO_HINTING)

-- Docs: https://docs.pytest.org/en/latest/warnings.html
 12 failed, 7823 passed, 38 skipped, 2 deselected, 10 xfailed, 13 xpassed, 39 warnings, 14 error in 388.60 seconds 

and then it just sits there forever.

is this a known issue? any idea how to debug/fix it? thanks!

[tacaswell]

I have seen this occasionally locally but have not been able to track it down

[sandrotosi]

i can reproduce it every time i run the test suite on my development box and in a clean chroot (where we build the packages for upload). let me know what i can do to help triage this

[tacaswell]

That is both reassuring and disappointing.

My knee-jerk reaction is that it is one of the tests where we spawn a subprocess that does not close down cleanly. Can you look at the running processes when it is hung?

[sandrotosi]

this is the pstree for the test suite now in the "frozen" phase:

xvfb-run /usr/bin/xvfb-run -a python3.7 tests.py --no-network --verbose --full-trace
  ├─Xvfb :99 -screen 0 1280x1024x24 -nolisten tcp -auth /tmp/xvfb-run.cO29aT/Xauthority
  │   └─8*[{Xvfb}]
  └─python3.7 tests.py --no-network --verbose --full-trace
      ├─gs -dNOPAUSE -sDEVICE=png16m
      ├─inkscape --without-gui --shell
      │   └─7*[{inkscape}]
      ├─python3.7 tests.py --no-network --verbose --full-trace
      ├─python3.7 tests.py --no-network --verbose --full-trace
      ├─xelatex -halt-on-error
      └─11*[{python3.7}]

so i guess that gs and inkscape are the reasons?

[sandrotosi]

any idea what to look next? we cant release the new package to debian until we fix this, and the pending update contains some important fixes people are waiting for.

the alternative is to skip entirely (and temporarily) the test suite, but it's really a last resort option

thanks :)

[timhoffm]

I just had the same problem in a plain pytest run. Ctrl+C:

^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/home/tim/dev/matplotlib/lib/matplotlib/backends/backend_pgf.py", line 306, in _cleanup
    self.latex.communicate()
  File "/home/tim/anaconda3/envs/mpl-old/lib/python3.7/subprocess.py", line 939, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/home/tim/anaconda3/envs/mpl-old/lib/python3.7/subprocess.py", line 1681, in _communicate
    ready = selector.select(timeout)
  File "/home/tim/anaconda3/envs/mpl-old/lib/python3.7/selectors.py", line 415, in select
    fd_event_list = self._selector.poll(timeout)
KeyboardInterrupt

and still hanging, so another Ctrl+C:

^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/home/tim/anaconda3/envs/mpl-old/lib/python3.7/multiprocessing/util.py", line 265, in _run_finalizers
    finalizer()
  File "/home/tim/anaconda3/envs/mpl-old/lib/python3.7/multiprocessing/util.py", line 189, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/home/tim/anaconda3/envs/mpl-old/lib/python3.7/multiprocessing/pool.py", line 581, in _terminate_pool
    cls._help_stuff_finish(inqueue, task_handler, len(pool))
  File "/home/tim/anaconda3/envs/mpl-old/lib/python3.7/multiprocessing/pool.py", line 566, in _help_stuff_finish
    inqueue._rlock.acquire()
KeyboardInterrupt

[Kojoley]

Maybe #16126 is the cause?

[timhoffm]

I didn't experience the freeze recently, so assuming fixed by #16126.
Please reopen if the issue is still present.