The cherrypy.engine can't handle DummyThreads on Python 3.4

[ghost]

Originally reported by: Joel Rivera (Bitbucket: cyraxjoe, GitHub: cyraxjoe)

---

I'm making a plugin to integrate Winpdb to CherryPy and I was having an issue with threads half-stopped on the block method of the cherrypy.engine (wspb) and it turns out that the problem is with a fix on cpython which set the _stop method body as pass and therefore never executing the expected method of the threading.Thread.

In particular this method sets the property: _is_stopped to True and because of that, the method threading.Thread.isAlive is unusable on the _DummyThread instances on Python 3.4. The problem was introduced after fixing the Issue 18808 of cpython the added a call to a new method _wait_for_tstate_loc that makes an assertion to the _is_stopped property and therefore ending with an AssertionError when the cherrypy.engine.block method reach the isAlive call.

After the isAlive call on the block method there is a comment which states:

 Note that any dummy (external) threads are always daemonic.

And indeed is true, but the AssertionErrorgets raised before reaching that line on Python 3.4.

This is an example of the resulting traceback:

#!python
[03/Oct/2014:20:22:47] ENGINE Waiting for child threads to terminate...
Traceback (most recent call last):
  File "/usr/lib64/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/joe/repos/arrive/arrive/devserv/__main__.py", line 102, in <module>
  File "/home/joe/repos/arrive/arrive/devserv/__main__.py", line 98, in main

  File "/home/joe/repos/arrive/arrive/devserv/__main__.py", line 82, in run_with_debugger
    cherrypy.engine.block()
  File "/home/joe/env/arrive/lib/python3.4/site-packages/cherrypy/process/wspbus.py", line 335, in block
    t.isAlive() and
  File "/usr/lib64/python3.4/threading.py", line 1119, in is_alive
    self._wait_for_tstate_lock(False)
  File "/usr/lib64/python3.4/threading.py", line 1075, in _wait_for_tstate_lock
    assert self._is_stopped
AssertionError

I suggest that we can change the isinstance validation of the _MainThread to:

  isinstance(t, (threading._MainThread, threading._DummyThread))

And set it before the isAlive call on the if conditional expression:

#!python
if (
      t != threading.currentThread() and
      not isinstance(t, (threading._MainThread, threading._DummyThread)) and
      t.isAlive()
   ):

This is valid from python 2.4 up to 3.4.

I can make this change, but I'm looking on a second opinion on this issue.

---

• Bitbucket: https://bitbucket.org/cherrypy/cherrypy/issue/1342

[ghost]

Original comment by Sylvain Hellegouarch (Bitbucket: Lawouach, GitHub: Lawouach):

---

I'm okay with the swap in the conditionnal but I'm not clear as to why we need to check if it is a MainThread instance.

[ghost]

Original comment by Joel Rivera (Bitbucket: cyraxjoe, GitHub: cyraxjoe):

---

The MainThread can be a blocker when cherrypy is deployed as a windows service. Check the Issue #1164.

[ghost]

Original comment by Lindsay Stevens (Bitbucket: lindsay-stevens-kirby, GitHub: @lindsay-stevens):

---

I have come across the same kind of problem, but have a different solution.

I am using Windows 7 x64 with Python 3.5.1, Cherrypy 4.0.0. The same issue will occur whether cherrypy is run from the console and inside of a service.

In a clean virtualenv, the problem does not appear until pywin32 is installed. It happens with both the Pypi "pypiwin32-219" wheel, and the "pywin32-220" wheel from http://www.lfd.uci.edu/~gohlke/pythonlibs.

Steps to reproduce:

• create a python virtualenv
• activate the virtualenv
• pip install cherrypy
• pip install pypiwin32 or download and install the pythonlibs pywin32-220-cp35-none-win_amd64.whl
• start a server using quickstart
• send a keyboard interrupt (Ctrl+C)
• observe crash & Traceback as above.

These are the commands to set it up, if your Windows is rusty.

C:\Python35\python -m venv testenv
call testenv\Scripts\activate.bat
pip install cherrypy
pip install pypiwin32
python -c "import cherrypy; cherrypy.quickstart()"
Ctrl+C

If it doesn't happen the first time, start the server then try kicking off some threads by opening a browser and hitting 127.0.0.1:8080 a few times.

The problem goes away if the t.isAlive() check mentioned in the Traceback is removed, as follows:

cherrypy/process/wspbus.py line 332 in block()
325        for t in threading.enumerate():
326            # Validate the we're not trying to join the MainThread
327            # that will cause a deadlock and the case exist when
328            # implemented as a windows service and in any other case
329            # that another thread executes cherrypy.engine.exit()
330            if (
331                    t != threading.currentThread() and
332 -                  t.isAlive() and
333                    not isinstance(t, threading._MainThread)
334            ):

This check to see if the thread is alive is not needed anyway, since from Python 2.6 to 3.5 [1], [2], the docs note that "The module function enumerate() returns a list of all alive threads". The objects being evaluated are from the enumerate call on line 325.

After making this change, the same sequence of steps results in successful shutdown, with the final log message "ENGINE Bus EXITED".

[1] https://docs.python.org/2.6/library/threading.html#threading.Thread.is_alive

[2] https://docs.python.org/3.5/library/threading.html#threading.Thread.is_alive

[webknjaz]

Btw, @cyraxjoe did you finish your pdb plugin?

[webknjaz]

@lindsay-stevens thanks for your investigation, it has helped to solve the original issue recently :)