graceful shutdown for tornado is not thread-safe. #1032

Closed
ccl0326 opened this Issue May 19, 2015 · 1 comment

Projects

None yet

2 participants

@ccl0326
ccl0326 commented May 19, 2015

ERROR:tornado.application:Exception in callback None
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/tornado/ioloop.py", line 840, in start
handler_func(fd_obj, events)
File "/Library/Python/2.7/site-packages/tornado/stack_context.py", line 275, in null_wrapper
return fn(_args, *_kwargs)
File "/Library/Python/2.7/site-packages/tornado/netutil.py", line 211, in accept_handler
connection, address = sock.accept()
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)

File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
File "/Library/Python/2.7/site-packages/gunicorn-19.3.0-py2.7.egg/gunicorn/sock.py", line 37, in getattr
return getattr(self.sock, name)
RuntimeError: maximum recursion depth exceeded

The testing script:

import os
import requests
import signal
import threading
import time

from optparse import OptionParser

class ReqThread(threading.Thread):
def run(self):
while True:
r = requests.get('http://127.0.0.1:8000')
print r.status_code

if name == 'main':
parser = OptionParser()
parser.add_option("-p", "--pid", dest="pid", help="")
(options, args) = parser.parse_args()
os.kill(int(options.pid), signal.SIGTERM)
for x in xrange(0, 10000):
thread = ReqThread()
thread.start()
time.sleep(10)

The implementation for graceful shutdown of tornado is not thread-safe.
PeriodicCallback and server.stop are both not thread-safe when ioloop is running.

@berkerpeksag
Collaborator

Fixed by d9b8959.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment