How to terminate a disconnected client? #301

christophevg · 2019-05-27T09:57:36Z

Hi,

I'm having trouble cleanly terminating a disconnected client using an interrupt (ctrl+c).
Given this example:

import socketio
import time

sio = socketio.Client()

@sio.on("connect")
def on_connect():
  print("connected")

@sio.on("disconnect")
def on_disconnect():
  print("disconnected")

sio.connect("http://localhost:8000")

try:
  # sio.wait() # doesn't raise it, so have to implement it manually ;-)
  while True:
    time.sleep(1)
except KeyboardInterrupt:
  print("handling interrupt...")
  sio.disconnect()

print("done")

All goes well when issuing the interrupt when connected:

$ python client.py
connected
^Cdisconnected
handling interrupt...
done
$

But when performing the same procedure, after the client was disconnected from the server (by the server), the same interrupt handling code isn't able to terminate the client cleanly, and the process continues, not returning to the console:

$ python client.py
connected

# stopping server now, after initial connected

disconnected
^Chandling interrupt...
done

The only way to terminate the process now is ...

^Z
[1]+  Stopped                 python client.py
$ kill %1

[1]+  Terminated: 15          python client.py

Is there a proper way to terminate a client, when it's in a disconnected state?

regards,
Christophe VG

The text was updated successfully, but these errors were encountered:

miguelgrinberg · 2019-05-27T11:34:44Z

Just call wait():

sio.connect("http://localhost:8000")
sio.wait()

print("done")

A keyboard interrupt exception will automatically trigger a disconnect, and then sio.wait() will return control to your application.

christophevg · 2019-05-27T11:54:42Z

That was wait I tried first (see comment in the example code ;-))

But when issuing a keyboard interrupt, it just "waits".

So given the minimal example:

import socketio

sio = socketio.Client()

@sio.on("connect")
def on_connect():
  print("connected")

@sio.on("disconnect")
def on_disconnect():
  print("disconnected")

sio.connect("http://localhost:8000")
sio.wait()
print("done")

Interrupting the application results in no done and no termination of the application:

$ python client.py 
connected
^C

👆no done and not terminated

Replacing sio.wait() by the sleep-loop and exception handler, allows for catching the exception, issuing a disconnect() and termination of the application, but it only works when connected! If the client gets in a disconnected state (e.g. after terminating the server), it still hangs after printing "done".

miguelgrinberg · 2019-05-27T12:49:55Z

This is what I get here:

(venv) $ python client.py
connected
^Cdisconnected
Traceback (most recent call last):
  File "client.py", line 14, in <module>
    sio.wait()
...
KeyboardInterrupt

So you were partially right in that the print never runs. I forgot that I let the keyboard interrupt exception bubble up after I do my own handling.

christophevg · 2019-05-27T13:35:20Z

Okay 🤔 how do I go about debugging this further? Since I don't get the KeyboardInterrupt Exception from the sio.wait() call, I'd like to help in finding the problem and maybe solving it.

These are the most condensed steps to reproduce the problem, maybe you can spot a "difference" that might hint at a possible problem on my side (note: do read to the bottom to see what I discovered while going through this in more detail 😉):

$ virtualenv venv
New python executable in /Users/xtof/Workspace/socketio-bug/venv/bin/python
Installing setuptools, pip, wheel...
done.

$ . venv/bin/activate

(venv) $ pip install python-socketio
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Collecting python-socketio
  Using cached https://files.pythonhosted.org/packages/ee/88/f38a71f1a99603ef988376aa73a48b453cddc009fa31fe612260cae75827/python_socketio-4.0.3-py2.py3-none-any.whl
Collecting six>=1.9.0 (from python-socketio)
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting python-engineio>=3.2.0 (from python-socketio)
  Using cached https://files.pythonhosted.org/packages/91/e1/8d600ac8003254bf6b9074ed06a134f6018357a964e034bfd06aae14db82/python_engineio-3.6.0-py2.py3-none-any.whl
Installing collected packages: six, python-engineio, python-socketio
Successfully installed python-engineio-3.6.0 python-socketio-4.0.3 six-1.12.0

(venv) $ pip install requests
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Collecting requests
  Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests)
  Using cached https://files.pythonhosted.org/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests)
  Using cached https://files.pythonhosted.org/packages/60/75/f692a584e85b7eaba0e03827b3d51f45f571c2e793dd731e598828d380aa/certifi-2019.3.9-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5 (from requests)
  Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Installing collected packages: urllib3, certifi, chardet, idna, requests
Successfully installed certifi-2019.3.9 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.3

(venv) $ vi client.py

(venv) $ python client.py 
connected
^C

👆 doesn't terminate

BUT ...

If I run it in a python 3 environment, ...

(venv3) $ python client.py 
connected
^Cdisconnected
Traceback (most recent call last):
  File "client.py", line 14, in <module>
    sio.wait()
  File "/Users/xtof/Workspace/socketio-bug/venv3/lib/python3.7/site-packages/socketio/client.py", line 219, in wait
    self.eio.wait()
  File "/Users/xtof/Workspace/socketio-bug/venv3/lib/python3.7/site-packages/engineio/client.py", line 175, in wait
    self.read_loop_task.join()
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 1032, in join
    self._wait_for_tstate_lock()
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 1048, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
  File "/Users/xtof/Workspace/socketio-bug/venv3/lib/python3.7/site-packages/engineio/client.py", line 41, in signal_handler
    return original_signal_handler(sig, frame)
KeyboardInterrupt
(venv3) $

👆tadaa...

So it seems that some python 2.7 implementation of some module somewhere causes the interrupt not to bubble up properly 😕

BUT ...

Again, when I first terminate the server, and am left with a disconnected client, interrupting it again doesn't terminate the application:

$ python client.py 
connected
disconnected
^CTraceback (most recent call last):
  File "client.py", line 14, in <module>
    sio.wait()
  File "/Users/xtof/Workspace/socketio-bug/venv3/lib/python3.7/site-packages/socketio/client.py", line 223, in wait
    self._reconnect_task.join()
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 1032, in join
    self._wait_for_tstate_lock()
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 1048, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
  File "/Users/xtof/Workspace/socketio-bug/venv3/lib/python3.7/site-packages/engineio/client.py", line 41, in signal_handler
    return original_signal_handler(sig, frame)
KeyboardInterrupt

👆not terminated 😔

BUT ...

when I issue a second keyboard interrupt now:

^CException ignored in: <module 'threading' from '/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py'>
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 1273, in _shutdown
    t.join()
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 1032, in join
    self._wait_for_tstate_lock()
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 1048, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
  File "/Users/xtof/Workspace/socketio-bug/venv3/lib/python3.7/site-packages/engineio/client.py", line 41, in signal_handler
    return original_signal_handler(sig, frame)
KeyboardInterrupt
(venv3) $

👆it finishes after all.

The traces differ (of course), so maybe you can spot a difference there.

In summary:

using python 2.7, interrupting sio.wait() doesn't terminate at all.
using python 3.7, interrupting sio.wait() while connected, correctly bubbles up the interrupt and terminates the app
using python 3.7, interrupting sio.wait() while disconnected, bubbles up the interrupt, but doesn't terminate the app, but allows for terminating it with a consecutive interrupt, somehow releasing an ignored interrupt call.

miguelgrinberg · 2019-05-27T13:59:23Z

Okay, yes, I was also using 3.7 here. Thanks for the detailed report.

I suspect the case where you Ctrl-C while the server is offline is in fact a bug, because at that point the client is in a reconnect loop waiting for the server to come back. I need to look into that particular case, I'm guessing the reconnect loop, which runs in a separate thread needs to be signaled to exit when the client is interrupted.

As far as the Python 2.7 problem I'll have to debug that as well, but at this point it is low priority since I'm not planning to support this version for long.

christophevg · 2019-05-27T18:52:42Z

I suspect the same ;-)

And for the python 2.7 problem; that is indeed not worth it anymore. Maybe something for a lazy Sunday.
Gotta finish something this week, but maybe afterwards I'll take a look and see if I can cook up a PR.

miguelgrinberg · 2019-05-27T18:54:49Z

@chrisdoehring Sounds good. Thank you, if you have any questions let me know!

cschar · 2019-06-03T02:43:03Z

I tried the above solution,

try:
  # sio.wait() # doesn't raise it, so have to implement it manually ;-)
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        print("handling interrupt...")
        sio.disconnect()

    print("done")

it still hung at 'done'

and resulted with the following error, not sure if its helpufl or not:

....
done
Exception ignored in: <module 'threading' from 'C:\\Users\\zoner\\AppData\\Loc
al\\conda\\conda\\envs\\tensorflow-gpu\\lib\\threading.py'>
Traceback (most recent call last):
  File "C:\Users\zoner\AppData\Local\conda\conda\envs\tensorflow-gpu\lib\threa
ding.py", line 1294, in _shutdown
    t.join()
  File "C:\Users\zoner\AppData\Local\conda\conda\envs\tensorflow-gpu\lib\threa
ding.py", line 1056, in join
    self._wait_for_tstate_lock()
  File "C:\Users\zoner\AppData\Local\conda\conda\envs\tensorflow-gpu\lib\threa
ding.py", line 1072, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
  File "C:\Users\zoner\AppData\Local\conda\conda\envs\tensorflow-gpu\lib\site-
packages\engineio\client.py", line 41, in signal_handler
    return original_signal_handler(sig, frame)
KeyboardInterrupt

I switched the code to use asyncio and the ctrl+c started working correctly.

Using windows 10, git-bash shell

fwjavox · 2019-09-17T10:28:40Z

When using threading (instead of asyncio) the problem still exists for me.

Exception ignored in: <module 'threading' from '/usr/lib/python3.7/threading.py'>
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 1308, in _shutdown
    lock.acquire()
  File "/usr/lib/python3.7/site-packages/engineio/client.py", line 41, in signal_handler
    return original_signal_handler(sig, frame)
KeyboardInterrupt

It hangs until I press Ctrl-C, which is a problem because I use it in automated tests (pytest).

mark-zarandi · 2019-12-30T02:32:16Z

If anyone finds this bug while diagnosing socketio and threading, I was able to fix this hanging issue by setting my thread Daemon value to TRUE. https://docs.python.org/3/library/threading.html#threading.Thread.daemon

miguelgrinberg · 2019-12-30T07:14:11Z

@mark-zarandi the correct solution in this case would be for you to handle the SIGINT signal, and when your handler is invoked tell your thread to gracefully exit. Finally you should invoke the original SIGINT handler for all other exit actions to be performed. Setting the thread to daemon works, but is not a great solution, as it prevents any clean up code from executing during exit.

mark-zarandi · 2020-01-07T00:20:58Z

Mr. Miguel - Thanks for that reply. I'm just good enough at Python to be dangerous, I'll review this later and see if I get what you mean.

miguelgrinberg added the question label May 27, 2019

miguelgrinberg added bug and removed question labels May 27, 2019

miguelgrinberg self-assigned this May 27, 2019

miguelgrinberg closed this as completed in fa53e38 Jun 29, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to terminate a disconnected client? #301

How to terminate a disconnected client? #301

christophevg commented May 27, 2019

miguelgrinberg commented May 27, 2019 •

edited

christophevg commented May 27, 2019 •

edited

miguelgrinberg commented May 27, 2019 •

edited

christophevg commented May 27, 2019 •

edited

miguelgrinberg commented May 27, 2019

christophevg commented May 27, 2019

miguelgrinberg commented May 27, 2019

cschar commented Jun 3, 2019

fwjavox commented Sep 17, 2019

mark-zarandi commented Dec 30, 2019

miguelgrinberg commented Dec 30, 2019

mark-zarandi commented Jan 7, 2020

How to terminate a disconnected client? #301

How to terminate a disconnected client? #301

Comments

christophevg commented May 27, 2019

miguelgrinberg commented May 27, 2019 • edited

christophevg commented May 27, 2019 • edited

miguelgrinberg commented May 27, 2019 • edited

christophevg commented May 27, 2019 • edited

miguelgrinberg commented May 27, 2019

christophevg commented May 27, 2019

miguelgrinberg commented May 27, 2019

cschar commented Jun 3, 2019

fwjavox commented Sep 17, 2019

mark-zarandi commented Dec 30, 2019

miguelgrinberg commented Dec 30, 2019

mark-zarandi commented Jan 7, 2020

miguelgrinberg commented May 27, 2019 •

edited

christophevg commented May 27, 2019 •

edited

miguelgrinberg commented May 27, 2019 •

edited

christophevg commented May 27, 2019 •

edited