TypeError: changelist must be an iterable of select.kevent objects on Python 3.9/OSX

[pwinston]

I'm using eventlet 0.29.1 with Flask-SocketIO 4.3.1 and Python 3.9 on MacOS.

I tried added eventlet.monkey_patch() hoping to cure my server of hanging on exit.

My program uses SharedMemoryManager to connect to a process, kind of on the other side of things away from the SocketIO part. And monkey_patch() causes this error when I call SharedMemoryManager.connect():

TypeError: changelist must be an iterable of select.kevent objects

Traceback (most recent call last):
File "/Users/pbw/dev/webmon/webmon.py", line 225, in <module>
    main()
File "/Users/pbw/miniconda3/envs/napari39/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
File "/Users/pbw/miniconda3/envs/napari39/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
File "/Users/pbw/miniconda3/envs/napari39/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
File "/Users/pbw/miniconda3/envs/napari39/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
File "/Users/pbw/dev/webmon/webmon.py", line 211, in main
    client = create_napari_client("webmon")
File "/Users/pbw/dev/webmon/napari_client.py", line 193, in create_napari_client
    return MonitorClient(config, client_name)
File "/Users/pbw/dev/webmon/napari_client.py", line 115, in __init__
    self._manager.connect()
File "/Users/pbw/miniconda3/envs/napari39/lib/python3.9/multiprocessing/managers.py", line 522, in connect
    conn = Client(self._address, authkey=self._authkey)
File "/Users/pbw/miniconda3/envs/napari39/lib/python3.9/multiprocessing/connection.py", line 507, in Client
    c = SocketClient(address)
File "/Users/pbw/miniconda3/envs/napari39/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
    s.connect(address)
File "/Users/pbw/miniconda3/envs/napari39/lib/python3.9/site-packages/eventlet/greenio/base.py", line 250, in connect
    self._trampoline(fd, write=True)
File "/Users/pbw/miniconda3/envs/napari39/lib/python3.9/site-packages/eventlet/greenio/base.py", line 208, in _trampoline
    return trampoline(fd, read=read, write=write, timeout=timeout,
File "/Users/pbw/miniconda3/envs/napari39/lib/python3.9/site-packages/eventlet/hubs/__init__.py", line 157, in trampoline
    listener = hub.add(hub.WRITE, fileno, current.switch, current.throw, mark_as_closed)
File "/Users/pbw/miniconda3/envs/napari39/lib/python3.9/site-packages/eventlet/hubs/kqueue.py", line 53, in add
    self._control([event], 0, 0)
File "/Users/pbw/miniconda3/envs/napari39/lib/python3.9/site-packages/eventlet/hubs/kqueue.py", line 39, in _control
    return self.kqueue.control(events, max_events, timeout)
TypeError: changelist must be an iterable of select.kevent objects

[temoto]

Hey that seems like a somewhat easy bug in kqueue hub.

Please try Python3.7-3.8 if you can.

[temoto]

And of course, EVENTLET_HUB=poll environment should be a good enough workaround for now. It's unlikely you'd run this under high load on OSX, right?

[pwinston]

I can't easily run older Python's right now, due to other dependencies unrelated to eventlet.

Setting EVENTLET_HUB=poll does fix the error!

Also just removing monkey_patch() fixes it. I thought I needed monkey_patch() to fix a hang on exit, but I fixed that another way.

So I can run two ways under Python 3.9:

1. Without monkey_patch()
2. With monkey_patch() but use EVENTLET_HUB=poll.

I'm totally new to eventlet. We are writing a Flask-SocketIO server that's updating at 60Hz, but should generally have just 1 connected user!

[temoto]

@pwinston IMHO Python threads are decent solution for under 100 concurrent connections. Maybe you don't need Eventlet after all.

Still there is a bug in kqueue hub. Could you provide a minimal issue reproduction script?

[pwinston]

This reproduces the problem:

import eventlet

eventlet.monkey_patch()
from multiprocessing.managers import BaseManager

manager = BaseManager(address=('localhost', 5010), authkey=str.encode('XXX'))
manager.connect()

Yeah we'll probably 1 concurrent connection!

Flask-SocketIO does not give the best explanation of the options, or the trade-offs. It says use eventlet, or gevent, or "The Flask development server based on Werkzeug" but it poo-poohs that option.

So I don't know if it has a "threads" option, maybe that's the Flask development server?

https://flask-socketio.readthedocs.io/en/latest/

[temoto]

Thanks a lot. So the bug is indeed Python 3.9 specific, this code works on 3.8. Shorter repro script:

import eventlet
eventlet.connect(('127.0.0.1', 1234))

Flask-SocketIO doc says dev server can't talk websocket, only long polling, which is really unfortunate. Before abandoning websocket option, maybe you should check that all different features of your browser-backend interaction work fine and under degraded network too. I know very little about SocketIO to suggest in which situations websocket transport is clearly superior to long polling.

[temoto]

Other kevent platforms like FreeBSD should suffer from this issue, in theory.