Incompatible with gevent monkey patch after pyzmq>=14.0.0

[faith0811]

I found this problem when I have trouble interacting with zerorpc, and I submit a issue in zerorpc before. Link: 0rpc/zerorpc-python#123

When gevent monkey patch is patched with pyzmq version >= 14.0.0, then the process get frozen after Socket.send is called(in the zerorpc case, the greenlet switched and never switched back).

And recently, I found pyzmq bumps its version to 15.1.0, so I tried again, and the problem is still there.

So I take a look into zerorpc source code, and I found out that zerorpc takes a non-copy send/recv which triggers the issue. Here is a small demo: https://gist.github.com/faith0811/34ae622abf2560820ab7

The copy version works well, but the non-copy version is broken when monkey patch is applied.

With some digging, I found that the garbage-collector thread caused this problem. Here is the commit: 47f7684

I think, the nogil cython code calling monkey patched thread(which is greenlet) might breaks the event loop(not sure but likely).

After I reverted this commit, the code works well, but I cannot sure there is no other side affects.

thanks

[minrk]

Yikes. It really is awful that gevent monkeypatches things.

[faith0811]

Monkey patch is evil, but necessary in many cases.
Is it possible to do garbage collect using native c code without importing the python threading lib?

[minrk]

The issue seems to be that gevent patches Thread, so the garbage collection thread is green, but the Context (and therefore sockets) in the thread are not. Telling the gc thread to use a green Context seems to work:

# patch zmq garbage-collection Thread to use green Context:
from zmq.utils.garbage import gc
gc.context = zmq.Context()

With this, your script works.

[minrk]

#766 should be enough to fix this.

[faith0811]

well nice work!!!

[lordnynex]

@minrk Your fix, has brought a week of frustrated debugging to an end. Thank you!