-
Notifications
You must be signed in to change notification settings - Fork 378
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Server response periodically blocks? #136
Comments
What's your |
Hello, This is using 15.2.0. If I downgrade to 13.1.0, I can not perform an equivalent test because it does not appear to have the XREP socket type. Any other ideas? |
I reworked this a bit and downgraded to 13.1.0, and you are correct, this works as expected now. This should be considered a duplicate of #123 |
Seems to be another case of monkey patching gevent everywhere messing up with pyzmq then. I have no easy solution in mind. |
But why do you need to monkey patch in the first place? You are basically begging for trouble. Monkey patching is really crazy and changing the behavior of some important functions. Either you write your code fully gevent compliant or not. Of course, zerorpc-python requires gevent, and there is no version of zerorpc working without gevent. Could a version independent of gevent fixes your problems? |
Thanks to @faith0811 see zeromq/pyzmq#766. In short: pyzmq has a gc thread to handle the lifetime of zerocopy messages. This thread uses a zmq context. When you gevent monkey patch the threading API, the gc thread is not using the green wrapper of the zmq context (zmq.green.Context). The gc thead hangs. The latest version of pyzmq does detect if the threading API was monkey patched, and uses zmq.green.Context in this case. This likely means you have to monkey patch the threading API early on, before pyzmq starts the gc thread. |
@bombela I think (at least in my case) it is not enough for zerorpc.Context to use zmq.green. I experimented with this a bit over the weekend, and in my case it did not change the problem at all. The issue @faith0811 points out (thank you!) works reliably. This explains the lockup I was seeing when using threads. Unfortunately, I think it is necessary to monkey patch my framework early on because it is simply providing multi processing wrappers to an existing single threaded code base. A lot of libs will be included indirectly. |
It has nothing to do with zerorpc here, zerorpc is based on gevent and thus already works with it. You would encounter the same freeze without zerorpc. It happenned when you combine gevent monkey patching of the threading API + using zerocopy messages with pyzmq. |
And yes @faith0811 provided the links to the real problem and solution, I was merely reporting it in this thread for closing it :) |
You are correct that it is not a zerorpc issue, however, zerorpc could proactively fix this in zmq.Context subclass by providing an initializer. Sorry I should have explained better, I haven't been sleeping much. |
Ok, I am not sure to follow 100%, do you mean providing something like zerorpc.green.Context similarly to zmq.green.Context? Or you mean providing a way to initialize a zerorpc.Context ( zerorpc-python/zerorpc/context.py Line 32 in 82dcbc2
I could subclass zmq.green.Context maybe? But I am still failing to understand what it would fix. |
Hello,
I will provide a reproduction example, but first, I am aware that what I'm doing isn't normal. I believe technically, this should be able to work.
It seems that sending return values is periodically blocking indefinitely. I'm not sure why this is occuring.
If you run the below example and run
You'll notice that ever 5th or so request results in a timeout. But from the debug output in the server, you can see that the rpc method is executed and only appears to block while sending the response to the broker.
Is there a better way to do this?
The text was updated successfully, but these errors were encountered: