-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DEALER socket losing messages #3
Conversation
…at the underlying issue is.
Hi, I encounter same problem, I notice that is caused by that the send operation is complete before we goes into reactor.run, namely, the select. For example:
It appears the fd will be notified only when the send operation complete, just the moment. Then it sets POLLIN event in socket. Therefore, reactor.run gets no chance to get the read event from fd.
To solve the problem, I think you can add def doRead() in send method
It works fine for me, but I'm not sure is that behavior of zeromq correct. |
A workaround
example usage:
|
Code for reproducing bug |
The first workaround proposed by @victorlin worked. I preferred to avoid the second one as it uses threads. |
ZMQ is not threadsafe, so we need to work with its socket in same thread where it was created. As a result:
|
I don't understand what you mean. As I know, reactor.callFromThread is same to callLater(0, ...). Let's say, reactor.callFromThread add the function call to reactor, so it will get called at next iteration. Likewise, callLater(0, ...) create a function call at 0 second later, it will be triggered at next iteration as well. They are all in reactor.run (main thread), what's different? Stack overflow? |
callFromThread - "Use this method when you want to run a function in the reactor's thread from another thread." |
I had: reactor.callLater(0, self.doRead) but started running into trouble when calling
The workaround/fix is: reactor.callLater(0, lambda: self.doRead() if self.socket else None) I will make this fix available in my fork of However, does anybody know if it would be cleaner or more efficient to store the |
Currently testing code with the fix... I'll try to make some implementation with compromise on performance overhead to call doRead on each send operation. Thanks for the reports! |
For some reason when storing the reactor.callLater(0, lambda: self.doRead() if self.socket else None) This solution is also much simpler—the hack-fix is contained in only one place/line of code. |
Thanks to everybody who submitted patches, testcases and filed bugs! I've comitted "fix" based on your ideas and suggestions. Could you please give current code a try? I will be testing it in my environment for some time and then push new version to PyPi if nothing scary comes out. |
The fix "works for me", so closing this pull request, as there were no comments. I'll release new version to PyPi ASAP. |
yes, the fix has seemed to work for us so far as well! |
Hi all,
I am seeing strange behavior in an application using DEALER and ROUTER sockets.
Requests made by the DEALER are always received by the ROUTER, but the responses made by the ROUTER are sometimes lost. I've added a test to demonstrate this behavior. It is the only change in this pull request.
https://github.com/wehriam/txZMQ/blob/master/txZMQ/test/test_async_dealer_router.py
In the application (but not demonstrated in the test) after some varying number of requests responses eventually fail to arrive entirely.
I was unsuccessful in resolving this issue and hope you'll have better luck in determining the underlying problem.
OSX Lion,
Python 2.7.1
ZMQ 2.1.9
Twisted==11.0.0
pep8==0.6.1
pyflakes==0.5.0
pyzmq==2.1.10
txZMQ==0.3
wsgiref==0.1.2
zope.interface==3.8.0