-
Notifications
You must be signed in to change notification settings - Fork 839
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
write starvation fixes #545
Commits on Apr 11, 2015
-
A fix the write starvation problem that we see with tornado and pika
By default (in base_connection) Pika buffers all writes and only attempts to send the next time it drops into the ioloop and detects the socket as writable. This causes some odd behaviour in a number of cases. 1. A process generating large numbers of messages will not actually send them until it finishes the processing and drops into the ioloop. 2. A process that is consuming a large queue will only send messages when it's read buffer is empty. If the messages are small this means it may end up consuming 000s of messages for every one it manages to publish. This behaviour stalls pipelines of processes. This patch tries to send the data on the socket as soon as it's generated and to avoid timeouts that might be generated by the use of sendall() it uses send() and handles partial writes by requeing the data.
Will Slater authored and Will Slater committedApr 11, 2015 Configuration menu - View commit details
-
Copy full SHA for 1f22edc - Browse repository at this point
Copy the full SHA 1f22edcView commit details -
Use non-blocking socket in Tornado
Will Slater authored and Will Slater committedApr 11, 2015 Configuration menu - View commit details
-
Copy full SHA for 69d17df - Browse repository at this point
Copy the full SHA 69d17dfView commit details -
Make _handle_write fixes to base_connection
Move the _handle_write changes from tornado_connection to base_connection. They're equally applicable to all connections and this fixes a bug in master where BlockingConnections wouldn't send messages until close was called.
Will Slater authored and Will Slater committedApr 11, 2015 Configuration menu - View commit details
-
Copy full SHA for a11ce5f - Browse repository at this point
Copy the full SHA a11ce5fView commit details
Commits on Apr 12, 2015
-
Make _handle_write fixes to base_connection
Move the _handle_write changes from tornado_connection to base_connection. They're equally applicable to all connections and this fixes a bug in master where BlockingConnections wouldn't send messages until close was called.
Will Slater authored and Will Slater committedApr 12, 2015 Configuration menu - View commit details
-
Copy full SHA for 576c1f0 - Browse repository at this point
Copy the full SHA 576c1f0View commit details
Commits on Apr 13, 2015
-
Merge branch 'master' into fix_write_handling
Will Slater authored and Will Slater committedApr 13, 2015 Configuration menu - View commit details
-
Copy full SHA for cca26df - Browse repository at this point
Copy the full SHA cca26dfView commit details -
Various cleanups as suggested by @vitaly-krugl
Will Slater authored and Will Slater committedApr 13, 2015 Configuration menu - View commit details
-
Copy full SHA for f2dc430 - Browse repository at this point
Copy the full SHA f2dc430View commit details -
remove unused import, dodgy error handling
Will Slater authored and Will Slater committedApr 13, 2015 Configuration menu - View commit details
-
Copy full SHA for 7b474bb - Browse repository at this point
Copy the full SHA 7b474bbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7d457f5 - Browse repository at this point
Copy the full SHA 7d457f5View commit details
Commits on Apr 14, 2015
-
Configuration menu - View commit details
-
Copy full SHA for 635957f - Browse repository at this point
Copy the full SHA 635957fView commit details