OpenSSL sockets spin indefinitely on timeout in handshake #549

Closed
robinmessage opened this Issue Feb 25, 2013 · 1 comment

Comments

Projects
None yet
2 participants
@robinmessage

OpenSSL sockets spin in a variety of ways (generally in an epoll wrapper) if there is a timeout during the handshake (might be other causes too but that seemed to cause the problem reliably.)

The culprit seems to be line 416 of SSLSocket.java, which is the only place the reads the return value of flushData. flushData returns true if there is no data left in the buffer - however, in the default case of an empty buffer, as far as I can see, this is going to cause an infinite loop.

I patched this on our production system by reversing the return value from flushData - so it returns true if there is still data in the buffer. This completely fixed our problem.

I suggest just swapping the return values from flushData, but not sure if that fits the bigger picture correctly.

diff --git a/src/org/jruby/ext/openssl/SSLSocket.java b/src/org/jruby/ext/openssl/SSLSocket.java
index b692528..579ee5c 100644
--- a/src/org/jruby/ext/openssl/SSLSocket.java
+++ b/src/org/jruby/ext/openssl/SSLSocket.java
@@ -448,9 +448,9 @@ public class SSLSocket extends RubyObject {
             throw ioe;
         }
         if (netData.hasRemaining()) {
-            return false;
-        }  else {
             return true;
+        }  else {
+            return false;
         }
     }

-- 
@BanzaiMan

This comment has been minimized.

Show comment
Hide comment
@BanzaiMan

BanzaiMan Mar 15, 2013

Member

I'm closing this one in favor of #554.

Member

BanzaiMan commented Mar 15, 2013

I'm closing this one in favor of #554.

@BanzaiMan BanzaiMan closed this Mar 15, 2013

headius added a commit that referenced this issue Mar 16, 2013

Merge pull request #554 from robinmessage/master
Fixes issue #549 (OpenSSL sockets spin indefinitely on timeout in handshake)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment