Large file uploads over SSL make _DispatchCiphertext loop endlessly #233

theflow opened this Issue Aug 3, 2011 · 6 comments


None yet
2 participants

theflow commented Aug 3, 2011


I ran into this weird bug trying to upload large files. Sending data bigger than 37MB to a HTTPS endpoint just hangs for exactly 5 minutes and consumes 100% CPU. It returns with an empty response and no status code. Looking at it in wireshark, it stops sending packets fairly quickly, so I don't think it's a timeout (with the new unbind reason I get Errno::NOERROR). Smaller files are working fine, the same file over HTTP works fine.

It works when I use patron to do the same request, so I think this a bug in em-http-request or eventmachine, but I don't know enough about eventmachine to further debug this.

This is a small example to reproduce the problem:
(ruby-1.9.2-p180, em-http-request 1.0.0.beta.4)

create a test file:

dd if=/dev/zero of=testfile bs=40214400 count=1

run this

require 'em-http-request'
require 'uri'

file ="testfile") {
  http ="").post(:body =>

  http.errback { 
    p 'Uh oh'
    p http.error
  http.callback {
    p http.response_header.status
    p http.response_header
    p http.response


Any ideas?

theflow commented Aug 3, 2011

I moved this issue from the em-http-request issues, since it's an eventmachine bug:


theflow commented Aug 3, 2011

gdb backtrace and strace are here:

Trying to debug this further, I got to here

SslBox->CanGetCiphertext() never returns false in this case


tmm1 commented Sep 29, 2011

What if you use post(:file => file) instead of reading the entire file into ruby?


tmm1 commented Sep 29, 2011

Can you try the following patch:

diff --git a/ext/ed.cpp b/ext/ed.cpp
index 10fcbfb..3b0a434 100644
--- a/ext/ed.cpp
+++ b/ext/ed.cpp
@@ -1202,16 +1202,18 @@ void ConnectionDescriptor::_DispatchCiphertext()

    char BigBuf [2048];
    bool did_work;
+   int num_chunks = 0;

    do {
        did_work = false;

        // try to drain ciphertext
-       while (SslBox->CanGetCiphertext()) {
+       while (SslBox->CanGetCiphertext() && num_chunks < 16) {
            int r = SslBox->GetCiphertext (BigBuf, sizeof(BigBuf));
            assert (r > 0);
            _SendRawOutboundData (BigBuf, r);
            did_work = true;
+           num_chunks += 1;

        // Pump the SslBox, in case it has queued outgoing plaintext

tmm1 closed this in b237c03 Sep 29, 2011

theflow commented Sep 30, 2011

Awesome, works for me now. Thanks!

tmm1 reopened this Oct 21, 2011

tmm1 closed this in 86784d2 Oct 21, 2011

tmm1 reopened this Oct 21, 2011


tmm1 commented Oct 21, 2011

The fix was causing problems so I've reverted it for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment