HTTPS upload hangs using Windows Server 2008

[lietusme]

This is continuation to issues found in 2013 and 2015:

https://sourceforge.net/p/curl/bugs/1275/
https://curl.haxx.se/mail/lib-2015-05/0063.html

Issue 1. This is main problem - as described in links above. Pinpointed that it is caused when all of these are true:

• Uploading file in 4MB POST/PUT reguests to service deployed in Windows Server 2008/IIS 7.5
• Reusing CURL handles (either with curl_easy_reset() or using CURLM)
• Using CURLOPT_POSTFIELDSIZE/CURLOPT_POSTFIELDSIZE_LARGE with CURLOPT_POSTFIELDS
  or using CURLOPT_INFILESIZE/CURLOPT_INFILESIZE_LARGE with CURLOPT_READFUNCTION.

Result (reproduceable 100%):

1. Request 1 uploads 4MB
2. Request 2 uploads 3.45MB (+- 100K) and stucks
3. Read function is no longer called.
4. CURL timeout happens.
5. Request hangs for some time on server side and keeps file locked.

3 workarounds that I know that work:

1. Do not use curl_easy_reset(), use curl_easy_init()
2. Do not use CURLM for uploads
3. Use plain CURLOPT_POSTFIELDS without setting request size manually. However, this does not work for binary uploads as it cannot calculate correct length.

We chose workarounds 1 (2013) and 2 (2015).

Issue 2. Side problem that we found recently with new servers and above workarounds:

• Uploading file in 4MB POST/PUT requests to service deployed in Windows Server 2012/IIS 8.0
• NOT reusing CURL handles and using curl_easy_init() for each request.

Result (varying results, same outcome when fails):

1. Upload fails after 100-900MBs, succeeds in rare cases for 1GB file.
2. CURL fails with:
   "Unknown SSL protocol error in connection to myHostName:443. SSL connect error(CURLE_SSL_CONNECT_ERROR)"

Workaround that I know:

1. Use curl_easy_reset() or CURLM.

So now I'm stuck, fixing issue 2 enables issue 1 and vice versa.

I would preffer fixing issue 1 because using CURLM is far more flexible approach. I currently have plain CURL C++ source code talking to specific server if that would help, but no server to test it on. I will try to get public server deployed for reproducing issue 1 but not sure when that could be.

Any ideas so far how to solve hanging upload?
Why would using plain CURLOPT_POSTFIELDS fix issue 1?

libCURL:7.50.3
OpenSSL:1.0.2j
Client OS: iOS/Windows x64

[jay]

If this is reproducible try the latest libcurl from the repo with the latest OpenSSL 1.0.2, use curl_version() to confirm, see if you can still reproduce. If you can then try the latest libcurl with WinSSL, use curl_version() to confirm, see if you can still reproduce. Post the curl versions here.

Also make sure you are using a variable of the documented type, or a typecast
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE_LARGE, (curl_off_t)num);

And please read https://curl.haxx.se/libcurl/c/threadsafe.html, specifically "you must never use a single handle from more than one thread at any given time." should you be doing that

[jay]

I'm also reminded of what I said in 2015 about this https://curl.haxx.se/mail/lib-2015-05/0176.html

[lietusme]

Thanks for reply!

Yes, I do use curl_off_t for CURLOPT_POSTFIELDSIZE_LARGE or CURLOPT_INFILESIZE_LARGE.

No, our code does not use one CURL handle in different threads at the same time. It could be given to different threads though but one at the time. However, I reproduced both issues in simple blocking code on one thread as well.

No, 0 is not returned prematuralery from read function. I also reproduced issue 1 with test data (4MB string of 'x' chars) with such code:

curl_easy_setopt(curl, CURLOPT_POSTFIELDS, str.c_str());
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, (long)str.length());

Removing CURLOPT_POSTFIELDSIZE solves the issue - so it seems that setting size itself causes it, somehow.

I could will try latest builds later

[bagder]

Removing CURLOPT_POSTFIELDSIZE solves the issue - so it seems that setting size itself causes it, somehow.

That would indicate that your code and libcurl itself would set different sizes. Are you saying that this is a work-around for the "Unknown SSL protocol error" too?

[lietusme]

My test code did exacly the same thing as CURL does to calculate post body size - strlen(). Even tested CURLOPT_POSTFIELDSIZE with strlen() instead of std::string::length() and reproduces same issue.

No, "Unknown SSL protocol error" is described as "issue 2" in first post. Main fix for it is to remove workaround for "issue 1" as far as I could find.

This is sample test code that I used. However, I don't have public server available yet so URLs are stubbed out. Technically I think this could reproduce with any repeaded upload scenario, but not sure. Anything wrong with this code in particular?

#include <curl/curl.h>
#include <string>
#include <map>

#define TESTLOG(fmt, ...) printf(fmt, ##__VA_ARGS__)

static size_t header_callback(char* buffer, size_t size, size_t nitems, std::map<std::string, std::string>* headers)
    {
    std::string bufferStr(buffer, size*nitems);
    size_t splitPos = bufferStr.find(": ");

    if (std::string::npos == splitPos)
        {
        (*headers)[bufferStr] = "";
        }
    else
        {
        std::string name = bufferStr.substr(0, splitPos);
        std::string value = bufferStr.substr(splitPos + 2);
        // Remove newlines
        if (value.length() && *value.rbegin() == '\n')
            value.pop_back();
        if (value.length() && *value.rbegin() == '\r')
            value.pop_back();
        (*headers)[name] = value;
        }
    return nitems * size;
    }

static size_t write_callback(char* buffer, size_t size, size_t nitems, void *userdata)
    {
    TESTLOG("Write: %s", std::string(buffer, size*nitems).c_str());
    return nitems * size;
    }

void CurlTestIssue1()
    {
    curl_global_init(CURL_GLOBAL_ALL);

    // ------------------------------------------------------------------------------------------------
    // Connection parameters (stubs):
    // Windows Server 2008
    std::string url2008 = "https://foo2008";
    // Windows Server 2012
    std::string url2012 = "https://foo2012";
    const char* username = "user";
    const char* password = "pass";
    // ------------------------------------------------------------------------------------------------

    // ------------------------------------------------------------------------------------------------
    // Test parameters that have effect on test outcome:
    // url2008 - request hangs after few chunks - fail.
    // url2012 - requests are sent indefinetely - success.
    std::string url = url2008; 
    // Decreasing chunk size makes more requests to succeed,
    // but eventually it fails at specific number.
    size_t bytesChunk = 4 * 1024 * 1024;
    // Not using CURLOPT_POSTFIELDSIZE and leaving CURL to calculate chunk size
    // somehow solves the issue. Not an option for binary upload.
    bool usePostFieldSize = true;
    // Using new CURL hadle each time makes all chunk requests to succeed -
    // probably resseting length somewhere?
    bool reuseCurlHandle = true;
    // ------------------------------------------------------------------------------------------------

    // Send handshake to initiate upload
    CURL *curl;
    CURLcode res;

    curl = curl_easy_init();
    if (!curl)
        return;

    auto setupCurl = [&] (CURL* curl)
        {
        curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
        curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
        curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
        curl_easy_setopt(curl, CURLOPT_USERNAME, username);
        curl_easy_setopt(curl, CURLOPT_PASSWORD, password);
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_callback);
        };

    struct curl_slist* chunk = nullptr;
    chunk = curl_slist_append(chunk, R"(Content-Range: bytes */100000000)");
    chunk = curl_slist_append(chunk, R"(Content-Disposition: attachment; filename="a.txt")");
    chunk = curl_slist_append(chunk, R"(Content-Length: 0)");
    res = curl_easy_setopt(curl, CURLOPT_HTTPHEADER, chunk);

    std::map<std::string, std::string> headers;
    curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, header_callback);
    curl_easy_setopt(curl, CURLOPT_HEADERDATA, &headers);
    curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT");
    curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
    setupCurl(curl);

    res = curl_easy_perform(curl);
    if (res != CURLE_OK)
        {
        TESTLOG("curl_easy_perform() failed: %s", curl_easy_strerror(res));
        assert(false);
        return;
        }

    long httpCode = 0;
    curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &httpCode);
    if (308 != httpCode)
        {
        assert(false);
        return;
        }

    TESTLOG("Handshake done", "");

    // Upload first chunk repeadetely for testing purposes only.
    // Failure if stops, successful if continues repeadetely.
    auto eTag = headers["ETag"];
    auto ifMatch = "If-Match: " + eTag;

    auto requestNumber = 0;
    while (true)
        {
        if (reuseCurlHandle)
            {
            curl_easy_reset(curl);
            }
        else
            {
            curl_easy_cleanup(curl);
            curl = curl_easy_init();
            }

        requestNumber ++;
        TESTLOG("Chunk %d uploading...", requestNumber);

        curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT");
        curl_easy_setopt(curl, CURLOPT_URL, url.c_str());

        curl_slist_free_all(chunk);
        chunk = nullptr;
        auto contentRange = "Content-Range: bytes 0-" + std::to_string(bytesChunk - 1) + "/100000000";
        chunk = curl_slist_append(chunk, contentRange.c_str());
        chunk = curl_slist_append(chunk, ifMatch.c_str());
        res = curl_easy_setopt(curl, CURLOPT_HTTPHEADER, chunk);

        auto body = std::string(bytesChunk, 'x');
        curl_easy_setopt(curl, CURLOPT_POSTFIELDS, body.c_str());
        if (usePostFieldSize)
            curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, (long) strlen(body.c_str()));

        setupCurl(curl);

        res = curl_easy_perform(curl);
        if (res != CURLE_OK)
            {
            TESTLOG("curl_easy_perform() failed: %s", curl_easy_strerror(res));
            assert(false);
            break;
            }

        TESTLOG("Chunk %d done", requestNumber);
        long httpCode = 0;
        curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &httpCode);
        TESTLOG("curl_easy_perform() code: %d", httpCode);
        if (308 != httpCode)
            {
            assert(false);
            break;
            }
        }

    curl_easy_cleanup(curl);
    curl_slist_free_all(chunk);
    }

[lietusme]

Checked - reproduced "Issue 1" with latest locally built libcurl/7.53.1 OpenSSL/1.0.2k and clean Visual Studio Win32 console application.

[bagder]

1. Your header callback seems to assume a trailing zero byte after the header, which is wrong. (I don't do C++ enough to tell if the regular data callback has it right or wrong.)
2. The example code does a funny-looking PUT first, is that required to trigger the issue?
3. Please file a single issue in one issue report. If the issues are separate, file two issues. If they're not separate, then we consider them the same issue!
4. You explicitly say this fails "to windows server 2008 and 2012". Is that another way to say that the same application works against some other servers or set of servers?

[lietusme]

1. Thanks, I just hacked up header_callback for this test and missed that, but it extracted headers correctly as far as test needs. Rewritten just in case, but it did not change anything:

static size_t header_callback(char* buffer, size_t size, size_t nitems, std::map<std::string, std::string>* headers)
    {
    std::string bufferStr(buffer, size*nitems);
    size_t splitPos = bufferStr.find(": ");

    if (std::string::npos == splitPos)
        {
        (*headers)[bufferStr] = "";
        }
    else
        {
        std::string name = bufferStr.substr(0, splitPos);
        std::string value = bufferStr.substr(splitPos + 2);
        // Remove newlines
        value.pop_back();
        value.pop_back();
        (*headers)[name] = value;
        }
    return nitems * size;
    }

2. Our service uses such upload protocol:

• Client sends "handshake" request with file size, name
• Service responds with generated ETag for upload tracking
• Client uses same ETag to upload chunks to it until finished.

Not sure if its required to trigger issue. I think it starts failing after some amound of data/requests sent with same CURL handle.

3. Agree - main issue here is "Issue 1", "Issue 2" is side effect of workarounds done for first one. I only added both because wanted to show that there is no way to solve both properly... I'll change subject and main post a bit then to focus on one issue now.

4. I don't know. I only reproduced this with our service deployed on Windows Server 2008 OS/ISS 7.5 machine. If same service is deployed to 2012/IIS 8.0 - no original issue, just that SSL one.

[jay]

This isn't a solution for the issue but re your code check the header line bytes before pop, they're not necessarily CRLF, do this

  if(value.length() && *value.rbegin() == '\n') {
    value.pop_back();
    if(value.length() && *value.rbegin() == '\r')
      value.pop_back();
  }

[bagder]

I would urge you to provide us with sample code that we can use against our own test servers that reproduces the problem. You seem to only see this issue against a specific server version with your custom application only (or did I get that wrong?), and that's not enough for us to go on!

[lietusme]

Jay - our production code uses more sophisticated header function, this one was only needed for ETag extraction for particular case, so used hardcoded logic. Added your suggestions to sample

Daniel - my provided sample code is written for purpose of reproducing this issue in post above, so it's not custom client app. We reproduced this only when uploading to service deployed on "Windows Server 2008". I don't have any other services to test on, so thats why that funny upload protocol is used there. And as mentioned - removing CURLOPT_POSTFIELDSIZE makes upload succeed, so something is happening on CURL side as well.

It would be great if I could use your server. It would take few weeks until I could provide our public server to test this code on...

Could you provide URL to your test server? I could try reproducing same issue and provide you with sample code. Service needs to support uploading multiple 4MB requests and be running on "Windows Server 2008" with IIS 7.5.

[bagder]

We don't have a single test server (unless you mean the one in the curl test directory). We all run servers and we could run test cases against our own servers to help debug this case. But as long as you really need Windows Server 2008 with IIS 7.5 to reproduce this problem, I'm leaning towards this being a server issue and not a problem in curl. I personally have no test servers running on windows at all so I can't run any such tests.

[lietusme]

While it may be server issue as well, it is caused by combination of CURL configuration + server. Using different server OS version or different CURL configuration with same server does not reproduce this issue, so it may be something wrong on both sides..

Would it help solving this if I would provide sample code (as above) and send you server URL in private (email)?

[bagder]

It might, and I'd be willing to give it a shot to see if I can learn something.

[jay]

Using different server OS version or different CURL configuration with same server does not reproduce this issue, so it may be something wrong on both sides..

I notice you did not post the curl_version() from the ones you tried, and did you ever try libcurl built against the other SSL backends such as mbedTLS or wolfSSL or WinSSL? Skimming the thread I don't see it mentioned. In other words if iOS and Windows curl builds are both built against the same OpenSSL maybe that's something. If you try libcurl built against the respective native SSL backend (ie DarwinSSL, WinSSL) I'm curious what happens.

To keep your sanity when you are running repro print curl_version() to confirm the repro case is using the expected version of libcurl that you are testing.

[lietusme]

Jay - originally I have tested with this:
libcurl/7.53.1 OpenSSL/1.0.2k WinIDN

As per your request, tested with WinSSL:
libcurl/7.53.1 WinSSL WinIDN
And indeed it did not have same issues - so it would probabably be safe to assume something happening with OpenSSL here.

I'll be providing test URL for Daniel, here's latest test code (Visual Studio console app):

#include "stdafx.h"

#include <curl/curl.h>
#include <string>
#include <map>
#include <assert.h>

#define TESTLOG(fmt, ...) printf(fmt, ##__VA_ARGS__)

static void myAssert(bool aa)
    {
    assert(aa);
    }

static size_t header_callback(char* buffer, size_t size, size_t nitems, std::map<std::string, std::string>* headers)
    {
    std::string bufferStr(buffer, size*nitems);
    size_t splitPos = bufferStr.find(": ");

    if (std::string::npos == splitPos)
        {
        (*headers)[bufferStr] = "";
        }
    else
        {
        std::string name = bufferStr.substr(0, splitPos);
        std::string value = bufferStr.substr(splitPos + 2);
        // Remove newlines
        if (value.length() && *value.rbegin() == '\n')
            value.pop_back();
        if (value.length() && *value.rbegin() == '\r')
            value.pop_back();
        (*headers)[name] = value;
        }
    return nitems * size;
    }

static size_t write_callback(char* buffer, size_t size, size_t nitems, void *userdata)
    {
    TESTLOG("Write: %s\n", std::string(buffer, size*nitems).c_str());
    return nitems * size;
    }

void CurlTestIssue1()
    {
    curl_global_init(CURL_GLOBAL_ALL);

    TESTLOG("curl_version():  %s\n", curl_version());
    TESTLOG("LIBCURL_VERSION: %s\n", LIBCURL_VERSION);

    // ------------------------------------------------------------------------------------------------
    // Connection parameters:
    std::string url2008 = "https://test-server";
    const char* username = "admin";
    const char* password = "admin";
    // ------------------------------------------------------------------------------------------------

    // ------------------------------------------------------------------------------------------------
    // Test parameters that affect test outcome:
    // FAILURE - request hangs after few (1-2) chunks.
    // SUCCESS - requests are sent indefinetely.
    std::string url = url2008;
    // Decreasing chunk size makes more requests to succeed, but eventually it fails at specific number.
    size_t bytesChunk = 4 * 1024 * 1024;
    // Not using CURLOPT_POSTFIELDSIZE and leaving CURL to calculate chunk size somehow solves the issue. Not an option for binary upload.
    bool usePostFieldSize = true;
    // Using new CURL hadle each time makes all chunk requests to succeed - probably resseting length somewhere?
    bool reuseCurlHandle = true;
    // ------------------------------------------------------------------------------------------------

    // Send handshake to initiate upload
    CURL *curl;
    CURLcode res;

    curl = curl_easy_init();
    if (!curl)
        return;

    auto setupCurl = [&] (CURL* curl)
        {
        curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
        curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
        curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
        curl_easy_setopt(curl, CURLOPT_USERNAME, username);
        curl_easy_setopt(curl, CURLOPT_PASSWORD, password);
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_callback);
        };

    struct curl_slist* chunk = nullptr;
    chunk = curl_slist_append(chunk, R"(Content-Range: bytes */10000000)");
    chunk = curl_slist_append(chunk, R"(Content-Disposition: attachment; filename="a.txt")");
    chunk = curl_slist_append(chunk, R"(Content-Length: 0)");
    res = curl_easy_setopt(curl, CURLOPT_HTTPHEADER, chunk);

    std::map<std::string, std::string> headers;
    auto foo = curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, header_callback);
    curl_easy_setopt(curl, CURLOPT_HEADERDATA, &headers);
    curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT");
    curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
    setupCurl(curl);

    res = curl_easy_perform(curl);
    if (res != CURLE_OK)
        {
        TESTLOG("curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
        myAssert(false);
        return;
        }

    long httpCode = 0;
    curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &httpCode);
    if (308 != httpCode)
        {
        myAssert(false);
        return;
        }

    TESTLOG("Handshake done\n", "");

    // Upload first chunk repeadetely for testing purposes only. Failure if stops, successful if continues repeadetely.
    auto eTag = headers["ETag"];
    auto ifMatch = "If-Match: " + eTag;

    auto requestNumber = 0;
    while (true)
        {
        if (reuseCurlHandle)
            {
            curl_easy_reset(curl);
            }
        else
            {
            curl_easy_cleanup(curl);
            curl = curl_easy_init();
            }

        requestNumber++;
        TESTLOG("Chunk %d uploading...\n", requestNumber);

        curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT");
        curl_easy_setopt(curl, CURLOPT_URL, url.c_str());

        curl_slist_free_all(chunk);
        chunk = nullptr;
        auto contentRange = "Content-Range: bytes 0-" + std::to_string(bytesChunk - 1) + "/10000000";
        chunk = curl_slist_append(chunk, contentRange.c_str());
        chunk = curl_slist_append(chunk, ifMatch.c_str());
        res = curl_easy_setopt(curl, CURLOPT_HTTPHEADER, chunk);

        auto body = std::string(bytesChunk, 'x');
        curl_easy_setopt(curl, CURLOPT_POSTFIELDS, body.c_str());
        if (usePostFieldSize)
            curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, (long) strlen(body.c_str()));

        setupCurl(curl);

        res = curl_easy_perform(curl);
        if (res != CURLE_OK)
            {
            TESTLOG("curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
            myAssert(false);
            break;
            }

        TESTLOG("Chunk %d done\n", requestNumber);
        long httpCode = 0;
        curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &httpCode);
        TESTLOG("curl_easy_perform() code: %d\n", httpCode);
        if (308 != httpCode)
            {
            myAssert(false);
            break;
            }
        }

    curl_easy_cleanup(curl);
    curl_slist_free_all(chunk);
    }


int _tmain(int argc, _TCHAR* argv[])
    {
    CurlTestIssue1();
    return 0;
    }

[lietusme]

Any news on this?

[bagder]

Not really. Using your code and your test server I could reproduce the issue. But I've failed to figure out what makes the requests different or why they would cause different responses. Haven't spent enough time on it yet, clearly.

[lietusme]

Any update?:)

[bagder]

It had indeed fallen off my radar, but now when I try to rerun the code to reproduce this issue the host name doesn't resolve any more...

[jay]

I have IIS 7.5 running in a Server 2008 R2 SP1 VM. What else do I need to reproduce this?

[lietusme]

I previously sent Daniel URL to our deployment, but it's down now, I'm figuring out how to reset it so it could be used again.

Jay - so far we only used our server product running on IIS, thats why we are deploying seperate instance for this issue to be debugged. I'm sure it would reproduce with other IIS based server that accepts multiple large POST requests as well, but I only implemented client side test code. If needed, test server could probably be implemented as well but thats what I have now.

[bagder]

It can be worth mentioning to @jay that @lietusme only sent me an updated URL for the recipe, the rest of the code is exactly as provided in #1294 (comment)

[lietusme]

@bagder - sent new URL to your email with service on azure that is dedicated for reproducing this issue

[bagder]

Good news and bad news!

I figured out why there's a difference in behavior between the two cases: we also use the info for the upcoming post size to set an internal "expect this amount of data to upload" (data->state.infilesize) when CURLOPT_POSTFIELDSIZE is used but we did not when CURLOPT_POSTFIELDSIZEwas not set. (I'll run more tests and plan to submit a fix soon).

But... when this is fixed in libcurl, the example program now always hangs! The reason it worked before was that libcurl would break the connection on the 308 response thinking it had more data to send (while in reality it didn't but it didn't keep track there of the total amount it should send) and when getting an error early it needs to cut the connection to be able to do that.

With this fix, it sees that the response code is not early and doesn't disconnect. Instead it reuses the connection much better and now runs into this problem consistently: something is wrong in the server end that makes it not respond anymore. There's never any response from the third PUT. Nothing. Not a single byte of HTTP header even. I extended the test application with a progress function to make it highly visible that libcurl is then just sitting waiting for a response.

Here's the diff I'm working with right now:

From 32bf744f1c5864e157738111dc8d1263ad0a87b6 Mon Sep 17 00:00:00 2001
From: Daniel Stenberg <daniel@haxx.se>
Date: Tue, 30 May 2017 00:02:56 +0200
Subject: [PATCH] transfer: init the infilesize from the postfields...

... with a strlen() if no size was set, and do this in the pretransfer
function so that the info is set early.
---
 lib/transfer.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/transfer.c b/lib/transfer.c
index 799fd4da8..43e8f64aa 100644
--- a/lib/transfer.c
+++ b/lib/transfer.c
@@ -1307,12 +1307,15 @@ CURLcode Curl_pretransfer(struct Curl_easy *data)
   Curl_safefree(data->info.wouldredirect);
   data->info.wouldredirect = NULL;
 
   if(data->set.httpreq == HTTPREQ_PUT)
     data->state.infilesize = data->set.filesize;
-  else
+  else {
     data->state.infilesize = data->set.postfieldsize;
+    if(data->set.postfields && (data->state.infilesize == -1))
+      data->state.infilesize = (curl_off_t)strlen(data->set.postfields);
+  }
 
   /* If there is a list of cookie files to read, do it now! */
   if(data->change.cookielist)
     Curl_cookie_loadfiles(data);
 
-- 
2.11.0

[bagder]

A slightly extended version of that patch (ending in a net loss of 2 lines of code!) is now merged.

[lietusme]

@bagder thanks! Does latest push fix the hang you mentioned? Also what/when CURL version would include this?

[jay]

@lietusme as @bagder said, even with the fix:

There's never any response from the third PUT. Nothing. Not a single byte of HTTP header even.

[bagder]

@lietusme: the libcurl fix is already merged to master and will be part of the pending 7.54.1 release due to ship on June 14th, but as I mentioned: the fix actually make "the hang" appear much more reproducible so it may not appear as much of a fix to your particular problem...

[lietusme]

Any idea why this does not hang when WinSSL is used?

[jay]

Any idea why this does not hang when WinSSL is used?

Is it possible that is arbitrary, in other words run master w/ WinSSL 100 times then run master w/ OpenSSL 100 times, are the results consistent?

[lietusme]

I did not run it 100 times (would need to make test automatic), but just ran 5 times manually. Results:
5x libcurl/7.54.1-DEV WinSSL WinIDN - test requests never hang - eatch time got more than 20x 4MB chunks uploaded.
5x libcurl/7.54.1-DEV OpenSSL/1.0.2k WinIDN - second 4MB chunk request hanged and failed each time. Request hangs and fails after around 2 minutes 18±2 seconds since starting upload. I don't think I saw these errors before:

< HTTP/1.1 100 Continue
* We are completely uploaded and fine
* OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 10054
* Closing connection 0
curl_easy_perform() failed: Failure when receiving data from the peer

[bagder]

We saw the SSL_ERROR_SYSCALL in another report just recently and in that case it happened because the TCP connection was cut off (RST received), and given the rest of the log here it seems you may experience something similar.

It's really hard for us to tell why schannel/winssl works and OpenSSL doesn't, as it seems a decision made by the server...

[lietusme]

Any tips on what to do next..? Analyze server-side TCP traffic, etc?

Original issues that require conflicting fixes from first post still exists:

1. Using curl_easy_init() and CURL* for each request. Issue 1 fixed, issue 2 reproduces.
2. Using curl_easy_reset() or CURLM* for each request. Issue 1 reproduces, issue 2 fixed.