HTTP POST with CURLOPT_READFUNCTION takes a long time to complete on Windows, libcurl 7.71.0

[tmkk]

I did this

Compiled and ran the following code:

#include <curl/curl.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/time.h>

#define POSTSIZE 1*1024*1024
static char *postdata;
static int bytesLeft = POSTSIZE;
static double start_ms;

static size_t read_callback(char *buffer, size_t size, size_t nitems, void *userdata)
{
    struct timeval tv;
    gettimeofday(&tv, NULL);
    double now = tv.tv_sec*1e+3 + tv.tv_usec*1e-3;
    int i,bytesToCopy = bytesLeft;
    if(bytesToCopy > size*nitems) bytesToCopy = size*nitems;
    for(i=0;i<bytesToCopy;i++) *buffer++ = *(postdata+POSTSIZE-bytesLeft--);
    fprintf(stderr,"Filled %d/%d bytes, %.3f msec elapsed\n",POSTSIZE-bytesLeft,POSTSIZE,now-start_ms);
    return bytesToCopy;
}

int main(void)
{
    postdata = (char *)calloc(POSTSIZE, 1);
    CURL *curl = curl_easy_init();
    curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
    curl_easy_setopt(curl, CURLOPT_URL, "http://192.168.1.160/");
    curl_easy_setopt(curl, CURLOPT_POST, 1L);
    curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, POSTSIZE);
    curl_easy_setopt(curl, CURLOPT_READFUNCTION, read_callback);

    struct timeval tv;
    gettimeofday(&tv, NULL);
    start_ms = tv.tv_sec*1e+3 + tv.tv_usec*1e-3;

    curl_easy_perform(curl);

    return 0;
}

An upload buffer is filled by a callback set with CURLOPT_READFUNCTION, and the callback is called 16 times to fill 1 MiB of data to be uploaded. On libcurl version 7.71.0 it takes about a second between every callback function calls, and results in more than 10 seconds to complete.

% ./a.exe
*   Trying 192.168.1.160:80...
* Connected to 192.168.1.160 (192.168.1.160) port 80 (#0)
> POST / HTTP/1.1
Host: 192.168.1.160
Accept: */*
Content-Length: 1048576
Content-Type: application/x-www-form-urlencoded

Filled 65536/1048576 bytes, 8.994 msec elapsed
Filled 131072/1048576 bytes, 206.329 msec elapsed
Filled 196608/1048576 bytes, 1207.389 msec elapsed
Filled 262144/1048576 bytes, 2208.778 msec elapsed
Filled 327680/1048576 bytes, 3208.970 msec elapsed
Filled 393216/1048576 bytes, 4210.744 msec elapsed
Filled 458752/1048576 bytes, 5212.402 msec elapsed
Filled 524288/1048576 bytes, 6213.489 msec elapsed
Filled 589824/1048576 bytes, 7216.166 msec elapsed
Filled 655360/1048576 bytes, 8218.369 msec elapsed
Filled 720896/1048576 bytes, 9220.328 msec elapsed
Filled 786432/1048576 bytes, 10221.817 msec elapsed
Filled 851968/1048576 bytes, 11223.812 msec elapsed
Filled 917504/1048576 bytes, 12225.841 msec elapsed
Filled 983040/1048576 bytes, 13227.434 msec elapsed
Filled 1048576/1048576 bytes, 14228.673 msec elapsed
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK

I expected the following

On version 7.70.0 and earlier it finishes almost instantly.

% ./a.exe
*   Trying 192.168.1.160:80...
* Connected to 192.168.1.160 (192.168.1.160) port 80 (#0)
> POST / HTTP/1.1
Host: 192.168.1.160
Accept: */*
Content-Length: 1048576
Content-Type: application/x-www-form-urlencoded

Filled 65536/1048576 bytes, 21.023 msec elapsed
Filled 131072/1048576 bytes, 22.016 msec elapsed
Filled 196608/1048576 bytes, 24.014 msec elapsed
Filled 262144/1048576 bytes, 25.014 msec elapsed
Filled 327680/1048576 bytes, 26.013 msec elapsed
Filled 393216/1048576 bytes, 26.013 msec elapsed
Filled 458752/1048576 bytes, 26.013 msec elapsed
Filled 524288/1048576 bytes, 27.013 msec elapsed
Filled 589824/1048576 bytes, 27.013 msec elapsed
Filled 655360/1048576 bytes, 28.013 msec elapsed
Filled 720896/1048576 bytes, 28.013 msec elapsed
Filled 786432/1048576 bytes, 29.012 msec elapsed
Filled 851968/1048576 bytes, 31.010 msec elapsed
Filled 917504/1048576 bytes, 31.010 msec elapsed
Filled 983040/1048576 bytes, 32.010 msec elapsed
Filled 1048576/1048576 bytes, 33.009 msec elapsed
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK

curl/libcurl version

libcurl version 7.71.0, a commit 8bc25c5 and later

operating system

Tested on Windows 10 1909 build 18363.904, the test code is built with i686-w64-mingw32-gcc.
Windows only issue, not affected on other platforms.

[bagder]

Just to clarify, if you build with 7.71.0 with commit 8bc25c5 reverted, everything is fine again?

[tmkk]

Confirmed that it's fine when the commit 8bc25c5 is reverted from the 7.71.0 release.