-
-
Notifications
You must be signed in to change notification settings - Fork 6.7k
Http2 multi stream, one of stream error cause Pipe broke #941
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I'm having trouble understanding the problem here. Can you try the latest version 7.50? |
I mean that i want to reuse the connection to send the HTTP/2 request, then I used multi handle and add two easyhandle to make request, but one of request finished by tomeout and cause the reused connect broken. I find the code in curl: It close the ssl connection, and cause the other request broken too. |
Can you please provide a minimalist example source code that reproduces the problem? |
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
/* somewhat unix-specific */
#include <sys/time.h>
#include <unistd.h>
/* curl stuff */
#include <curl/curl.h>
#ifndef CURLPIPE_MULTIPLEX
/* This little trick will just make sure that we don't enable pipelining for
libcurls old enough to not have this symbol. It is _not_ defined to zero in
a recent libcurl header. */
#define CURLPIPE_MULTIPLEX 0
#endif
#define NUM_HANDLES 50
void *curl_hnd[NUM_HANDLES];
int num_transfers;
int request_stop = 0;
static int
progress_cb(void *clientp,
curl_off_t dltotal,
curl_off_t dlnow,
curl_off_t ultotal,
curl_off_t ulnow)
{
if (request_stop) {
// If I return > 0, the libcurl will reconnect the connect
printf("progress_callback exit\n");
return 1;
}
return 0;
}
static void setup(CURL *hnd, int num)
{
FILE *out;
char filename[128];
snprintf(filename, 128, "dl-%d", num);
out = fopen(filename, "wb");
/* write to this file */
curl_easy_setopt(hnd, CURLOPT_WRITEDATA, out);
/* set the same URL */
curl_easy_setopt(hnd, CURLOPT_URL, "https://localhost:8443/index.html");
/* send it verbose for max debuggaility */
curl_easy_setopt(hnd, CURLOPT_VERBOSE, 1L);
/* HTTP/2 please */
curl_easy_setopt(hnd, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_2_0);
// Set the timeout val as mini as you can
curl_easy_setopt(hnd, CURLOPT_TIMEOUT, 1L);
curl_easy_setopt(hnd, CURLOPT_NOPROGRESS, 0);
curl_easy_setopt(hnd, CURLOPT_XFERINFOFUNCTION, progress_cb);
/* we use a self-signed test server, skip verification during debugging */
curl_easy_setopt(hnd, CURLOPT_SSL_VERIFYPEER, 0L);
curl_easy_setopt(hnd, CURLOPT_SSL_VERIFYHOST, 0L);
#if (CURLPIPE_MULTIPLEX > 0)
/* wait for pipe connection to confirm */
curl_easy_setopt(hnd, CURLOPT_PIPEWAIT, 1L);
#endif
curl_hnd[num] = hnd;
}
/*
* Simply download two files over HTTP/2, using the same physical connection!
*/
int main(int argc, char **argv)
{
CURL *easy[NUM_HANDLES];
CURLM *multi_handle;
int i;
int still_running; /* keep number of running handles */
if(argc > 1)
/* if given a number, do that many transfers */
num_transfers = atoi(argv[1]);
if(!num_transfers || (num_transfers > NUM_HANDLES))
num_transfers = 3; /* a suitable low default */
/* init a multi stack */
multi_handle = curl_multi_init();
for(i=0; i<num_transfers; i++) {
easy[i] = curl_easy_init();
/* set options */
setup(easy[i], i);
/* add the individual transfer */
curl_multi_add_handle(multi_handle, easy[i]);
}
curl_multi_setopt(multi_handle, CURLMOPT_PIPELINING, CURLPIPE_MULTIPLEX);
curl_multi_setopt(multi_handle, CURLMOPT_MAX_HOST_CONNECTIONS, 1L);
/* we start some action by calling perform right away */
curl_multi_perform(multi_handle, &still_running);
do {
struct timeval timeout;
int rc; /* select() return code */
CURLMcode mc; /* curl_multi_fdset() return code */
fd_set fdread;
fd_set fdwrite;
fd_set fdexcep;
int maxfd = -1;
long curl_timeo = -1;
FD_ZERO(&fdread);
FD_ZERO(&fdwrite);
FD_ZERO(&fdexcep);
/* set a suitable timeout to play around with */
timeout.tv_sec = 1;
timeout.tv_usec = 0;
curl_multi_timeout(multi_handle, &curl_timeo);
if(curl_timeo >= 0) {
timeout.tv_sec = curl_timeo / 1000;
if(timeout.tv_sec > 1)
timeout.tv_sec = 1;
else
timeout.tv_usec = (curl_timeo % 1000) * 1000;
}
/* get file descriptors from the transfers */
mc = curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);
if(mc != CURLM_OK) {
fprintf(stderr, "curl_multi_fdset() failed, code %d.\n", mc);
break;
}
/* On success the value of maxfd is guaranteed to be >= -1. We call
select(maxfd + 1, ...); specially in case of (maxfd == -1) there are
no fds ready yet so we call select(0, ...) --or Sleep() on Windows--
to sleep 100ms, which is the minimum suggested value in the
curl_multi_fdset() doc. */
if(maxfd == -1) {
#ifdef _WIN32
Sleep(100);
rc = 0;
#else
/* Portable sleep for platforms other than Windows. */
struct timeval wait = { 0, 100 * 1000 }; /* 100ms */
rc = select(0, NULL, NULL, NULL, &wait);
#endif
}
else {
/* Note that on some platforms 'timeout' may be modified by select().
If you need access to the original value save a copy beforehand. */
rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout);
}
switch(rc) {
case -1:
/* select error */
break;
case 0:
default:
/* timeout or readable/writable sockets */
curl_multi_perform(multi_handle, &still_running);
break;
}
} while(still_running);
curl_multi_cleanup(multi_handle);
for(i=0; i<num_transfers; i++)
curl_easy_cleanup(easy[i]);
return 0;
}
That is my sample code to test, I donot have a http2 server to have a test. |
Is that a problem of curl or that my way to used curl incorrect??? |
It looks like a problem in curl. |
Is there any way to fix this? |
Sure, first we figure out what the problem is exactly, then we change the code to make sure it works for this case as well. Then we land the change, ideally together with a new test or two. |
Hello bagder: |
Assuming that we have a fix before that is to be released, sure. I've verified that it is easy to reproduce this problem but I haven't had time to work on a fix yet. |
Can you give me an idea for fix this: I will try by myself. TKS! |
It is not a small and quick fix. If I had one I would already have provided it. I'm working on this. |
I have a patch in progress that seems to work. It would be awesome if you could try it out before I land this, as it would make me feel more safe it doesn't break things too bad. I'm attaching it like this for now: 0001-http2-make-sure-stream-errors-don-t-needlessly-close.patch |
I did this
I add two easyhandle(one of them make http2 post requset, the ohter make http2 get request) to a multihandle, i already set CURLMOPT_MAX_HOST_CONNECTIONS and CURLMOPT_PIPELINING to reused the connction, if one of the easyhandle request failed because of timeout or readcallback abort, then cause "Pipe broke" and the connection will reconnect.
I expected the following
curl/libcurl version
curl --version
curl 7.46.0 (mipsel-unknown-linux-gnu) libcurl/7.46.0 OpenSSL/1.0.2e zlib/1.2.3 nghttp2/1.9.0
Protocols: http https smb smbs
Features: Largefile NTLM SSL libz HTTP2 UnixSockets
operating system
linux
The text was updated successfully, but these errors were encountered: