Skip to content

Commit 2f44e94

Browse files
bagderbagder
committed
pipelining: removed
As previously planned and documented in DEPRECATE.md, all pipelining code is removed. Closes #3651
1 parent aba1c51 commit 2f44e94

28 files changed

+183
-1394
lines changed

‎docs/DEPRECATE.md

-40
Original file line numberDiff line numberDiff line change
@@ -5,46 +5,6 @@ email the curl-library mailing list as soon as possible and explain to us why
55
this is a problem for you and how your use case can't be satisfied properly
66
using a work around.
77

8-
## HTTP pipelining
9-
10-
HTTP pipelining is badly supported by curl in the sense that we have bugs and
11-
it is a fragile feature without enough tests. Also, when something turns out
12-
to have problems it is really tricky to debug due to the timing sensitivity so
13-
very often enabling debug outputs or similar completely changes the nature of
14-
the behavior and things are not reproducing anymore!
15-
16-
HTTP pipelining was never enabled by default by the large desktop browsers due
17-
to all the issues with it. Both Firefox and Chrome have also dropped
18-
pipelining support entirely since a long time back now. We are in fact over
19-
time becoming more and more lonely in supporting pipelining.
20-
21-
The bad state of HTTP pipelining was a primary driving factor behind HTTP/2
22-
and its multiplexing feature. HTTP/2 multiplexing is truly and really
23-
"pipelining done right". It is way more solid, practical and solves the use
24-
case in a better way with better performance and fewer downsides and problems.
25-
26-
In 2018, pipelining *should* be abandoned and HTTP/2 should be used instead.
27-
28-
### State
29-
30-
In 7.62.0, we will add code that ignores the "enable pipeline" option
31-
setting). The *setopt() function would still return "OK" though so the
32-
application couldn't tell that this is happening.
33-
34-
Users who truly need pipelining from that version will need to modify the code
35-
(ever so slightly) and rebuild.
36-
37-
### Removal
38-
39-
Six months later, in sync with the planned release happen in April 2019,
40-
(might be 7.66.0), assuming no major riots have occurred due to this in the
41-
mean time, we rip out the pipelining code. It is in the order of 1000 lines of
42-
libcurl code.
43-
44-
Left to answer: should the *setopt() function start to return error when these
45-
options are set to be able to tell when they're trying to use options that are
46-
no longer around or should we maintain behavior as much as possible?
47-
488
## `CURLOPT_DNS_USE_GLOBAL_CACHE`
499

5010
This option makes libcurl use a global non-thread-safe cache for DNS if

‎docs/libcurl/opts/CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE.3

+3-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
.\" * | (__| |_| | _ <| |___
66
.\" * \___|\___/|_| \_\_____|
77
.\" *
8-
.\" * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al.
8+
.\" * Copyright (C) 1998 - 2019, Daniel Stenberg, <daniel@haxx.se>, et al.
99
.\" *
1010
.\" * This software is licensed as described in the file COPYING, which
1111
.\" * you should have received as part of this distribution. The terms
@@ -28,6 +28,8 @@ CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE \- chunk length threshold for pipelining
2828

2929
CURLMcode curl_multi_setopt(CURLM *handle, CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE, long size);
3030
.SH DESCRIPTION
31+
No function since pipelining was removed in 7.62.0.
32+
3133
Pass a long with a \fBsize\fP in bytes. If a pipelined connection is currently
3234
processing a chunked (Transfer-encoding: chunked) request with a current chunk
3335
length larger than \fICURLMOPT_CHUNK_LENGTH_PENALTY_SIZE(3)\fP, that pipeline

‎docs/libcurl/opts/CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE.3

+3-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
.\" * | (__| |_| | _ <| |___
66
.\" * \___|\___/|_| \_\_____|
77
.\" *
8-
.\" * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al.
8+
.\" * Copyright (C) 1998 - 2019, Daniel Stenberg, <daniel@haxx.se>, et al.
99
.\" *
1010
.\" * This software is licensed as described in the file COPYING, which
1111
.\" * you should have received as part of this distribution. The terms
@@ -28,6 +28,8 @@ CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE \- size threshold for pipelining penalty
2828

2929
CURLMcode curl_multi_setopt(CURLM *handle, CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE, long size);
3030
.SH DESCRIPTION
31+
No function since pipelining was removed in 7.62.0.
32+
3133
Pass a long with a \fBsize\fP in bytes. If a pipelined connection is currently
3234
processing a request with a Content-Length larger than this
3335
\fICURLMOPT_CONTENT_LENGTH_PENALTY_SIZE(3)\fP, that pipeline will then not be

‎docs/libcurl/opts/CURLMOPT_MAX_PIPELINE_LENGTH.3

+3-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
.\" * | (__| |_| | _ <| |___
66
.\" * \___|\___/|_| \_\_____|
77
.\" *
8-
.\" * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al.
8+
.\" * Copyright (C) 1998 - 2019, Daniel Stenberg, <daniel@haxx.se>, et al.
99
.\" *
1010
.\" * This software is licensed as described in the file COPYING, which
1111
.\" * you should have received as part of this distribution. The terms
@@ -28,6 +28,8 @@ CURLMOPT_MAX_PIPELINE_LENGTH \- maximum number of requests in a pipeline
2828

2929
CURLMcode curl_multi_setopt(CURLM *handle, CURLMOPT_MAX_PIPELINE_LENGTH, long max);
3030
.SH DESCRIPTION
31+
No function since pipelining was removed in 7.62.0.
32+
3133
Pass a long. The set \fBmax\fP number will be used as the maximum amount of
3234
outstanding requests in an HTTP/1.1 pipelined connection. This option is only
3335
used for HTTP/1.1 pipelining, not for HTTP/2 multiplexing.

‎docs/libcurl/opts/CURLMOPT_PIPELINING.3

+5-5
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
.\" * | (__| |_| | _ <| |___
66
.\" * \___|\___/|_| \_\_____|
77
.\" *
8-
.\" * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al.
8+
.\" * Copyright (C) 1998 - 2019, Daniel Stenberg, <daniel@haxx.se>, et al.
99
.\" *
1010
.\" * This software is licensed as described in the file COPYING, which
1111
.\" * you should have received as part of this distribution. The terms
@@ -71,12 +71,12 @@ HTTP(S)
7171
.SH EXAMPLE
7272
.nf
7373
CURLM *m = curl_multi_init();
74-
/* try HTTP/1 pipelining and HTTP/2 multiplexing */
75-
curl_multi_setopt(m, CURLMOPT_PIPELINING, CURLPIPE_HTTP1 |
76-
CURLPIPE_MULTIPLEX);
74+
/* try HTTP/2 multiplexing */
75+
curl_multi_setopt(m, CURLMOPT_PIPELINING, CURLPIPE_MULTIPLEX);
7776
.fi
7877
.SH AVAILABILITY
79-
Added in 7.16.0. Multiplex support bit added in 7.43.0.
78+
Added in 7.16.0. Multiplex support bit added in 7.43.0. HTTP/1 Pipelining
79+
support was disabled in 7.62.0.
8080
.SH RETURN VALUE
8181
Returns CURLM_OK if the option is supported, and CURLM_UNKNOWN_OPTION if not.
8282
.SH "SEE ALSO"

‎docs/libcurl/opts/CURLMOPT_PIPELINING_SERVER_BL.3

+3-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
.\" * | (__| |_| | _ <| |___
66
.\" * \___|\___/|_| \_\_____|
77
.\" *
8-
.\" * Copyright (C) 1998 - 2014, Daniel Stenberg, <daniel@haxx.se>, et al.
8+
.\" * Copyright (C) 1998 - 2019, Daniel Stenberg, <daniel@haxx.se>, et al.
99
.\" *
1010
.\" * This software is licensed as described in the file COPYING, which
1111
.\" * you should have received as part of this distribution. The terms
@@ -28,6 +28,8 @@ CURLMOPT_PIPELINING_SERVER_BL \- pipelining server blacklist
2828

2929
CURLMcode curl_multi_setopt(CURLM *handle, CURLMOPT_PIPELINING_SERVER_BL, char **servers);
3030
.SH DESCRIPTION
31+
No function since pipelining was removed in 7.62.0.
32+
3133
Pass a \fBservers\fP array of char *, ending with a NULL entry. This is a list
3234
of server types prefixes (in the Server: HTTP header) that are blacklisted
3335
from pipelining, i.e server types that are known to not support HTTP

‎docs/libcurl/opts/CURLMOPT_PIPELINING_SITE_BL.3

+3-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
.\" * | (__| |_| | _ <| |___
66
.\" * \___|\___/|_| \_\_____|
77
.\" *
8-
.\" * Copyright (C) 1998 - 2014, Daniel Stenberg, <daniel@haxx.se>, et al.
8+
.\" * Copyright (C) 1998 - 2019, Daniel Stenberg, <daniel@haxx.se>, et al.
99
.\" *
1010
.\" * This software is licensed as described in the file COPYING, which
1111
.\" * you should have received as part of this distribution. The terms
@@ -28,6 +28,8 @@ CURLMOPT_PIPELINING_SITE_BL \- pipelining host blacklist
2828

2929
CURLMcode curl_multi_setopt(CURLM *handle, CURLMOPT_PIPELINING_SITE_BL, char **hosts);
3030
.SH DESCRIPTION
31+
No function since pipelining was removed in 7.62.0.
32+
3133
Pass a \fBhosts\fP array of char *, ending with a NULL entry. This is a list
3234
of sites that are blacklisted from pipelining, i.e sites that are known to not
3335
support HTTP pipelining. The array is copied by libcurl.

‎lib/Makefile.inc

+2-2
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ LIB_CFILES = file.c timeval.c base64.c hostip.c progress.c formdata.c \
5252
openldap.c curl_gethostname.c gopher.c idn_win32.c \
5353
http_proxy.c non-ascii.c asyn-ares.c asyn-thread.c curl_gssapi.c \
5454
http_ntlm.c curl_ntlm_wb.c curl_ntlm_core.c curl_sasl.c rand.c \
55-
curl_multibyte.c hostcheck.c conncache.c pipeline.c dotdot.c \
55+
curl_multibyte.c hostcheck.c conncache.c dotdot.c \
5656
x509asn1.c http2.c smb.c curl_endian.c curl_des.c system_win32.c \
5757
mime.c sha256.c setopt.c curl_path.c curl_ctype.c curl_range.c psl.c \
5858
doh.c urlapi.c altsvc.c
@@ -72,7 +72,7 @@ LIB_HFILES = arpa_telnet.h netrc.h file.h timeval.h hostip.h progress.h \
7272
curl_gethostname.h gopher.h http_proxy.h non-ascii.h asyn.h \
7373
http_ntlm.h curl_gssapi.h curl_ntlm_wb.h curl_ntlm_core.h \
7474
curl_sasl.h curl_multibyte.h hostcheck.h conncache.h \
75-
curl_setup_once.h multihandle.h setup-vms.h pipeline.h dotdot.h \
75+
curl_setup_once.h multihandle.h setup-vms.h dotdot.h \
7676
x509asn1.h http2.h sigpipe.h smb.h curl_endian.h curl_des.h \
7777
curl_printf.h system_win32.h rand.h mime.h curl_sha256.h setopt.h \
7878
curl_path.h curl_ctype.h curl_range.h psl.h doh.h urlapi-int.h \

‎lib/conncache.h

+1-2
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
* | (__| |_| | _ <| |___
88
* \___|\___/|_| \_\_____|
99
*
10-
* Copyright (C) 2015 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al.
10+
* Copyright (C) 2015 - 2019, Daniel Stenberg, <daniel@haxx.se>, et al.
1111
* Copyright (C) 2012 - 2014, Linus Nielsen Feltzing, <linus@haxx.se>
1212
*
1313
* This software is licensed as described in the file COPYING, which
@@ -40,7 +40,6 @@ struct conncache {
4040

4141
#define BUNDLE_NO_MULTIUSE -1
4242
#define BUNDLE_UNKNOWN 0 /* initial value */
43-
#define BUNDLE_PIPELINING 1
4443
#define BUNDLE_MULTIPLEX 2
4544

4645
struct connectbundle {

‎lib/http.c

+2-24
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,6 @@
7373
#include "http_proxy.h"
7474
#include "warnless.h"
7575
#include "non-ascii.h"
76-
#include "pipeline.h"
7776
#include "http2.h"
7877
#include "connect.h"
7978
#include "strdup.h"
@@ -1280,7 +1279,6 @@ CURLcode Curl_add_buffer_send(Curl_send_buffer **inp,
12801279
This needs FIXing.
12811280
*/
12821281
return CURLE_SEND_ERROR;
1283-
Curl_pipeline_leave_write(conn);
12841282
}
12851283
}
12861284
Curl_add_buffer_free(&in);
@@ -3722,16 +3720,9 @@ CURLcode Curl_http_readwrite_headers(struct Curl_easy *data,
37223720
}
37233721
else if(conn->httpversion >= 11 &&
37243722
!conn->bits.close) {
3725-
/* If HTTP version is >= 1.1 and connection is persistent
3726-
server supports pipelining. */
3723+
/* If HTTP version is >= 1.1 and connection is persistent */
37273724
DEBUGF(infof(data,
3728-
"HTTP 1.1 or later with persistent connection, "
3729-
"pipelining supported\n"));
3730-
/* Activate pipelining if needed */
3731-
if(conn->bundle) {
3732-
if(!Curl_pipeline_site_blacklisted(data, conn))
3733-
conn->bundle->multiuse = BUNDLE_PIPELINING;
3734-
}
3725+
"HTTP 1.1 or later with persistent connection\n"));
37353726
}
37363727

37373728
switch(k->httpcode) {
@@ -3816,19 +3807,6 @@ CURLcode Curl_http_readwrite_headers(struct Curl_easy *data,
38163807
data->info.contenttype = contenttype;
38173808
}
38183809
}
3819-
else if(checkprefix("Server:", k->p)) {
3820-
if(conn->httpversion < 20) {
3821-
/* only do this for non-h2 servers */
3822-
char *server_name = Curl_copy_header_value(k->p);
3823-
3824-
/* Turn off pipelining if the server version is blacklisted */
3825-
if(conn->bundle && (conn->bundle->multiuse == BUNDLE_PIPELINING)) {
3826-
if(Curl_pipeline_server_blacklisted(data, server_name))
3827-
conn->bundle->multiuse = BUNDLE_NO_MULTIUSE;
3828-
}
3829-
free(server_name);
3830-
}
3831-
}
38323810
else if((conn->httpversion == 10) &&
38333811
conn->bits.httpproxy &&
38343812
Curl_compareheader(k->p,

‎lib/http2.c

+1-1
Original file line numberDiff line numberDiff line change
@@ -620,7 +620,7 @@ static int push_promise(struct Curl_easy *data,
620620

621621
/*
622622
* multi_connchanged() is called to tell that there is a connection in
623-
* this multi handle that has changed state (pipelining become possible, the
623+
* this multi handle that has changed state (multiplexing become possible, the
624624
* number of allowed streams changed or similar), and a subsequent use of this
625625
* multi handle should move CONNECT_PEND handles back to CONNECT to have them
626626
* retry.

0 commit comments

Comments
 (0)