curl: (55) Failed sending HTTP request when trying to send a request with large header size. >8.1.2 #11405
Comments
|
Could this be related to #10720 which changed internal header handling in 8.0.2?
Note that the -X option in that command should be removed.
|
|
That seems likely. I assume you see different behaviour when adding |
|
@icing yes, adding that is fixing the issue. |
|
We introduced a limit when we refactored the http2 code. Most servers I know have a limit on header length to protect themselves from malicious input. Is this an experiment you are running or do you have a use case for such headers? If yes, what is the limit you are looking for? |
|
I have discovered this, as libcurl is a downstream dependency of a library that I use in (ethon) |
|
A number of popular web servers support up to 16KB headers (a few support even longer), so 16KB is probably a reasonable limit (if libcurl needs to limit them at all).
|
|
If it helps, below is result with Go https://developer.mozilla.org/docs/Web/HTTP/Status/431 package main
import (
"net/http"
"strings"
"time"
)
func main() {
req, err := http.NewRequest("", "https://postman-echo.com/get", nil)
if err != nil {
panic(err)
}
i := 7989
for {
req.Header.Set("MyHeader", strings.Repeat("A", i))
res, err := new(http.Transport).RoundTrip(req)
if err != nil {
panic(err)
}
res.Body.Close()
println(res.Status, i)
time.Sleep(time.Second)
i++
}
}result: |
- not quite to infinity - refs curl#11405 - rewrote the implementation of our internal HTTP/1.x request parsing to work with very large lines using dynbufs. - new default limit is `DYN_HTTP_REQUEST`, aka 1MB, which is also the limit of curl's general HTTP request processing.
|
Thanks. I made #11407 just now which overcomes these limitations. Curl internally has a general buffer limited to 1MB for assembling a HTTP request. This limit is now also in effect for HTTP/2 and HTTP/3 request processing with this PR. |
|
Thank you! |
- not quite to infinity - rewrote the implementation of our internal HTTP/1.x request parsing to work with very large lines using dynbufs. - new default limit is `DYN_HTTP_REQUEST`, aka 1MB, which is also the limit of curl's general HTTP request processing. Fixes curl#11405 Closes curl#11407
- not quite to infinity - rewrote the implementation of our internal HTTP/1.x request parsing to work with very large lines using dynbufs. - new default limit is `DYN_HTTP_REQUEST`, aka 1MB, which is also the limit of curl's general HTTP request processing. Fixes curl#11405 Closes curl#11407
Hi, I am not certain this is a bug, perhaps this was mentioned somewhere, however, I did not discover it, looking at the changelog.
After updating curl from 8.0.1 I have started seeing requests failing to be sent. Further investigation showed, that this is only happening for requests with large header size (10K characters)
I did this
Send a request with large header size. Example:
curl -X GET -L -H "MyHeader: $(printf '%*s' 10000 | tr ' ' 'A')" https://postman-echo.com/getThis is working ok on 8.0.1, but not on 8.1.2.
8.1.2 is returning
curl: (55) Failed sending HTTP requestI expected the following
Response from the server.
curl/libcurl version
Error happening with:
Request is sent successfully with:
operating system
Alpine Linux 3.16 Docker image.
Linux 5.15.49-linuxkitScreenshot
The text was updated successfully, but these errors were encountered: