transfer: close connection after excess data has been read

[mkauf]

For HTTP 1.x, it's a protocol error when the server sends more bytes than announced. If this happens, don't reuse the connection, because the start position of the next response is undefined.

I'm not sure whether this is also the case for other protocols, so please review carefully.

Test:

response.sh:

#!/bin/sh
printf "%s\r\n" "HTTP/1.1 200 OK"
printf "%s\r\n" "Content-Length: 13"
printf "%s\r\n" ""
printf "%s\r\n" "response 1 - excess data"
sleep 5
printf "%s\r\n" "HTTP/1.1 200 OK"
printf "%s\r\n" "Content-Length: 12"
printf "%s\r\n" ""
printf "%s\r\n" "response 2"
printf "%s\r\n" "HTTP/1.1 200 OK"
sleep 5
printf "%s\r\n" "Content-Length: 21"
printf "%s\r\n" ""
printf "%s\r\n" "response 3 - finish"

Start the test server:

./response.sh | nc -l 9090

Send three requests with curl:

curl http://127.0.0.1:9090/request1 http://127.0.0.1:9090/request2 http://127.0.0.1:9090/request3

Output:

response 1 - response 2
curl: (1) Received HTTP/0.9 when not allowed

So the second response is processed, because after reading the excess bytes, the next response line happens to be "HTTP/1.1 200 OK". The third response is not processed, because the next response line is "Content-Length: 21" - even though the size of the second response is announced correctly by the server. Whether curl processes the response depends on the position of the "sleep" command in the response script.

[bagder]

This seems entirely correct and I'm a little bit puzzled why we didn't do this already!

Did you look into adding a test case for this? I would be so good to make sure that we do right and have that verified with a test!

[mkauf]

Thanks for the feedback! I will work on a testcase, but I need some time.

[bagder]

I had a go at a test case for this in #5458

[bagder]

I could verify your fix with my test so I think it's fine to merge.

The test it self has problems so we can't merge that until it get fixed - if I can figure out a fix. Otherwise I'll just have to discard it.

[bagder]

Thanks!