wsgi response header handling disregards RFC7230 with regard to an HTTP 204 response code #746

juliakreger · 2022-01-13T18:42:12Z

RFC7230 specifies that transfer encoding must not include a Transfer-Encoding header in an HTTP 204 response. However, eventlet appears to always do so by default based upon the class of request, and if the response does not have a content-length field explicitly defined. Then regardless of the response code, it is stamping the reply with a Transfer-Encoding header.

A server MUST NOT send a Transfer-Encoding header field in any response with a status code of 1xx (Informational) or 204 (No Content). A server MUST NOT send a Transfer-Encoding header field in any 2xx (Successful) response to a CONNECT request (Section 4.3.6 of [RFC7231]).

Specifically, as noted, it is basing it's response on the presence of a Content-Length header. However, HTTP 204 responses must not have a Content-Length header based upon the RFC.

A server MUST NOT send a Content-Length header field in any response with a status code of 1xx (Informational) or 204 (No Content). A server MUST NOT send a Content-Length header field in any 2xx (Successful) response to a CONNECT request (Section 4.3.6 of [RFC7231]).

This creates a situation where, as previously mentioned the Transfer-Encoding header is injected, and a HTTP/1.1 client can find itself confused, and actually issue an error or exception as a result of the response because a client interpreting the Transfer-Encoding before the result code is expecting more data. That data, however, will never come and depending on the application, the socket may close as well, ultimately generating additional errors.

@dtantsur was kind enough to whip up a reproducer utilizing Flask and eventlet wsgi:

import eventlet
import eventlet.wsgi
import flask

app = flask.Flask(name)

@app.route('/', methods=['PUT', 'DELETE'])
def test():
return flask.make_response('', 204)

if name == 'main':
sock = eventlet.listen(('127.0.0.1', 5000))
eventlet.wsgi.server(sock, app, debug=True)

Test and result:

$ curl -vX DELETE http://localhost:5000/

Trying ::1:5000...

connect to ::1 port 5000 failed: Connection refused

Trying 127.0.0.1:5000...

Connected to localhost (127.0.0.1) port 5000 (#0)

DELETE / HTTP/1.1
Host: localhost:5000
User-Agent: curl/7.74.0
Accept: /

Mark bundle as not supporting multiuse
< HTTP/1.1 204 NO CONTENT
< Content-Type: text/html; charset=utf-8
< Date: Thu, 13 Jan 2022 18:33:20 GMT
< Transfer-Encoding: chunked
<

Excess found: excess = 5 url = / (zero-length body)

Connection #0 to host localhost left intact
$

Curl doesn't return a non-zero return code in this precise reproducer case, but you can see that the header is injected inappropriately when compared to RFC7230. This has varying effects on different client implementations. In one case we encountered this was aiding in Apache httpd returning a incomplete response error to the end client. In another we found python-requests trying to parse the response body and an exception being raised as a result there.

This was observed on Eventlet 0.30.2 and comparing the code to the present development branch indicates it is still present.
Both encountered on Python 3.8.5 and Python 3.6.8. Debian and Red Hat Enterprise Linux, respectively.

Ideally, the injection of headers would be skipped if the HTTP result code is 204.

temoto · 2022-01-13T20:05:56Z

Thank you, I'll fix it.

juliakreger · 2022-01-13T20:10:07Z

@temoto Much appreciated! Thanks!

… response with a status code of 1xx, 204 or (2xx to CONNECT request) https://httpwg.org/specs/rfc7230.html#rfc.section.3.3.1 #746

It turns out that eventlet has been injecting a ``Transfer-Encoding`` header as of recent into WSGI application response headers. The result of this ultimately depends on how the HTTP client which is passing the request to the server is written to handle data. Apache, for example, will return that an invalid response was received. In part because it sees the request end, with an HTTP 204 response code, but also an encoding indicating there is a multipart body encoding inbound. Which is confusing. Other C based HTTP clients can have any number of reactions up to and including disconnecting sessions. Curl, depending on the headers present either returns success but notes body weirdness or actually returns return code 18. Python-Requests kind of has it a little worse, and we see this with clients. With it, it tries to prepare a respones content body based upon the presence of the header indicating there is a body. But it blows up thinking there is more data to read on the socket when there is not more data to read. Regardless, all of this is an RFC7230 violation. Neither Content-Length nor Transfer-Encoding should be on an HTTP 204 response. However, Content-Length is the lesser evil, and we have a similar endpoing in Ironic which *does* explicitly get returned with a zero length content-length, and does not demonstrate such issues. As such, in the interest of the lesser evils until Eventlet's evil ways of header injection are remedied, we're explicitly going to force a Content-Length header to be sent indicating a zero length response. For more information, please see: eventlet/eventlet#746 Change-Id: I014cc65c79222f4d4d7c2b6ff11a76e56659340c

* Update ironic-inspector from branch 'master' to 55e47c630beb99f880045113e4f86bc1817b50da - Return a content-length on HTTP204 to prevent client failures It turns out that eventlet has been injecting a ``Transfer-Encoding`` header as of recent into WSGI application response headers. The result of this ultimately depends on how the HTTP client which is passing the request to the server is written to handle data. Apache, for example, will return that an invalid response was received. In part because it sees the request end, with an HTTP 204 response code, but also an encoding indicating there is a multipart body encoding inbound. Which is confusing. Other C based HTTP clients can have any number of reactions up to and including disconnecting sessions. Curl, depending on the headers present either returns success but notes body weirdness or actually returns return code 18. Python-Requests kind of has it a little worse, and we see this with clients. With it, it tries to prepare a respones content body based upon the presence of the header indicating there is a body. But it blows up thinking there is more data to read on the socket when there is not more data to read. Regardless, all of this is an RFC7230 violation. Neither Content-Length nor Transfer-Encoding should be on an HTTP 204 response. However, Content-Length is the lesser evil, and we have a similar endpoing in Ironic which *does* explicitly get returned with a zero length content-length, and does not demonstrate such issues. As such, in the interest of the lesser evils until Eventlet's evil ways of header injection are remedied, we're explicitly going to force a Content-Length header to be sent indicating a zero length response. For more information, please see: eventlet/eventlet#746 Change-Id: I014cc65c79222f4d4d7c2b6ff11a76e56659340c

It turns out that eventlet has been injecting a ``Transfer-Encoding`` header as of recent into WSGI application response headers. The result of this ultimately depends on how the HTTP client which is passing the request to the server is written to handle data. Apache, for example, will return that an invalid response was received. In part because it sees the request end, with an HTTP 204 response code, but also an encoding indicating there is a multipart body encoding inbound. Which is confusing. Other C based HTTP clients can have any number of reactions up to and including disconnecting sessions. Curl, depending on the headers present either returns success but notes body weirdness or actually returns return code 18. Python-Requests kind of has it a little worse, and we see this with clients. With it, it tries to prepare a respones content body based upon the presence of the header indicating there is a body. But it blows up thinking there is more data to read on the socket when there is not more data to read. Regardless, all of this is an RFC7230 violation. Neither Content-Length nor Transfer-Encoding should be on an HTTP 204 response. However, Content-Length is the lesser evil, and we have a similar endpoing in Ironic which *does* explicitly get returned with a zero length content-length, and does not demonstrate such issues. As such, in the interest of the lesser evils until Eventlet's evil ways of header injection are remedied, we're explicitly going to force a Content-Length header to be sent indicating a zero length response. For more information, please see: eventlet/eventlet#746 Change-Id: I014cc65c79222f4d4d7c2b6ff11a76e56659340c (cherry picked from commit 55e47c6)

temoto added feature-wsgi importance-bug labels Jan 13, 2022

temoto added a commit that referenced this issue Jan 13, 2022

wsgi: server MUST NOT send Content-Length/Transfer-Encoding header in…

ab746cb

… response with a status code of 1xx, 204 or (2xx to CONNECT request) https://httpwg.org/specs/rfc7230.html#rfc.section.3.3.1 #746

temoto linked a pull request Jan 13, 2022 that will close this issue

wsgi: server MUST NOT send Content-Length/Transfer-Encoding header in response with a status code of 1xx, 204 or (2xx to CONNECT request) #747

Open

bt90 mentioned this issue Dec 22, 2023

UI - 304 'Not Modified' cached handling causes some requests to fail, can affect reverse-proxies dgtlmoon/changedetection.io#2053

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

wsgi response header handling disregards RFC7230 with regard to an HTTP 204 response code #746

wsgi response header handling disregards RFC7230 with regard to an HTTP 204 response code #746

juliakreger commented Jan 13, 2022

temoto commented Jan 13, 2022

juliakreger commented Jan 13, 2022

wsgi response header handling disregards RFC7230 with regard to an HTTP 204 response code #746

wsgi response header handling disregards RFC7230 with regard to an HTTP 204 response code #746

Comments

juliakreger commented Jan 13, 2022

temoto commented Jan 13, 2022

juliakreger commented Jan 13, 2022