Fix security issue related to RFC 9112

[4383]

Reject requests which contains headers content-length and transfer-encoding at the same time.

That's not allowed by RFC 9112 and that could lead to potential security attacks.

https://www.rfc-editor.org/rfc/rfc9112#section-6.1-15

This is an extract of the RFC:

A server MAY reject a request that contains both Content-Length and
Transfer-Encoding or process such a request in accordance with the
Transfer-Encoding alone. Regardless, the server MUST close the
connection after responding to such a request to avoid the potential
attacks.

A server or client that receives an HTTP/1.0 message containing
a Transfer-Encoding header field MUST treat the message as if the
framing is faulty, even if a Content-Length is present, and close the
connection after processing the message. The message sender might have
retained a portion of the message, in buffer, that could be
misinterpreted by further use of the connection.

The following request would lead to this scenario:

POST / HTTP/1.1
Host: a.com
Transfer-Encoding: chunked
Content-Length: 0
Content-Type: application/x-"##-form-urlencoded
14
id=1'or sleep(1);###
0

With these changes, when this kind of request is received the connection is closed and an error 400 is returned.

This scenario can be tested by using the following process:

1. run a wsgi server either:

• by using the wsgi sample in official examples (http://eventlet.net/doc/examples.html#wsgi-server)
• or by using my previous patch (Add entry points to allow to run examples simply by installing eventlet #825) that allow you to run wsgi server from the command line ($ eventlet_wsgi_example)

2. send the following HTTP request to the running server:

curl -d "param1=value1&param2=value2" -X POST -H 'Transfer-Encoding:
chunked' -H 'Content-Length: 0' --http1.1 http://0.0.0.0:8090 -i

The previous curl command display returned headers and status code. You can observe that now, with these changes, bad requests are rejected.

These changes also remove content-lenght from the chunk tests to avoid reflecting something that's not a bad practice.

This security issue was originally discovered by Keran Mu (mkr22@mails.tsinghua.edu.cn) and Jianjun Chen
(jianjun@tsinghua.edu.cn), from Tsinghua University and Zhongguancun Laboratory

Thanks to them for raising our attention about this security problem.

[tipabu]

From the RFC:

Early implementations of Transfer-Encoding would occasionally send both a chunked transfer coding for message framing and an estimated Content-Length header field for use by progress bars. This is why Transfer-Encoding is defined as overriding Content-Length, as opposed to them being mutually incompatible.

Given that, should we maybe make this in some way configurable, but defaulting to the more-secure behavior? That would allow us an escape hatch for working with old clients that cannot be updated (for example, because their source code may have been lost), particularly if the operator and/or application developer know that the service will not be forwarding any messages.

I'd also like to note that since we're relying on stdlib's HTTP parsing, there are still cases where a client may send both headers but we don't 400. See also: python/cpython#81274 and #574 (which unfortunately I never properly followed up on).

That said, I agree, the danger is certainly real.

[tipabu]

"MAY"

[4383]

ack

[tipabu]

That seems like a long status line -- any reason not to do something more like

msg = b"Content-Length and Transfer-Encoding are not allowed together\n"
self.wfile.write(
    b"HTTP/1.0 400 Bad Request\r\n"
    b"Connection: close\r\n"
    b"Content-Length: %d\r\n"
    b"\r\n%s" % (len(msg), msg))

? Maybe even throw in a Content-Type?

[4383]

Good idea, you are right, the body of the request is more suited to forward error details. Will update.

[4383]

I have a subsidiary observation, personally I prefer using HTTP/1.0 according to the max_http_version option documentation:

Set to "HTTP/1.0" to make the server pretend it only supports HTTP 1.0. This can help with applications or clients that don't behave properly using HTTP 1.1.

However, during my development I seen that, in responses, the HTTP protocole is hardcoded and sometime the version is set to HTTP 1.0 (the majority of the time) and sometime to HTTP 1.1:

• https://github.com/eventlet/eventlet/blob/master/eventlet/wsgi.py#L746
• https://github.com/eventlet/eventlet/blob/master/eventlet/wsgi.py#L28
• https://github.com/eventlet/eventlet/blob/master/eventlet/wsgi.py#L433
• https://github.com/eventlet/eventlet/blob/master/eventlet/wsgi.py#L439
• https://github.com/eventlet/eventlet/blob/master/eventlet/wsgi.py#L454
• and so on...

Still, according to the previous doc sentence, I think that everything should be set to HTTP/1.0, right?

This could be fixed later through a follow up patch.
Just want to confirm the current case related to my patch.

[4383]

From the RFC:

Early implementations of Transfer-Encoding would occasionally send both a chunked transfer coding for message framing and an estimated Content-Length header field for use by progress bars. This is why Transfer-Encoding is defined as overriding Content-Length, as opposed to them being mutually incompatible.

Given that, should we maybe make this in some way configurable, but defaulting to the more-secure behavior? That would allow us an escape hatch for working with old clients that cannot be updated (for example, because their source code may have been lost), particularly if the operator and/or application developer know that the service will not be forwarding any messages.

Totally agree with the "configurable but defaulting to the more secure behavior" path. I'll rework my patch to introduce an option.

I'd also like to note that since we're relying on stdlib's HTTP parsing, there are still cases where a client may send both headers but we don't 400. See also: python/cpython#81274 and #574 (which unfortunately I never properly followed up on).

Thanks for the heads up.

That said, I agree, the danger is certainly real.

[codecov]

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (dbb411d) 54% compared to head (457e760) 54%.

Files	Patch %	Lines
eventlet/wsgi.py	33%	5 Missing and 1 partial ⚠️

Additional details and impacted files

@@          Coverage Diff          @@
##           master   #826   +/-   ##
=====================================
- Coverage      54%    54%   -1%     
=====================================
  Files          88     88           
  Lines        9744   9752    +8     
  Branches     1814   1816    +2     
=====================================
- Hits         5296   5295    -1     
- Misses       4082   4089    +7     
- Partials      366    368    +2     

Flag	Coverage Δ
ipv6	22% <11%> (-1%)	⬇️
py310epolls	53% <33%> (-1%)	⬇️
py310poll	53% <33%> (-1%)	⬇️
py310selects	52% <33%> (-1%)	⬇️
py311epolls	53% <33%> (-1%)	⬇️
py312epolls	50% <33%> (-1%)	⬇️
py37epolls	50% <33%> (-1%)	⬇️
py38epolls	52% <33%> (-1%)	⬇️
py38openssl	51% <33%> (-1%)	⬇️
py38poll	52% <33%> (-1%)	⬇️
py38selects	52% <33%> (-1%)	⬇️
py39dnspython1	51% <33%> (-1%)	⬇️
py39epolls	52% <33%> (-1%)	⬇️
py39poll	52% <33%> (-1%)	⬇️
py39selects	52% <33%> (-1%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[4383]

Removed to do not reflect and encourage bad practices, I wonder if this test itself could be removed.
Another option, would be to remove the transfer-encoding and rename the test and keep it alive.

Any opinion?

[4383]

Removed to do not reflect and encourage bad practices, I wonder if this test itself could be removed.
Another option, would be to remove the transfer-encoding and rename the test and keep it alive.

Any opinion?

[4383]

Removed to do not reflect and encourage bad practices.

[4383]

Removed to do not reflect and encourage bad practices.

[4383]

Removed to do not reflect and encourage bad practices.

[4383]

codecov is a bit flaky...

[4383]

Is this level of configuration is enough for you?

[4383]

@tipabu is the latest version is ok for you?

[itamarst]

If no answer in a few days should probably just proceed, given it's a security issue.

[4383]

If no answer in a few days should probably just proceed, given it's a security issue.

yes