Should servers interpret Transfer-Encoding in 1.0 requests?

[mattiasgrenfeldt]

The Transfer-Encoding header was first introduced in version 1.1. So one would assume that 1.0 servers should not interpret it. But in RFC 7230 the following paragraph was introduced under section 2.6:

The interpretation of a header field does not change between minor
versions of the same major HTTP version, though the default behavior
of a recipient in the absence of such a field can change. Unless
specified otherwise, header fields defined in HTTP/1.1 are defined
for all versions of HTTP/1.x. In particular, the Host and Connection
header fields ought to be implemented by all HTTP/1.x implementations
whether or not they advertise conformance with HTTP/1.1.

This paragraph suggests that 1.0 servers should interpret the Transfer-Encoding header. This paragraph also exists in the current draft version of the Semantics spec. Later in RFC 7230 however, under section 3.3.1, we see the following paragraph:

Transfer-Encoding was added in HTTP/1.1. It is generally assumed
that implementations advertising only HTTP/1.0 support will not
understand how to process a transfer-encoded payload. A client MUST
NOT send a request containing Transfer-Encoding unless it knows the
server will handle HTTP/1.1 (or later) requests; such knowledge might
be in the form of specific user configuration or by remembering the
version of a prior received response. A server MUST NOT send a
response containing Transfer-Encoding unless the corresponding
request indicates HTTP/1.1 (or later).

This paragraph suggests that it is ok for implementations that only support 1.0 to not implement the Transfer-Encoding header.

Taken together, these two paragraphs tell us that servers which implement both 1.0 and 1.1 should in their 1.0 implementation interpret Transfer-Encoding. While servers which only implement 1.0 should not. Let's call these two different behaviors the 1.1-behavior and the 1.0-behavior respectively.

If we have a setup with a reverse proxy (gateway) which has the 1.0-behavior in front of a server with the 1.1-behavior, then HTTP Request Smuggling is possible, but both systems follow the specification. If the following request is sent to the proxy:

GET / HTTP/1.0
Host: example.com
Content-Length: 50
Transfer-Encoding: chunked

0

GET /smuggled HTTP/1.0
Host: example.com

Then the proxy will ignore the Transfer-Encoding header, but forward it as-is. The proxy will instead interpret the Content-Length header and see the second GET as the body of the first one. The server will instead interpret and prioritize the Transfer-Encoding header and will therefore see two different requests.

While looking for Request Smuggling in various proxies and servers we have observed three different behaviors in the wild.

• Most systems adopt the 1.1-behavior.
• One server adopted the 1.0-behavior.
• One proxy adopts a mix between the 1.1-behavior and the 1.0-behavior. The system ignores Transfer-Encoding but never forwards it together with the Content-Length header, making the smuggling attack impossible.

If a proxy with the 1.0-behavior is discovered, then request smuggling is possible, but there is no bug, since both parties follow the RFC. Should the spec somehow be changed/clarified to avoid this?

(Authors: Mattias Grenfeldt and Asta Olofsson)

[reschke]

Interesting issue.

AFAIU, we really can't make any new requirements on servers that only support 1.0.

[mnot]

Julian is correct; in this specification we can't retrofit new requirements onto HTTP/1.0 intermediaries.

However, note that the RFC-to-be says:

3. If a message is received with both a Transfer-Encoding and a Content-Length header field, the Transfer-Encoding overrides the Content-Length. Such a message might indicate an attempt to perform request smuggling (Section 11.2) or response splitting (Section 11.1) and ought to be handled as an error.

... so the server in your scenario is already encouraged to handle that as an attack.

Is that sufficient? Arguably we could increase that 'ought' to a SHOULD or even a MUST.

[reschke]

@mattiasgrenfeldt ?

[asta12]

Sorry for the late response.

In the 6 proxies and 6 servers we have looked for Request Smuggling in, only one server rejects requests with both Content-Length and Transfer-Enconding. So even though the current spec warns of an attack, that advise is generally not followed.

Changing the spec to reject requests with both headers would be very good for stopping Request Smuggling issues.

But we can only speak from the Request Smuggling point of view. There might be other reasons to not make that change.

(Asta Olofsson & Mattias Grenfeldt)

[reschke]

So the spec already warns about that case. Do you really believe if another change in the spec would affect those implementations that do not follow the current spec?

Maybe it would be better to open bug reports and see how the developers answer?

[mattiasgrenfeldt]

Here is the current paragraph:

  If a message is received with both a Transfer-Encoding and a
  Content-Length header field, the Transfer-Encoding overrides the
  Content-Length.  Such a message might indicate an attempt to
  perform request smuggling (Section 9.5) or response splitting
  (Section 9.4) and ought to be handled as an error.  A sender MUST
  remove the received Content-Length field prior to forwarding such
  a message downstream.

Source

When we read the current spec, we interpreted it as being optional to reject messages with both headers. Partly because the spec says that one header should be prioritized over the other. This information isn't needed if requests with both headers should be rejected. Partly also because 'ought' is not one of the keywords (SHOULD, MUST etc.) that are defined in the beginning. But we realize when reading the paragraph now that this actually is a requirement.

Considering that it is rather unclear right now that this is a hard requirement, rather than an optional requirement, we believe that adding a stricter keyword would make the situation more clear for implementors.

With the current formulation of the paragraph we think that server authors would respond with: "We chose to accept messages with both headers to be in line with the Robustness Principle".

[reschke]

With the current formulation of the paragraph we think that server authors would respond with: "We chose to accept messages with both headers to be in line with the Robustness Principle".

Or, maybe they didn't read the spec.

That's why I suggested to actually open bugs and see how the authors reply.

[mattiasgrenfeldt]

Many authors do not read the spec closely, indeed. And if an issue is raised on their projects, they might change the behavior.

But there are also projects which do closely read the spec and have yet chosen to accept requests with both headers. Such as Apache HTTP Server (httpd), Nginx and HAProxy. What we wrote before about how authors will probably ignore the issue and quote the Robustness Principle was really meant about these projects.

Looking again at the paragraph from the spec, the third sentence also suggests that it is ok to receive requests with both headers to begin with:

A sender MUST remove the received Content-Length field prior to forwarding such a message downstream.

So if the correct interpretation of this paragraph is meant to be "do not accept requests with both headers" then it could definitely be made more clear.

Regarding opening issues in projects, we don't have the time to do that, sorry. :( But as you say, it would be interesting to see how the authors respond.

---

As a side note, our goal with opening this issue was to bring this strange situation regarding Transfer-Encoding in 1.0 requests to your attention. We don't have any idea for how this should be solved in a good way.

[mnot]

I've pinged the list to get some implementer input: https://www.w3.org/mid/09EACA5C-E560-4C76-A596-8F7C3D890766@mnot.net

[wtarreau]

I just checked, and in haproxy we do pass the transfer-encoding header to the other side even in for 1.0 requests. That's not particularly pretty, I admit, but at least it does resist to this specific case.

What we should probably state is that transfer-encoding ought to simply be rejected in 1.0 requests. It's too dangerous to let such requests come in when they can add a fake body to methods that do not usually take one. I'd possibly even go as far as not accepting a body at all with GET/HEAD in 1.0, because even a content-length with a GET could possibly result in some smuggling attacks on 1.0 servers that do not even look at content-length with GET.

Now another question is what to do for 1.0 responses. Because while requests can be dangerous, a bad acting server could also be used to poison a cache. I'm not totally enthousiast at the idea of blocking T-E on 1.0 responses given that the transition from 1.0 to 1.1 was very slow and we find about all flavors of 1.1 advertised as 1.0.

Maybe a reasonable approach would be to suggest that any 1.0 message containing Transfer-Encoding must ignore both transfer-encoding and content-length, and must work in close mode ? A gateway would then not consume a request body, and would pass all response body until the connection closes. That could be the most conservative approach and possibly the least prone to breaking existing deployments.

[icing]

I am unable to reproduce the described behaviour with a recent 2.4.x Apache httpd. Do you have a working file that you send against Apache to trigger 2 responses?

According to my analysis, in Apache "Content-Length" overrides any "Transfer-Encoding: chunked". So, in your example, it would never see the second request. Assuming this not being the case, it would still close the connection after answering the first request, as your example does not trigger a keep-alive behaviour. But even if I add that to the request, the connection is still being closed after seeing the mixed CT+TE headers.

Reading this again: are you only talking about Apache httpd proxying such requests against a backend that is vulnerable?

[wtarreau]

Yes Stefan, that's what Matthias is testing. In fact not exactly against a vulnerable backend but against any backend which could interpret the request differently. Based on your description I guess that if you turn to close mode there shouldn't be any issue though.

[icing]

Thanks @wtarreau for clarifying. I guess the "smuggled" request could still order 50 refrigerators?

[icing]

tested a http proxy setup with a slightly modified example:

GET /proxy/files/empty.txt HTTP/1.0
Host: cgi.tests.httpd.apache.org
Content-Length: 50
Connection: Keep-Alive
Transfer-Encoding: chunked

3
xxx
0

GET /smuggled HTTP/1.0
Host: example.com

The request is forwarded as (wireshark):

    GET /files/empty.txt HTTP/1.1\r\n
    Host: cgi.tests.httpd.apache.org\r\n
    X-Forwarded-For: 127.0.0.1\r\n
    X-Forwarded-Host: cgi.tests.httpd.apache.org\r\n
    X-Forwarded-Server: cgi.tests.httpd.apache.org\r\n
    Content-Length: 3\r\n
    Connection: Keep-Alive\r\n
    \r\n
    xxx

and the response to the client delivered as:

HTTP/1.1 200 OK
Date: Tue, 13 Jul 2021 10:09:08 GMT
Server: Apache/2.5.1-dev (Unix) OpenSSL/1.1.1k
Last-Modified: Tue, 13 Jul 2021 09:08:56 GMT
ETag: "0-5c6fd96685a00"
Accept-Ranges: bytes
Content-Length: 0
Content-Type: text/plain
Connection: close

The smuggled request is never inspected or processed.