Security considerations: _ in field names

[mnot]

Many implementations normalise - to _ when passing field names to applications, for historical reasons. This is a vector for smuggling attacks; see here.

Suggestions:

1. Consistently recommend that _ not be used in field names.
2. When an implementation normalises - to _, it MUST remove fields whose names contain _; it MAY make them available through an alternative method.

[reschke]

AFAIR, we have discussed this in the past, and decided not to do this. Has anything changed since then?

[mnot]

These attacks have become pretty prominent; I don't think wishing them away is helping. Do you have a ref to the previous discussion?

[reschke]

There is #30 - I also think this was discussed during the last AUTH48, but I may be wrong.

(and yes, I realize that recommended char set and security considerations are not the same thing)

[reschke]

Consistently recommend that _ not be used in field names.

We do that already for new field names; for existing names, there's little one can do, right?

When an implementation normalises - to _, it MUST remove fields whose names contain _; it MAY make them available through an alternative method.

IIUC, a common attack is to override messaging related fields such as Content-Length, Content-Encoding and Transfer-Encoding. For fields like these, wouldn't it be better to recommend dropping the underscore variant?

[royfielding]

Suggestions:

1. Consistently recommend that `_` not be used in field names.

That would have no effect on such attacks. Recommending that neither hyphen nor underscore be used in the names of sensitive DIY header field passing would work, as would recommending that applications choosing to do sensitive DIY field passing using header fields ought to be prepared for normalization of both (and a dozen other weird characters, unicode, CTLs, etc.). But, to be clear, that is not about HTTP.

2. When an implementation normalises `-` to `_`, it MUST remove fields whose names contain `_`; it MAY make them available through an alternative method.

These are not implementations of HTTP. When we are talking about gateways to other protocols (like CGI), then yes. There is no historical normalization that occurs from HTTP-to-HTTP. There is a spec for HTTP-to-CGI that does normalization because of restrictions in env variables, and that's what needs updating. The problem comes when an implementation that used to be behind a CGI interface is moved to be behind an HTTP interface, and the damn kids think that should work without any changes in the application code. Well, app platforms often get this wrong. It is not an HTTP issue.

At most, we can warn application platforms that HTTP != CGI and that trying to "handle" an HTTP message as if it were received by CGI requires that the platforms implement CGI normalization properly.

An HTTP gateway sending a message to an HTTP application platform has no way of knowing that the receiving implementation isn't expecting HTTP, and there's no way in hell that we are going to castrate HTTP just in case it happens to be received by a poor app implementation. HTTP is not CGI.

[mnot]

We do that already for new field names; for existing names, there's little one can do, right?

We don't explicitly mention this issue in Considerations for New Field Names

IIUC, a common attack is to override messaging related fields such as Content-Length, Content-Encoding and Transfer-Encoding. For fields like these, wouldn't it be better to recommend dropping the underscore variant?

Yes.

That would have no effect on such attacks.

On its own, no - but these measures are designed to work together.

These are not implementations of HTTP [...] It is not an HTTP issue.

Given how widespread these practices are, I don't think it's responsible for us to take that stance.

At most, we can warn application platforms that HTTP != CGI and that trying to "handle" an HTTP message as if it were received by CGI requires that the platforms implement CGI normalization properly.

I'm fine with avoiding normative language here, but I do think we need to go into the details. I'll do a PR.