Add setting to control parsing of headers

[richdougherty]

In Play 2.6 we switched from Netty to Akka HTTP as the default HTTP backend. This has led to a number of bugs because of the different way that Akka HTTP handles headers compared to Netty. Netty headers are raw string whereas Akka HTTP uses type-safe objects for many headers. These type safe objects are parsed from the underlying header values according to the their specification.

Type-safe headers are a good idea in principle but it runs into problems when:

1. Play users need to send/receive non-spec compliant headers (HTTP header 'Authorization: XXXXXXXX' is not allowed in responses playframework/playframework#7549)
2. Akka HTTP isn't currently spec-compliant (Fix Content-Disposition filename handling playframework/playframework#7515, Illegal header: Illegal 'user-agent' playframework/playframework#7997, Incorrect Filename when downloading from Play 2.6 application Ask Question playframework/playframework#7501)
3. Akka HTTP is spec compliant, but there is more than one way to be spec compliant and Play users need to send/receive headers in a specific way (Provide raw headers when using akka-http backend playframework/playframework#7737, Akka HTTP backend strips quotes out of link headers playframework/playframework#7733, Authorization header contents are modified (double-quotes are removed) #966, Support multipart Content-Type in results playframework/playframework#7498)

Akka HTTP does make it possible to use RawHeaders. Play uses RawHeaders for HTTP responses, but we can't use RawHeaders when receiving requests, since the header objects are created internally by Akka HTTP inside the header parsing code.

My suggestion is that we add a new setting to Akka HTTP to give Play (and other users) more control over how HTTP headers are parsed. Here are a few ideas for how a setting could work:

1. A Boolean setting that switches Akka HTTP into "raw" mode - all headers are parsed as RawHeaders.
2. A String => Boolean setting that maps header names to a Boolean value, giving more fine-grained control over which headers can be parsed into typed headers and which should be parsed as RawHeaders.
3. A HeaderParser => HeaderParser (e.g. where type HeaderParser = (String, String) => Either[ErrorResult, HttpHeader]) setting that allows the user to override the default HeaderParser logic and patch their own logic in. This trivially allows replacing all headers with ReadHeaders. Users would also be able to continue to use typed headers and even alter the parsing logic for those headers if they need.

[jrudolph]

Thanks, @richdougherty for filing this ticket. We also thought about moving more of the header parsing out of the core to the higher levels. I guess for now (and for play's purposes) the simplest thing would be to add the simple boolean setting that disables header parsing for most headers (a few basic ones like Content-Length, Content-Type etc need to be parsed for consumption in the core).

[jrudolph]

Fixed by #1577 which introduces a flag implementing the first option of adding a setting to disable parsing of all headers.

[richdougherty]

Thanks @jrudolph!