net/http: allow control of case of header key

[gopherbot]

by tyson@stovepipestudios.com:

http.Header.Add() changes the case of the given header key, which causes
problems. See: http://play.golang.org/p/ci5_qeAYRh

I'm on MacOS X 10.8.2 using Go 1.0.3, but this issue applies to all platforms
and current versions of Go.

I understand that this was done intentionally. The comment above the
http.Request.Header declaration says:

    // HTTP defines that header names are case-insensitive.
    // The request parser implements this by canonicalizing the
    // name, making the first character and any characters
    // following a hyphen uppercase and the rest lowercase.

It's likely referring to section 3.4.7 of RFC 822, which says:

    When matching any other syntactic unit, case is to be ignored.
    For  example, the field-names "From", "FROM", "from", and even
    "FroM" are semantically equal and should all be treated ident-
    ically.

    When generating these units, any mix of upper and  lower  case
    alphabetic  characters  may  be  used.  The case shown in this
    specification is suggested for message-creating processes.

In an ideal world, all servers / remote APIs would respect this guideline when
parsing and handling HTTP headers. However, they don't, which can cause grief
for users of the http package. I spent a fair amount of time myself trying to
figure out why a particular API endpoint wasn't finding and accepting my
authentication header.

It's for this reason that I believe that http.Header.Add() should not change
header key casing *because* the RFC says that casing isn't important. If casing
isn't important, there's no reason to change it, anyways, and users can ensure
that their headers will be handled properly in cases when the remote service
doesn't respect the guideline.

[bradfitz]

Comment 1:

This is the first report I've seen in 2+ years of issues with this design.  It's
impractical to change at this point.
Maybe we could provide some way for you to white-list certain header keys, or to provide
your own WriteRequest function to the Transport.
But that will have to wait until after Go 1.1.

Labels changed: removed priority-triage.

Status changed to Thinking.

[rsc]

Comment 2:

What API endpoint are we talking about? You should also a file a bug with them, since
they have a broken HTTP server.

[rsc]

Comment 4:

Also, can't you just do
header["yourKey"] = whatever
and bypass Add?

[bradfitz]

Comment 5:

Re: #4, indeed. I forgot that worked. (I thought WriteHeader did the canonicalization) 
I added a test in https://golang.org/cl/7712043 to verify we don't break that
going forward.
Then I'll close this.

Owner changed to @bradfitz.

Status changed to Started.

[bradfitz]

Comment 6:

This issue was closed by revision f0396ca.

Status changed to Fixed.

[gopherbot]

Comment 7 by tyson@stovepipestudios.com:

Works for me -- thanks!

[gopherbot]

Comment 8 by dan@ezoic.com:

Example where case is important:  In mod_pagespeed, the header "PageSpeed" to enable or
disable the module for a specific request, is case sensitive.  Two google products not
working nicely together.  :)
In a broader way, if the app is acting like a proxy and the headers are not used
directly by the app but rather passed thru to two other http servers, it may matter.

[bradfitz]

Comment 9:

Please move this discussion to the mailing list.

[marcosnils]

@bradfitz This also happens when you write a response Header from the server side perspective. Here's an example: http://play.golang.org/p/o37Js5lb8f

Are you planning to make this change also for response headers?. Should I open another issue?

Thanks

[minux]

this issue has been closed, please file a new one. Thanks.

[bradfitz]

It's closed but I'll reply again for the record, to unify conversations that have happened in several places:

We won't be changing this. HTTP/2 doesn't even have case in headers.

If you have an application that cares, it's broken, has always been broken, and needs to be fixed sooner than later.

[marcosnils]

Perfect @bradfitz. I do also agree with you that applications should comply with the specs, but as you know that usually doesn't happen 😄.

Thanks for the response.

[elgs]

I have an http server application reading header like this:

r.Header.Get("some_header")

In go1.4.2, the code worked well. However, after upgrading to go1.5 and go1.6, the header was magically changed to Some_header. It's really annoying.

[elgs]

Now I have to change my code everywhere to be like this in order to be compatible with old versions:

apiTokenId := r.Header.Get("api_token_id")
if apiTokenId == "" {
    apiTokenId = r.Header.Get("Api_token_id")
}

I don't understand why the library should manipulate the header key, as if I don't know what I am doing.

[marcosnils]

@elgs just wrap your r.Header.Get around a lowercase and that's it. No need to call Get twice.

[elgs]

@marcosnils thanks, but I don't understand what you meant by wrapping my r.Header.Get around a lowercase. The client puts api_token_id as the header key. On the server side, when I do fmt.Println(r.Header), the shows Api_token_id.

[minux]

Because Header.Get already handles CanonicalHeaderKey transformations for you. This issue has already been closed for some time, please ask questions elsewhere (see https://golang.org/wiki/Questions)

[elgs]

@minux I don't mind talking a bit more here as here is the perfect context for this question. I think CanonicalHeaderKey is a bit too smart and does nothing but causing confusions.

[minux]

HTTP headers are case insensitive, and net/http.Header is a map, so there has to be a canonical case mapping somewhere. And Header.Get already handles all these for you. We don't do discussions on closed issues, so please ask questions on the mailing list. Thanks.

[elgs]

Aha! OK, that makes sense to me. Thanks @minux.