Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: Allow obtaining original header capitalization #37834

Open
JohnRusk opened this issue Mar 13, 2020 · 7 comments
Open

net/http: Allow obtaining original header capitalization #37834

JohnRusk opened this issue Mar 13, 2020 · 7 comments
Milestone

Comments

@JohnRusk
Copy link

@JohnRusk JohnRusk commented Mar 13, 2020

What version of Go are you using (go version)?

1.13

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Windows, x64

What did you do?

Make a HTTP request, read it's headers, send those headers somewhere else.

What did you expect to see?

Headers are preserved exactly

What did you see instead?

Header capitialization gets canonicalized

I realize this has been discussed before (about 7 years ago).

However, in the time since then many users have been adversely affected by this. E.g.
containous/traefik#466
Azure/azure-storage-azcopy#113

So I'd like to find out, would the Go team consider not a change to the current behaviour, but simply a way for an HTTPResponse to provide an additional map, that maps canonicalizedName -> originalName. If we could just get that map, then those of us who really need header case preservation could use the information it contains to achieve what we need. (Find out the original capitalization when we read a response, and then directly manipulate the outbound request map when we forward that data on).

I'd be happy to contribute the code for the above, if we expected it to be accepted.

@toothrot

This comment has been minimized.

Copy link
Contributor

@toothrot toothrot commented Mar 13, 2020

Hey @JohnRusk! I understand that one of your concerns is with Azure/azure-storage-azcopy#113, in which the API is expecting capitalized headers, despite RFC2616 specifying that header fields are to be treated as case-insensitive.

Issue #5022 was resolved by preserving case on outbound requests:
f0396ca

I don't think it's reasonable to keep a mapping of every header field transformation around for every request. As @bradfitz mentioned, http/2 explicitly lowercases all fields. I feel like making changes here for broken servers (which I understand are out of your control) doesn't seem reasonable.

Is it possible to keep a map of these special header fields around in your application and re-write them on your outbound request? I assume there is a subset that are sensitive, and it is not all HTTP headers. Their casing is preserved on outbound requests as I understand #5022.

/cc @bradfitz @rsc

@JohnRusk

This comment has been minimized.

Copy link
Author

@JohnRusk JohnRusk commented Mar 13, 2020

Is it possible to keep a map of these special header fields around in your application and re-write them on your outbound request?

Not really, because the set of header fields that exists is customer-determined, and may vary from customer to customer and even from file to file.

I don't think it's reasonable to keep a mapping of every header field transformation around for every request.

What about some kind of hook then, some kind of function that get's called as they are pulled of the wire, and before they are canonlicalized. If the hook is null, it doesn't get called.

e.g.(roughly)

headerName := ... <raw value off the wire>
if request.headerNameNotifier != nil {
   request.headerNameNotifier(headerName)
}
headerName := canonalicalize(headerName)

Then customers like us could provide request.headerNameNotifier and everyone else could leave it nil.

@bradfitz

This comment has been minimized.

Copy link
Contributor

@bradfitz bradfitz commented Mar 14, 2020

There are a few dup bugs in this space with a bunch of conversation. I can't find them at the moment (GitHub search is failing me) but they're somewhere.

@JohnRusk

This comment has been minimized.

Copy link
Author

@JohnRusk JohnRusk commented Mar 14, 2020

I had trouble searching too. I have managed to find a few though. Are any of these the ones you were thinking of?

  • #5022 . I see that there you proposed a workaround for writing (which has been implemented, albeit in a different form, if I understand correctly. But there's still no workaround for reading).

  • #18495 I notice there you wrote "I don't want to complicate Go and encourage buggy libraries from assuming case". I think the issue many of us have is that we have to interact with non-Go code that already assumes case, and assumes that we will preserve case (which we can't, when we're coding in Go). To be clear, no other library is assuming that we will be case sensitive, but they are assuming we will be case preserving when we handle their data.

  • #18196

  • #22864

  • Related : #18476

  • Related, but not a dup: #13767

  • Related, I think: #29965

@JohnRusk

This comment has been minimized.

Copy link
Author

@JohnRusk JohnRusk commented Mar 23, 2020

@ACECEO yes, it's a workaround in the sense that it works. No in the sense that it's a totally different product, with a different emphasis and style of usage. For customers who use the tool I work on, AzCopy, this Go issue remains a blocker.

@JohnRusk

This comment has been minimized.

Copy link
Author

@JohnRusk JohnRusk commented Mar 26, 2020

@bradfitz Any update re thoughts on this? As noted above, the key issue is that we currently can't write case-preserving code that reads headers and forwards them on. In our tool, AzCopy, this is a big deal for certain customers who need to move their data around in Azure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.