Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
proposal: net/http: header fields order #24375
We have a transparant http proxy server, used for calculating statistics, not modifying requests.
Keeping the original order will probably be hard to do? However if it could be defined, people could work around this by defining the default order of the user agent.
What is the actual problem this kind of fingerprinting causes for you? It's unlikely to be prevented by sorting the headers. There is probably enough specific behaviour in Go's HTTP and/or TLS libraries (and every single other implementation, FWIW) to allow for an educated guess regardless of the order of sent HTTP headers.
There is indeed many specific behaviour, that comes from default implementations (e.g. keep alive periods, supported cipher suites,...), but these are way more subtle, headers aren't (and most of these can be overwritten, headers can't).
Also, bot-detection services check for header order, to verify if the request came from an actual browser, since major browsers send their headers in a specific order. Thus every browser request passing through the golang proxy gets blocked by such hosted server.
Apart from that, I don't see why a request that has been proxied by golang code should come out with scrambled headers. It completely takes away the transparency of the server.
The only way to do this, considering a lot of our existing API, would be to overload the net/http.Header type and shove the order in there somewhere.
Since the representation of a
Fortunately we've gone down that ugly-ish path before with net/http.TrailerPrefix:
The godoc doesn't explain why that value is safe, but it's safe because it has a colon in it, and colons aren't valid in header field names.
So we could do something like "Header-Order:".
Assuming we do that, there are two paths to consider: serialization, and parsing.
Serializing is easy. We just modify WriteSubset to respect the order instead of sorting.
For parsing, we might want it to be opt-in to not pay the cost each time & to not confuse code that's not ready for that weirdo key to be present. The question is where to put that opt-in mechanism.
In a Handler, it's too late. A Server option is kinda too broad.
It could be kept always in the internal responseWriter and made available via a special function that takes the ResponseWriter interface and returns it, I suppose.
That solves the problem for sending the response back to the client. However while executing requests, you get stuck on this line, from the Transport's RoundTrip:
Should the Transport have an extra option as well?
Getting back to the original question:
It seems like it's actually hard. It's invasive to the APIs and we're not sure where to stuff the data. We can put the order into Header["Header-Order:"] or some hack like that, but we can't set that by default, for fear of confusing old code. So everything carrying an http.Header has to also have a new field listing the order, and code that wants to set the order has to copy that into the map. That means http.Request and http.Response both need a new field, modify ReadRequest, ReadResponse, Header.WriteSubset. And any code iterating over the header separately would have to know not to look at this fake header, at least when people have opted in to setting that fake header.
This seems pretty hard for not much benefit. Let's leave this for a future rethink of the HTTP library, which is #5465.