Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
net/http: send User-Agent in proxy CONNECT #13290
I am using a http proxy to connect to an https target. This proxy forbids connections from unknown user-agents.
But even with this change in the user-agent, I am getting a 403 Forbidden answer from the proxy. When dumping the http connection with wireshark, I see that the go http stack is using the default user-agent instead of the one I specified on the request.
In this function, a new request is being built to the CONNECT to the proxy, with a new empty header being allocated. This header is only filled with the proxy authentication if any. I would expect this Header to be filled, as well, with the elements set in the upper request. Alternatively, the Header should be copied from the other request and the proxy authentication later appended.
Go Version: go1.5.1 linux/amd64
The question of what headers to send to proxy is indeed an interesting one, and a good place to see this issue develop is in the CURL library, which (after some iterations, and a vulnerability) figured out what I think is the correct approach.
You simply have to specify explicitly the headers that you want sent to proxy.
There are several ways to achieve this, the most trivial being:
Going in a direction @minux suggests - defining a fixed (and hidden) list of headers which are also sent to proxy feels like a move in the wrong direction because the behavior is hard to debug for user, and the list of headers and their impact (over time) can be dangerous -- what others headers will we add, how will their function change over time, etc.
I understand that introducing a new field to Request struct is not something to be taken lightly, and I'll be happy to hear other thoughts, as well as help with the implementation and testing of this feature.
My understanding is that browsers will usually send only the User-Agent header to the proxy, so if the goal is to satisfy the ACL on the proxy that will most likely suffice. Even though Squid for example allows ACL also on Referer and MIME headers (in addition to User-Agent).
Mostly solving by whitelisting means we decide which of the headers it is OK to share with the proxy, but we lose the ability to share certain headers only with the proxy and not with destination host.
I don't know if that is something other people might find useful (but some do).
changed the title
net/http: CONNECT to proxy ignores request headers, namely User-Agent
Dec 28, 2015
This is more involved than I thought it might be: it's not simply a matter of adding the User-Agent value to the empty header map, since the original request is not in scope where we dial. And we can't simply pass the original *http.Request to the dialConn code because of socket late binding.
I think we'll probably have to address this with new API surface, but that would have to wait until Go 1.7.
As a workaround in the meantime, perhaps you can just set the