net/http: document how to create client CONNECT requests

[Audrey-Randall]

What version of Go are you using (go version)?

Go1.9

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

windows/amd64

What did you do?

This error is reproducible at https://play.golang.org/p/9m7rlnqsTm

I attempted to create a new HTTP request with http.NewRequest() using a URI that starts with a number, with no protocol, that also contains a port:
_, err = http.NewRequest(http.MethodConnect, "0.docs.google.com:443", nil)

What did you expect to see?

The request should be created with no error.

What did you see instead?

I get the error "parse : first path segment in URL cannot contain colon"

[bradfitz]

This is actually working as intended. "host:port" is not a valid URL, and http.NewRequest takes a URL, not a host:port.

For a CONNECT request (which is a weirdo type of request), you'll probably just want to construct the *http.Request by hand. It's specially recognized by some but not all of net/http helpers. In particular, http.NewRequest does not go out of its way to support CONNECT weirdness.

[bradfitz]

Actually, I suppose we could also make NewRequest special-case CONNECT and also support host:port there.

@tombergan, thoughts?

[sergeyfrolov]

Thanks for reopening this!

I would like to note that it's not necessarily about CONNECT. If user wants to specify port for other types of requests, say, http.MethodGet, then behavior is the same:

• "0google.com:443" fails
• "google.com:443" doesn't fail

[bradfitz]

@sergeyfrolov, for GET requests or anything other than CONNECT, we will not change behavior. "foo.com:443" is not a valid URL. You need a scheme (http://, https://, etc).

[tombergan]

I would like to note that it's not necessarily about CONNECT. If user wants to specify port for other types of requests, say, http.MethodGet, then behavior is the same:

"0google.com:443" fails
"google.com:443" doesn't fail

That's because "google.com:443" doesn't parse how you think it does ... it parses "google.com" into the scheme, not the host. If "google.com:443" is working for you, that's accidental rather than intentional.
https://play.golang.org/p/omOEwB02Me

Actually, I suppose we could also make NewRequest special-case CONNECT and also support host:port there.

I don't think this would work, for two reasons. First, what scheme does the following example use when connecting to foo.com?

// Assume that NewRequest special-cases CONNECT to support host:port.
// req.URL = {Scheme: ??? , Host = "google.com:443"}
req, _ := http.NewRequest("CONNECT", "google.com:443", nil)

// Connect over TCP to www.proxy.com, which will forward the TCP connection
// to google.com:443. Then, do we TLS to google.com:443 or use raw TCP?
// The :443 suggests that the caller wants TLS, but in general we cannot know
// since req.URL does not have a scheme.
tr := http.Transport{Proxy: http.ProxyURL("http://www.proxy.com/")}
tr.RoundTrip(req)

Second, the following would be confusing:

// Assume that NewRequest special-cases CONNECT to support host:port.
// Note that r1.Host != r2.Host
// r1.URL = {Scheme: ??? , Host = "google.com:443"}
// r2.URL = {Scheme: "google.com", Opaque: "443", Host = ""}
r1, _ := http.NewRequest("CONNECT", "google.com:443", nil)
r2, _ := http.NewRequest("GET", "google.com:443", nil)

@sergeyfrolov's example is a good one. It is confusing that "0google.com:443" fails but "google.com:443" does not. More precisely:

// This is what happens currently.
// r1.URL = {Scheme: "google.com", Opaque: "443", Host = ""}
// r1.URL = nil
r1, _ := http.NewRequest("CONNECT", "google.com:443", nil)
r2, _ := http.NewRequest("CONNECT", "0google.com:443", nil)

Maybe it would be sufficient to fail both of the above http.NewRequest calls? We could do this by adding a requirement to http.NewRequest: when the method is "CONNECT", the URL must contain a non-empty host. In theory, this could break callers ... I can't imagine there are any callers that legitimately want to create a CONNECT request with an empty host, but the possibility is there.

[bradfitz]

I don't think this would work, for two reasons. First, what scheme does the following example use when connecting to foo.com?

Oh, right.

Okay, there's nothing we can do here except documentation.

I'll repurpose this as a documentation bug, to make it clear how to construct CONNECT requests.