net/http: allow processing requests without Host header

[awy]

What version of Go are you using (go version)?

$ go version
go version go1.15.7 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Not relevant

What did you do?

Run up an HTTP server using net/http.Server.ListenAndServe()

Send an illegal HTTP/1.1 request (missing Host header):

POST /endpoint HTTP/1.1
Content-Type: application/octet-stream
Content-Length: 12

Some content

I also looked at #35283 as being a possible solution but clearly not as it has been rejected.

What did you expect to see?

Valid response to request.

I am trying to write a server application that will accept these (invalid) requests from client software over which I have no control. Some of the clients in the field no longer have firmware updates available and so no fix can be expected there. I am looking for a means to modify the behaviour of only a small part of the standard http server.

What did you see instead?

Error response generated by server.go:(c *conn)readRequest()

	hosts, haveHost := req.Header["Host"]
	isH2Upgrade := req.isH2Upgrade()
	if req.ProtoAtLeast(1, 1) && (!haveHost || len(hosts) == 0) && !isH2Upgrade && req.Method != "CONNECT" {
		return nil, badRequestError("missing required Host header")
	}

[networkimprov]

cc @bradfitz @neild @odeke-em

[BlackHole1]

I think this should be removed, the language itself should not make this judgment, but should be given to the web framework to achieve.

I'd like to hear from the community and would be happy to submit a relevant PR if I can.

---

I have written a behavior for this case in different languages for this purpose: https://github.com/BlackHole1/http-header-host

Here is the outline:

rfc2822 - 2.2. Header Fields

There is no mention in this specification of a requirement for the host to exist

rfc7230 - 5.4 Host

When a proxy receives a request with an absolute-form of request-target, the proxy MUST ignore the received Host header field (if any) and instead replace it with the host information of the request-target. A proxy that forwards such a request MUST generate a new Host field-value based on the received request-target rather than forward the received Host field-value.

A server MUST respond with a 400 (Bad Request) status code to any HTTP/1.1 request message that lacks a Host header field and to any request message that contains more than one Host header field or a Host header field with an invalid field-value.

When there is no Host, we can treat it as absolute-form. So it shouldn't report an error either

Result

nodejs / python is normal

golang will report an error

[BlackHole1]

I understand why golang does this, but I personally don't feel that we should make blocking judgments about this at the language level. Like I said above, it should be left to the web framework to implement

Just like nodejs and python, only simple assignment operations are done.

@neild @bradfitz @seankhliao What do you think?

[networkimprov]

cc @ianlancetaylor as possible proposal

[bradfitz]

I don't think we should remove this. The spec says these are the rules, and we follow them.

If you're dealing with some non-compliant software or using some HTTP-like protocol that's not HTTP, you can write a net.Conn adapter that HTTP-ifies it for net/http, which speaks HTTP.

[neild]

RFC 7230 is extremely clear:

A client MUST send a Host header field in all HTTP/1.1 request messages.

A server MUST respond with a 400 (Bad Request) status code to any HTTP/1.1 request message that lacks a Host header field

We aren't going to gratuitously violate the specification here. As @bradfitz says, if you're working with something that speaks almost-but-not-quite HTTP, then you can write an adapter to convert it to actual HTTP.

[umeat]

I am in the unfortunate position of dealing with an almost-HTTP protocol, where this Host header problem is the only one I can't easily get around with the http library. I was hoping to just be able to disable the Host header check, but it sounds like that won't happen.

Do you have an example of what you're describing with a net.Conn implementation which converts the requests to HTTP. It would be nice if I could leave the rest of my normal HTTP server implementation as is.

[FFengIll]

I meet the same problem and after check the implement in golang, I doubt the Host header process is too rude.

As comment by @BlackHole1 (#44388 (comment)),
the Host check should change to be more graceful.

[yan-foto]

@neild for testing/measurement/security/etc. purposes it would be extremely helpful to see how a server responds exactly when violating RFCs. The alternative of "write your own HTTP protocol handler" means that I have to go about and reimplement everything that go std lib does just to remove one bloody header.