net/http: support content negotiation

[kevinburke]

Content negotiation is, roughly, the process of figuring out what content type the response should take, based on an Accept header present in the request.

An example might be an image server that figures out which image format to send to the client, or an API that wants to return HTML to browsers but JSON to command line clients.

It's tricky to get right because the client may accept multiple content types, and the server may have multiple types available, and it can be difficult to match these correctly. I think this is a good fit for Go standard (or adjacent) libraries because:

• there's a formal specification for how it should behave: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
• it's annoying to implement yourself, correctly; you have to write a mini-parser.
• it would take one function to implement, which makes it annoying to import an entire third party library for (assuming you find the right one)

I've seen people hack around this in various ways:

• checking whether the Accept header contains a certain string
• checking for the first matching value,
• returning different content types based on the User-Agent
• requiring different URI's to get different content.

There's a sample implementation here with a pretty good API: https://godoc.org/github.com/golang/gddo/httputil#NegotiateContentType

// NegotiateContentType returns the best offered content type for the request's
// Accept header. If two offers match with equal weight, then the more specific
// offer is preferred.  For example, text/* trumps */*. If two offers match
// with equal weight and specificity, then the offer earlier in the list is
// preferred. If no offers match, then defaultOffer is returned.
func NegotiateContentType(r *http.Request, offers []string, defaultOffer string) string

offers are content-types that the server can respond with, and can include wildcards like text/* or */*. defaultOffer is the default content type, if nothing in offers matches. The returned value never has wildcards.

So you'd call it with something like

availableTypes := []string{"application/json", "text/plain", "text/html", "text/*"}
ctype: = NegotiateContentType(req, availableTypes, "application/json")

If the first value in offers is text/* and the client requests text/plain, NegotiateContentType will return text/plain. This is why you have to have a default - you can't just return the first value in offers because it might include a wildcard.

In terms of where it could live, I'm guessing that net/http is frozen at this point. Maybe one of the packages in x/net would be a good fit? There is also a similar function for parsing Accept-Language headers in x/text/language. Open to ideas.

[quentinmit]

Funny you should mention this, I was just looking for this for use in x/perf. Your proposed API is reasonable, but you should explicitly document what happens in exceptional circumstances:

• What happens when multiple content types end up with the same q? I think I would expect the first one in offers to be chosen.
• What happen if the Accept header(s) cannot be parsed?

[kevinburke]

What happens when multiple content types end up with the same q? I think I would expect the first one in offers to be chosen.

That sounds reasonable to me.

What happen if the Accept header(s) cannot be parsed?

The spec includes a grammar but doesn't really say what happens if you can't match it, other than you should return a 406 if you can't match anything. I think we should just return the default?

[kevinburke]

@bradfitz suggested (if we want to go forward with this proposal) it could live in net/http/httputil.

[bradfitz]

I also mentioned that it seems like defaultOffer string could go away and we say that the first element in the slice was the default.

[kevinburke]

Sorry, I tried to edit the description to cover that case.

I also mentioned that it seems like defaultOffer string could go away and we say that the first element in the slice was the default.

I think the problem with this would be, if you had "text/*" as the first element in offers, it would match if the client sent text/plain and text/plain would be returned from the function.

If we return the first value in offers as the default, then the response value would contain a wildcard ("text/*"), which it doesn't in any other circumstance. Maybe that's okay.

[jimmyfrasche]

@kevinburke What's wrong with returning text/plain if the first offer is text/*? I don't see why it has to return the exact string passed in—don't you need to know the negotiated type to set the header in the response?

Also, if negotiation fails does it return the empty string? I'd rather have an error to distinguish between expected failure, negotiation failed, and unexpected failure, bad request.

[bradfitz]

@kevinburke, any response to @jimmyfrasche? This seems fine if the simplified signature works. You want to prepare a CL?