proposal: net/http: add support for the upcoming "Structured Field Values for HTTP" RFC

[dunglas]

Structured Field Values for HTTP is an upcoming RFC from the HTTP Wording Group defining a set of well-defined data types to use in HTTP headers and trailers.

This new format will improve the interoperability and the safety of HTTP by allowing to create generic parsers and serializers suitable for all HTTP headers. It is already used in the wild, for instance for the new security headers supported by Google Chrome (Sec-Fetch-Dest etc), the Signature proposal in Prefer-Push or in Vulcain.

After the RFC publication, it would be nice to be able to parse and generate such headers directly using the standard library.

I proposed a patch adding support for this spec in #41045. The code is also available as a standalone library: https://github.com/dunglas/httpsfv

[gopherbot]

Change https://golang.org/cl/250837 mentions this issue: net/http: add a package to parse and serialize Structured Field Values

[bradfitz]

I think this is probably best outside the standard library until the RFC is accepted. Once it's in the standard library it's pretty frozen and we can't change APIs or break behavior easily.

[dunglas]

For sure!

[rsc]

@dunglas, your README has a broken link. Should be https://pkg.go.dev/github.com/dunglas/httpsfv.

[rsc]

Also, given your comment above (For sure!) it sounds like you are saying to close this proposal until at least the RFC is accepted?

[dunglas]

@rsc thanks, link fixed.

We should at least wait for the Internet-Draft to become a RFC before merging the patch related to this proposal. However, it should happen soon. The specification is in last stages of standardization. The algorithms described in the I-D and implemented in the patch will most likely not change anymore.

The proposal can probably be kept open, and we can use the time window before the publication as a RFC to review and improve the patch.

[rsc]

@dunglas, I looked at at httpsfv and my first thought is: can we make this simpler? It seems like a huge amount of new API surface for HTTP.

[dunglas]

@rsc we can maybe reduce the API surface a bit, but probably not much. The spec defines many data structures, and several of them cannot be implemented using only Go's builtins: https://httpwg.org/http-extensions/draft-ietf-httpbis-header-structure.html#types

The main difference is that SFV's maps (Dictionary and Params) are ordered while Go's map isn't. Much of the API surface is related to these two types and the related methods. We can maybe remove the Del() methods from them, but I'm not sure if it's worth it (having the ability to remove elements from maps is convenient in some cases).

Another option would be to use interface{} in most places. This would allow to merge all Unmarshal* methods and maybe to also merge Dictionary and Params, but it will make the library less safe and less practical to use.

However I may have missed opportunities to reduce the API surface. I'm open to any suggestion to simplify this.

[rsc]

It would be nice to see how widely this is adopted. Being an RFC is one thing; being widely used is another.
It might be that x/net/http/sfv would be a decent starting place?
Even if we do put something in x/net I think we'd need to do a very careful API review to try to reduce the amount here.

/cc @neild

[rsc]

It seems like we should put this on hold for now. We are probably going to have to rethink a bit for HTTP/3, and since there are not many uses of structured field values yet, it might make sense to wait until that work is going on too, to try to deal with potential API changes all at the same time.

For now, we've already seen that nothing prevents using a third-party package for this functionality.

[dunglas]

Structured Field Values are now officially an RFC: https://www.rfc-editor.org/rfc/rfc8941.html

Should we consider adding this implementation in x/net/http/sfv or should we wait for more headers to adopt it?