x/net/http2: toggle HPACK dynamic table indexing for header

[blixt]

Often a service will have certain header fields that have no discernible repeatable pattern (or the possible combinations far outnumber the HPACK table slots). It would be good to be able to turn off HPACK for those headers so that these non-repeating values don't use up slots in the dynamic table.

Here's a quote from Apple's APNs documentation touching on this:

APNs requires the use of HPACK (header compression for HTTP/2), which prevents repeated header keys and values. APNs maintains a small dynamic table for HPACK. To help avoid filling up the APNs HPACK table and necessitating the discarding of table data, encode headers in the following way—especially when sending a large number of streams:

• The :path header should be encoded as a literal header field without indexing,
• The apns-id and apns-expiration headers should be encoded differently depending on initial or subsequent POST operation, as follows:
  • The first time you send these headers, encode them with incremental indexing to allow the header names to be added to the dynamic table
  • Subsequent times you send these headers, encode them as literal header fields without indexing

[bradfitz]

That's some weird documentation. You can't use HTTP/2 without HPACK, so of course "APNs requires the use of HPACK". Do they actually hang up on you if you encode things differently than they'd like?

In any case, what are you asking for? Do you want a change in the hpack interface, or a new bool option on *http2.Transport, or at which level?

[blixt]

I can't comment on the quality of Apple's documentation, but what they recommend in terms of dynamic table indexing appears to be reasonable.

The feature request would be to allow a way to tell the HTTP/2 HPACK encoder to encode a specific header/field pair as a literal without indexing.

I see that hpack.HeaderField already has the options indexedTrue, indexedFalse, and indexedNever which is what we'd need to expose when encoding a request. I don't know the full design of the HTTP libraries, but the most straight-forward thing I can think of is req.Header.AddWithOptions("apns-id", "abcdef123456", http.HPACKIndexedFalse). (This would not have any effect on pre-HTTP/2 requests and may be incompatible with future HTTP versions.)

Other than my above suggestion I'm unsure what the best plan would be to expose the indexing options, other than that it probably should not be a bool considering it's already a choice between three options, and more may be added in future versions of HTTP.

[bradfitz]

So you're using the http2 package & hpack encoder directly, right? You're far away from that level of detail up in *http.Client land, but you want to control low-level details?

[blixt]

I'm currently configuring a client with http2.ConfigureTransport(…). You tell me how far away setting the index options of an HTTP/2 header needs to be from *http.Client land, but it currently doesn't seem to be possible at all. This is merely a request to make it possible to set indexing of a particular header key/value pair to on/off/never, from wherever that may be.

[bradfitz]

Does Apple actually reject your requests if they're encoded with indexing?

[blixt]

Not as far as I am aware – at the very least they allow the connection to be alive long enough to successfully deliver all notifications.

[bradfitz]

This seems pretty low priority, then.

It sounds like somebody at Apple just got really excited about HTTP/2 and started writing docs from an ideal HTTP/2 point of view without considering that most HTTP client libraries wouldn't provide such nitty-gritty details via their high-level APIs (e.g. req.Header.AddWithOptions feel super gross, since practically 0% of people should care about that)

[rabbbit]

While this doesn't make it any less ugly, or any more important - just leaving it here that this came up in my corporation today. We're basically sending 2*4kb headers on every request, one of them never changes, the other always changes.

An ability to disable dynamic indexing would have been helpful - even if it's configured once statically (per transport? connection), as opposed to per request.