x/net/http2: add support for SETTINGS_HEADER_TABLE_SIZE

[elindsey]

Proposal

Support HPACK dynamic table sizes other than the default, specifically:

• Add ability to configure the max dynamic table size for HPACK encoders/decoders in both the http2 server and client
• Add correct handling of sent/received SETTINGS_HEADER_TABLE_SIZE

Context

HTTP/2 uses HPACK for header compression. HPACK is a stateful compression algorithm where both the encoder and decoder reference a static table (defined in the RFC) and maintain a dynamic table. This dynamic table defaults to 4096 bytes and has header fields added/removed in FIFO order over the course of the connection lifetime.

SETTINGS_HEADER_TABLE_SIZE is an h2 setting used to adjust the size of the HPACK dynamic table. The decoder communicates the largest dynamic table size it's willing to support via SETTINGS_HEADER_TABLE_SIZE, then the encoder on the other end of the connection chooses what size it wants to use (up to the decoder's max) and communicates that back to the decoder via a dynamic header table size update instruction. Thus, there are two relevant client settings: the allowable size we want to advertise for our decoder, and the max size we want to use for our encoder.

Use Case

The main motivation is to allow increasing the dynamic table size for http2 connections that multiplex disparate requests, specifically in proxies and load balancers. The default size of 4096 works well for connections servicing a single client, but when multiple clients are bundled together on a reverse proxy's shared upstream connection and sharing the same decoder/encoder, the small table size and churning hurts the HPACK compression ratio, increases bytes on the wire and increases latency.

References

• Previous discussion about this here x/net/http2: missing SETTINGS_HEADER_TABLE_SIZE support #29356
• Current CL here https://go.dev/cl/435899
• HTTP/2 RFC section on SETTINGS_HEADER_TABLE_SIZE
• HPACK RFC

[neild]

The specific API is in https://go.dev/cl/435899:

type Server struct {
	// MaxDecoderHeaderTableSize optionally specifies the http2
	// SETTINGS_HEADER_TABLE_SIZE to send in the initial settings frame. It
	// informs the remote endpoint of the maximum size of the header compression
	// table used to decode header blocks, in octets. If zero, the default value
	// of 4096 is used.
	MaxDecoderHeaderTableSize uint32

	// MaxEncoderHeaderTableSize optionally specifies an upper limit for the
	// header compression table used for encoding request headers. Received
	// SETTINGS_HEADER_TABLE_SIZE settings are capped at this limit. If zero,
	// the default value of 4096 is used.
	MaxEncoderHeaderTableSize uint32
}

type Transport struct {
	// MaxDecoderHeaderTableSize optionally specifies the http2
	// SETTINGS_HEADER_TABLE_SIZE to send in the initial settings frame. It
	// informs the remote endpoint of the maximum size of the header compression
	// table used to decode header blocks, in octets. If zero, the default value
	// of 4096 is used.
	MaxDecoderHeaderTableSize uint32

	// MaxEncoderHeaderTableSize optionally specifies an upper limit for the
	// header compression table used for encoding request headers. Received
	// SETTINGS_HEADER_TABLE_SIZE settings are capped at this limit. If zero,
	// the default value of 4096 is used.
	MaxEncoderHeaderTableSize uint32
}

This seems fine to me.

[ianlancetaylor]

CC @neild @bradfitz

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

[rsc]

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

[gopherbot]

Change https://go.dev/cl/435899 mentions this issue: http2: add SETTINGS_HEADER_TABLE_SIZE support