Implement MaxFrameSize and HeaderTableSize for HTTP/2#2838
Implement MaxFrameSize and HeaderTableSize for HTTP/2#2838JunTaoLuo merged 1 commit intorelease/2.2from
Conversation
9fcf9c1 to
1de5817
Compare
| _hpackDecoder = new HPackDecoder((int)_serverSettings.HeaderTableSize); | ||
| _serverSettings.MaxConcurrentStreams = (uint)context.ServiceContext.ServerOptions.Limits.Http2.MaxStreamsPerConnection; | ||
| _serverSettings.MaxFrameSize = (uint)context.ServiceContext.ServerOptions.Limits.Http2.MaxFrameSize; | ||
| _incomingFrame = new Http2Frame(_serverSettings.MaxFrameSize); |
There was a problem hiding this comment.
This allocates a 16 MB buffer heap allocated buffer per connection... ?
There was a problem hiding this comment.
Yea... we haven't really optimized any of this and have been allocating all over the place.
There was a problem hiding this comment.
16 KB by default, but we should definitely used pooled memory for this.
There was a problem hiding this comment.
Should we do this now? I think there are tons of allocations everywhere and we should do it systematically at some point. But I suppose we could start piecemeal at a time.
| public void PrepareContinuation(Http2ContinuationFrameFlags flags, int streamId) | ||
| { | ||
| PayloadLength = MinAllowedMaxFrameSize - HeaderLength; | ||
| PayloadLength = (int)_maxFrameSize; |
There was a problem hiding this comment.
Remove this since it always has to be set afterwards.
| _hpackDecoder = new HPackDecoder((int)_serverSettings.HeaderTableSize); | ||
| _serverSettings.MaxConcurrentStreams = (uint)context.ServiceContext.ServerOptions.Limits.Http2.MaxStreamsPerConnection; | ||
| _serverSettings.MaxFrameSize = (uint)context.ServiceContext.ServerOptions.Limits.Http2.MaxFrameSize; | ||
| _incomingFrame = new Http2Frame(_serverSettings.MaxFrameSize); |
There was a problem hiding this comment.
16 KB by default, but we should definitely used pooled memory for this.
| var padded = padLength != null; | ||
|
|
||
| PayloadLength = MinAllowedMaxFrameSize; | ||
| PayloadLength = (int)_maxFrameSize; |
| _echoApplication = async context => | ||
| { | ||
| var buffer = new byte[Http2Frame.MinAllowedMaxFrameSize]; | ||
| var buffer = new byte[Http2Limits.MaxAllowedMaxFrameSize]; |
There was a problem hiding this comment.
There's a test that I added that needs to echo a data frame that's bigger than the MinAllowedMaxFrameSize and I wanted to reuse the _echoApplication. I could write a separate request delegate so that each test doesn't allocate 16 MB though.
There was a problem hiding this comment.
Nevermind, I forgot I restructured the test in question so this limit increase is no longer needed.
|
|
||
| if (_clientSettings.MaxFrameSize != previousMaxFrameSize) | ||
| { | ||
| _frameWriter.UpdateMaxFrameSize(_clientSettings.MaxFrameSize); |
There was a problem hiding this comment.
Nit: do all the business logic either before or after acking. Don't ack in the middle.
There was a problem hiding this comment.
I moved the ack to the start since there's a comment saying it should be done before updating the window. But I'm curious since I don't see why the current implementation has any downsides?
| public void PrepareHeaders(Http2HeadersFrameFlags flags, int streamId) | ||
| { | ||
| PayloadLength = MinAllowedMaxFrameSize - HeaderLength; | ||
| PayloadLength = (int)_maxFrameSize; |
| _frameWriter = new Http2FrameWriter(context.Transport.Output, context.ConnectionContext, _outputFlowControl, this, context.ConnectionId, context.ServiceContext.Log); | ||
| _hpackDecoder = new HPackDecoder((int)_serverSettings.HeaderTableSize); | ||
| _serverSettings.MaxConcurrentStreams = (uint)context.ServiceContext.ServerOptions.Limits.Http2.MaxStreamsPerConnection; | ||
| _serverSettings.MaxFrameSize = (uint)context.ServiceContext.ServerOptions.Limits.Http2.MaxFrameSize; |
There was a problem hiding this comment.
I'm not a fan of these conversions between int and uint everywhere. Currently the values on Limits.Http2 are all ints and the values on Http2PeerSettings are uints so we have to convert between the two. We should just pick one and stick to it.
|
Also addressing #2816 in the same PR since it's an even smaller change. |
| _serverSettings.MaxConcurrentStreams = (uint)context.ServiceContext.ServerOptions.Limits.Http2.MaxStreamsPerConnection; | ||
| _serverSettings.MaxFrameSize = (uint)context.ServiceContext.ServerOptions.Limits.Http2.MaxFrameSize; | ||
| _serverSettings.HeaderTableSize = (uint)context.ServiceContext.ServerOptions.Limits.Http2.HeaderTableSize; | ||
| _hpackDecoder = new HPackDecoder((int)_serverSettings.HeaderTableSize); |
There was a problem hiding this comment.
There is no corresponding logic required to handle the HeaderTableSize of the _clientSettings since we are not currently doing header compression in our HPackEncoder. Our HPackEncoder currently does a direct literal encoding without using a compression table or huffman encoding; I should file an issue for that.
89a37da to
1a02735
Compare
|
🆙📅 |
|
@davidfowl https://ci3.dot.net/job/aspnet_KestrelHttpServer/job/release_2.2/job/linux-Configuration_Debug_prtest/159/ looks like a regression possibly caused by your recent change. Notice that the missing trace, "Connection id "0HLG9A5H9REP4" stopped.", was indeed logged, but not before the mock verification which occurs after the WebHost is disposed. There also aren't any logs indication ungraceful shutdown. |
| await StartStreamAsync(1, _browserRequestHeaders, endStream: true); | ||
|
|
||
| await ExpectAsync(Http2FrameType.HEADERS, | ||
| withLength: 16390, |
There was a problem hiding this comment.
Can this be calculated from MinAllowedMaxFrameSize?
f45cfe8 to
64127e6
Compare
| private int _maxFrameSize = MinAllowedMaxFrameSize; | ||
|
|
||
| // These are limits defined by the RFC https://tools.ietf.org/html/rfc7540#section-4.2 | ||
| public const int MaxAllowedHeaderTableSize = 4096; |
There was a problem hiding this comment.
Why are these constants public?
They should also be listed above members.
| /// <summary> | ||
| /// Limits the size of the header compression table, in octets, the HPACK decoder on the server can use. | ||
| /// <para> | ||
| /// Defaults to 4096 |
| // Ack before we update the windows, they could send data immediately. | ||
| await _frameWriter.WriteSettingsAckAsync(); | ||
|
|
||
| if (_clientSettings.MaxFrameSize != previousMaxFrameSize) |
There was a problem hiding this comment.
You could do this before writing the settings ack to avoid the async await.
There was a problem hiding this comment.
That's what I originally had, but @halter73 wanted it moved.
There was a problem hiding this comment.
Why can't you just store the task in a local and return it like before?
| _connectionContext.ServiceContext.ServerOptions.Limits.Http2.MaxFrameSize = length; | ||
| _connection = new Http2Connection(_connectionContext); | ||
|
|
||
| await InitializeConnectionAsync(_echoApplication, expectedSettingsLegnth: 12); |
There was a problem hiding this comment.
Legnth -> Length. And I'd rather it be Count: 3
| withLength: 37, | ||
| withFlags: (byte)Http2HeadersFrameFlags.END_HEADERS, | ||
| withStreamId: 1); | ||
| // The client's settings is still defaulted to Http2PeerSettings.MinAllowedMaxFrameSize so the echo response will come back in two separate frames |
| public Http2Frame(uint maxFrameSize) | ||
| { | ||
| _maxFrameSize = maxFrameSize; | ||
| _data = new byte[HeaderLength + _maxFrameSize]; |
| public class Http2Limits | ||
| { | ||
| private int _maxStreamsPerConnection = 100; | ||
| private int _headerTableSize = MaxAllowedHeaderTableSize; |
There was a problem hiding this comment.
Http2PeerSettings.DefaultHeaderTableSize

Addresses #2817