Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HTTP/3] Support for HTTP/3 multiple connections #101531

Draft
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

ManickaP
Copy link
Member

@ManickaP ManickaP commented Apr 25, 2024

Feel free to review and discuss the API changes. This is NO MERGE until we have the APIs approved and all the TODOs resolved

The implementation takes H/2 connection pooling and slightly adjusts it for H/3 specific behaviors (see a diff between HttpConnectionPool.Http2.cs and HttpConnectionPool.Http3.cs).

The PR depends on 2 API reviews:

TODO: Add numbers from benchmarks when it runs
TODO: Stress run depends on dotnet/aspnetcore#55282

Fixes #51775
Fixes #54968
Fixes #68380
Resolves #101535
Resolves #101534

Copy link

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ManickaP ManickaP added the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Apr 25, 2024
@ManickaP ManickaP requested a review from a team April 25, 2024 08:30
@ManickaP
Copy link
Member Author

ManickaP commented Apr 25, 2024

No noticeable regression on perf from main:

main

  1. 1x10000 (clients x threads)
crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=10000
| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 35                                    |
| Max Cores usage (%)         | 971                                   |
| Max Working Set (MB)        | 3,810                                 |
| Max Private Memory (MB)     | 4,415                                 |
| Build Time (ms)             | 6,988                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24224.9             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 519                                   |
| Requests                    | 345,446                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 23,069                                |
  1. 1x100 (clients x threads)
crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=100
| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 41                                    |
| Max Cores usage (%)         | 1,150                                 |
| Max Working Set (MB)        | 577                                   |
| Max Private Memory (MB)     | 1,119                                 |
| Build Time (ms)             | 4,926                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 520                                   |
| Requests                    | 989,468                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 65,955                                |
  1. 20x500 (clients x threads)
crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=20 --variable concurrencyPerHttpClient=500
| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 97                                    |
| Max Cores usage (%)         | 2,710                                 |
| Max Working Set (MB)        | 6,526                                 |
| Max Private Memory (MB)     | 7,268                                 |
| Build Time (ms)             | 2,544                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 518                                   |
| Requests                    | 4,976,883                             |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 331,779                               |

PR, multiple H3 connections OFF

  1. 1x10000 (clients x threads)
crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=10000 --client.options.outputFiles artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Http.dll --client.options.outputFiles  artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Quic.dll
| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 36                                    |
| Max Cores usage (%)         | 998                                   |
| Max Working Set (MB)        | 3,772                                 |
| Max Private Memory (MB)     | 4,371                                 |
| Build Time (ms)             | 2,866                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 596                                   |
| Requests                    | 342,841                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 22,914                                |
  1. 1x100 (clients x threads)
crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=100 --client.options.outputFiles artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Http.dll --client.options.outputFiles  artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Quic.dl
| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 46                                    |
| Max Cores usage (%)         | 1,296                                 |
| Max Working Set (MB)        | 587                                   |
| Max Private Memory (MB)     | 1,174                                 |
| Build Time (ms)             | 2,897                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 594                                   |
| Requests                    | 985,119                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 65,691                                |
  1. 20x500 (clients x threads)
rank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=20 --variable concurrencyPerHttpClient=500 --client.options.outputFiles artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Http.dll --client.options.outputFiles  artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Quic.dll
| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 95                                    |
| Max Cores usage (%)         | 2,667                                 |
| Max Working Set (MB)        | 6,494                                 |
| Max Private Memory (MB)     | 7,246                                 |
| Build Time (ms)             | 2,660                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 594                                   |
| Requests                    | 5,334,724                             |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 355,707                               |

PR, multiple H3 connections ON

  1. 1x10000 (clients x threads)
crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=10000 --client.options.outputFiles artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Http.dll --client.options.outputFiles  artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Quic.dll
| client                      |                                      |
| --------------------------- | ------------------------------------ |
| Max CPU Usage (%)           | 92                                   |
| Max Cores usage (%)         | 2,572                                |
| Max Working Set (MB)        | 6,491                                |
| Max Private Memory (MB)     | 7,228                                |
| Build Time (ms)             | 2,736                                |
| Start Time (ms)             | 0                                    |
| Published Size (KB)         | 74,777                               |
| Symbols Size (KB)           | 19                                   |
| .NET Core SDK Version       | 9.0.100-preview.5.24272.19           |
| ASP.NET Core Version        | 9.0.0-preview.6.24272.7+49c1c68bf1ac |
| .NET Runtime Version        | 9.0.0-preview.5.24272.2+5c06e5d01fa0 |
| Processor Count             | 28                                   |
| First request duration (ms) | 594                                  |
| Requests                    | 4,325,325                            |
| Bad Status Code Requests    | 0                                    |
| Exceptions                  | 0                                    |
| Mean RPS                    | 288,664                              |

/// Occurres when an additional stream capacity has been released by the peer. Corresponds to receiving MAX_STREAMS frame.
/// </summary>
public event QuicConnectionStreamsAvailableEventHandler? StreamsAvailable;
private async void OnStreamsAvailable(int bidirectionalStreamsCountIncrement, int unidirectionalStreamsCountIncrement)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
private async void OnStreamsAvailable(int bidirectionalStreamsCountIncrement, int unidirectionalStreamsCountIncrement)
private async Task OnStreamsAvailable(int bidirectionalStreamsCountIncrement, int unidirectionalStreamsCountIncrement)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gag reflex whenever I see async void 😃
E.g. it could be a regular async Task with the caller doing _ = OnStreamsAvailable(), like we do for InjectNewHttp3ConnectionAsync.

It's probably fine in this case though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually prefer async void to _ = BlaBlaAsync() 😄 And we have a precedent for it in S.N.Quic (which I authored 😃 )

@ManickaP
Copy link
Member Author

Change of the logic: StreamsAvailable has been made a regular callback and moved to QuicConnectionOptions (thanks @rzikm for the suggestion). This is more aligned with how we handle callbacks/events in SslStream. After all, events are not a common occurrence in our stack. It also solves an issue with the correctness of the provided values in regards to thread safety. There's no chance of registering to the event in the middle of opening stream or processing STREAMS_AVAILABLE event. The callback is registered before the connection is created and the callback is always invoked as a reaction to a MsQuic event. The only thing that could happen is that invocations of the callback might swap order since we must offload the callback execution from MsQuic thread to the thread-pool. However, we are providing increments, so it doesn't matter that much as a+b == b+a.

ManickaP added a commit to dotnet/dotnet-api-docs that referenced this pull request May 16, 2024
ManickaP added a commit to dotnet/dotnet-api-docs that referenced this pull request May 16, 2024
Copy link
Member

@MihaZupan MihaZupan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few more comments but otherwise this LGTM.

It's good that it substantially mimics the H2 logic, makes it much easier to follow.

@@ -210,6 +284,7 @@ public async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, lon

if (quicStream == null)
{
ReleaseStream();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a 100% reliable way to detect whether the underlying QuicConnection counted the stream as consumed or not?
E.g. even when OpenOutboundStreamAsync is cancelled?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, this is not actually right, in both cases we should not re-increment the count (only if we didn't even try to open the stream). I did some testing and added some comments to QuicStream to make sure we have consistent behavior wrt stream counts regardless whether the call gets cancelled or not.

Debug.Assert(_availableRequestStreamsCount >= 0);

if (NetEventSource.Log.IsEnabled()) Trace($"ReleaseStream: _availableRequestStreamsCount = {_availableRequestStreamsCount}");
++_availableRequestStreamsCount;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll have to signal the _availableStreamsWaiter from here as well because StreamsAvailableCallback may not be called for a while if a bunch of streams got consumed by requests that were immediately canceled and released their streams.

}
public delegate void QuicConnectionStreamsAvailableCallback(System.Net.Quic.QuicConnection connection, int bidirectionalStreamsCountIncrement, int unidirectionalStreamsCountIncrement);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More of an API review question, but is there any chance we'll want to expose more info here such that we'd want the arguments to be a struct instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have only stream count related info, technically the transport sends MAX_STREAM_ID and separately for Uni/Bi directional streams.
The only think we could expose are different ways to express the new stream limit, which is part of alternative design in #101534

/// Provided via <see cref="StartAsync(Action{QuicStreamType}, CancellationToken)" /> from <see cref="QuicConnection" /> so that <see cref="QuicStream"/> can decrement its available stream count field.
/// When <see cref="HandleEventStartComplete(ref START_COMPLETE_DATA)">START_COMPLETE</see> arrives it gets invoked and unset back to <c>null</c> to not to hold any unintended reference to <see cref="QuicConnection"/>.
/// </summary>
private Action<QuicStreamType>? _decrementAvailableStreamCount;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this functionally the same as if we instead temporarily stored a QuicConnection and exposed the decrement function there as internal?

@@ -603,6 +674,15 @@ private unsafe int HandleEventPeerStreamStarted(ref PEER_STREAM_STARTED_DATA dat
data.Flags |= QUIC_STREAM_OPEN_FLAGS.DELAY_ID_FC_UPDATES;
return QUIC_STATUS_SUCCESS;
}
private unsafe int HandleEventStreamsAvailable(ref STREAMS_AVAILABLE_DATA data)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get MsQuic to expose increments instead of current counts?
(presumably, that's the data that they're reacting to that triggered these events in the first place?)

That'd let us drop the whole DecrementAvailableStreamCount dance

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea. I can try that. It would also resolve the issue with the _decrementAvailableStreamCount callback. However, I'll not push on adding this to MsQuic, until we have the API approved, we might end up with MAX_STREAM_ID or different value being reported by the callback...

@@ -427,7 +498,7 @@ public async ValueTask<QuicStream> OpenOutboundStreamAsync(QuicStreamType type,
NetEventSource.Info(this, $"{this} New outbound {type} stream {stream}.");
}

await stream.StartAsync(cancellationToken).ConfigureAwait(false);
await stream.StartAsync(DecrementAvailableStreamCount, cancellationToken).ConfigureAwait(false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using DecrementAvailableStreamCount directly here will allocate a new object for every stream.
We could cache a single Action<QuicStreamType> instance on the connection.

Comment on lines +448 to +451
// The new connection could not handle even one request, either because it shut down before we could use it for any requests,
// or because it immediately set the max concurrent streams limit to 0.
// We don't want to get stuck in a loop where we keep trying to create new connections for the same request.
// So, treat this as a connection failure.
Copy link
Member

@MihaZupan MihaZupan May 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is right for HTTP/3.

With HTTP/2, this would only happen if the connection was already closed as it otherwise always starts with a non-zero amount of available streams.

With HTTP/3, _availableRequestStreamsCount may still be 0 at this point, especially given that StreamsAvailableCallback is queued to the thread pool and may therefore signal the initial available streams after we hand the connection to the pool and reach this block.
In which case we'd be throwing away a good connection and failing a request.

connectionException = e is OperationCanceledException oce && oce.CancellationToken == cts.Token && !waiter.CancelledByOriginatingRequestCompletion ?
CreateConnectTimeoutException(oce) :
e;
// If the connection hasn't been initialized with QuicConnection there's no need to dispose it.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes that InitQuicConnection may never throw, right?

/// Occurres when an additional stream capacity has been released by the peer. Corresponds to receiving MAX_STREAMS frame.
/// </summary>
public event QuicConnectionStreamsAvailableEventHandler? StreamsAvailable;
private async void OnStreamsAvailable(int bidirectionalStreamsCountIncrement, int unidirectionalStreamsCountIncrement)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gag reflex whenever I see async void 😃
E.g. it could be a regular async Task with the caller doing _ = OnStreamsAvailable(), like we do for InjectNewHttp3ConnectionAsync.

It's probably fine in this case though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-System.Net.Http new-api-needs-documentation NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons)
Projects
None yet
3 participants