[QUIC] Decide how to handle QUIC stream limits

[scalablecory]

How should we be validating these limits?

runtime/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicClientConnectionOptions.cs

Lines 29 to 43 in eab397d

	/// <summary>
	/// Limit on the number of bidirectional streams the peer connection can create
	/// on an accepted connection.
	/// Default is 100.
	/// </summary>
	// TODO consider constraining these limits to 0 to whatever the max of the QUIC library we are using.
	public long MaxBidirectionalStreams { get; set; } = 100;
	/// <summary>
	/// Limit on the number of unidirectional streams the peer connection can create
	/// on an accepted connection.
	/// Default is 100.
	/// </summary>
	// TODO consider constraining these limits to 0 to whatever the max of the QUIC library we are using.
	public long MaxUnidirectionalStreams { get; set; } = 100;

How do we get notified that a connection's stream limit has changed(MAX_STREAMS frame event)? This is required for HTTP/3

runtime/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs

Lines 89 to 97 in eab397d

	/// <summary>
	/// Gets the maximum number of bidirectional streams that can be made to the peer.
	/// </summary>
	public long GetRemoteAvailableUnidirectionalStreamCount() => _provider.GetRemoteAvailableUnidirectionalStreamCount();
	/// <summary>
	/// Gets the maximum number of unidirectional streams that can be made to the peer.
	/// </summary>
	public long GetRemoteAvailableBidirectionalStreamCount() => _provider.GetRemoteAvailableBidirectionalStreamCount();

runtime/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http3Connection.cs

Lines 235 to 256 in eab397d

	// TODO: how do we get this event?
	private void OnMaximumStreamCountIncrease(long newMaximumStreamCount)
	{
	lock (SyncObj)
	{
	if (newMaximumStreamCount <= _maximumRequestStreams)
	{
	return;
	}
	_requestStreamsRemaining += (newMaximumStreamCount - _maximumRequestStreams);
	_maximumRequestStreams = newMaximumStreamCount;
	while (_requestStreamsRemaining != 0 && _waitingRequests.TryDequeue(out TaskCompletionSourceWithCancellation<bool> tcs))
	{
	if (tcs.TrySetResult(true))
	{
	--_requestStreamsRemaining;
	}
	}
	}
	}

[geoffkizer]

Why does HTTP3 need to be notified?

I would assume that QuicConnection itself enforces the limits. That is, if we try to create a stream that exceeds the current limit, the first operation on that QuicStream will pend until the limit is increased (or cancellation occurs, of course). Is that not how it works currently?

[geoffkizer]

We also need to figure out the reverse issue -- meaning, how does a connection control and update its inbound stream limit.

[scalablecory]

There are three scenarios I'd like to enable here.

Just open me a stream

This is the most user-easy operation: just open a stream, and if none are available, wait for one to become available.

using QuicStream stream = await quicConnection.OpenBidirectionalStreamAsync(cancellationToken);
// await only completes once a stream is available and reserved.

Try to open a stream, without waiting

This is something useful for for both YARP and HttpClient. -- if a stream is available right now, let me use it. otherwise, fail immediately. This can be used to decide when to try a request against another server.

using QuicStream? stream = quicConnection.OpenBidirectionalStreamIfAvailable();
if (stream is not null)
{
   // stream is reserved for us and immediately available.
}
else
{
   // a null return indicates no streams available -- maybe try your HTTP/3 request on a different IP that DNS/config provided.
}

Allowing the caller to perform their own wait

This will be useful if the standard queuing behavior is not ideal. You could use it for diagnostics. A proxy might use this to start more aggressively load balancing to other hosts if the stream count is low.

There is an inherent race condition here in that, when we tell the user they have X streams available, they may, by the time they get that notification, have already used those streams elsewhere, or the peer may have given us even more streams to use. This is something ultimately I don't think we can 100% solve for the user -- they will need to do some synchronization -- but having this plus the IfAvailable overload will allow them to solve this themselves.

QuicConnection con = ...;
SemaphoreSlim sem = ...;

while(true)
{
   // note: this tells you how many new streams can be opened, not a new "maximum stream count" value.
   int newStreamCount = await con.WaitForAvailableBidirectionalStreamsAsync();
   sem.Release(newStreamCount);
}

async Task<HttpResponseMessage> SendAsync(...)
{
   await sem.WaitAsync(cancellationToken); // wait for a stream to become available.

   // Stream limits can never be decreased in QUIC, and we are managing our own queuing, so this call should always succeed.
   using QuicStream? stream = quicConnection.OpenBidirectionalStreamIfAvailable();
   Debug.Assert(stream is not null);

   // ... now send out the request.
}

[karelz]

@scalablecory I still don't get the value of the last scenario "Allowing the caller to perform their own wait":
Why would users want to do their own wait? Typically the increase will be 1 stream (when another stream finishes), unless server sends new higher stream count.
I think the Open and TryOpen should be enough -- either I want to wait, or I have some fallback option.

For more aggressive load balancing strategies, I think that current number of available streams is more useful data point -- how close am I to the limit, so that I can choose between multiple options without any waiting.

[ManickaP]

Meeting notes:

• We may want to expose current available streams count.
• We may want to implement reservation of streams (and not just rely on msquic info that may become out of date if highly concurrent scenario).
• Async APIs might be better than just callbacks.

[scalablecory]

@scalablecory I still don't get the value of the last scenario "Allowing the caller to perform their own wait". Why would users want to do their own wait?

I'm YARP and I have 2 servers I'm load balancing against.

I'm at my stream limit for both servers, so I need to wait. I don't want to pick a connection and wait -- I want to wait for the first connection to become available.

Maybe when streams become available, I don't want to wake up requests in a strictly FIFO order, but instead want to prioritize them.

Typically the increase will be 1 stream (when another stream finishes), unless server sends new higher stream count.

QUIC doesn't work this way --

In HTTP/2, you configure a maximum stream count and when one stream finishes you can open a new one.

In QUIC, a stream finishing doesn't mean you can open a new one. The remote side must send you a new chunk of streams to use. It is very similar to how HTTP/2 handles receive window updates.

[scalablecory]

Triage: @ManickaP is doing research on this right now, will have opinions later.

[ManickaP]

Notes:

• QUIC_CONNECTION_EVENT_TYPE.STREAMS_AVAILABLE connection event happens when the initial MAX_STREAMS is received and whenever the peer sends the update. We're getting the actual available stream count, not the max stream id that's being sent in QUIC.
• we don't need to manage remaining requests count in Http3Connection by ourselves, msquic will do that for us. This fields then can be removed:
  

  runtime/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http3Connection.cs

  Lines 52 to 55 in 1c9e200

  	// Manages MAX_STREAM count from server.
  	private long _maximumRequestStreams;
  	private long _requestStreamsRemaining;
  	private readonly Queue<TaskCompletionSourceWithCancellation<bool>> _waitingRequests = new Queue<TaskCompletionSourceWithCancellation<bool>>();

• msquic offers multiple option for starting the stream: https://github.com/microsoft/msquic/blob/main/docs/Streams.md#opening-and-starting-streams
• currently, we're starting streams asynchronously QUIC_STREAM_START_FLAGS.ASYNC, from ctor, and we're returning immediately. We're not doing much with the START_COMPLETE event:
  

  runtime/src/libraries/System.Net.Quic/src/System/Net/Quic/Implementations/MsQuic/MsQuicStream.cs

  Lines 622 to 640 in 5d4e742

  	private static uint HandleStartComplete(State state)
  	{
  	bool shouldComplete = false;
  	lock (state)
  	{
  	// Check send state before completing as send cancellation is shared between start and send.
  	if (state.SendState == SendState.None)
  	{
  	shouldComplete = true;
  	}
  	}
  	if (shouldComplete)
  	{
  	state.SendResettableCompletionSource.Complete(MsQuicStatusCodes.Success);
  	}
  	return MsQuicStatusCodes.Success;
  	}

• we can and we currently do expose the current available streams counts in GetRemoteAvailableBidirectionalStreamCount and GetRemoteAvailableUnidirectionalStreamCount. I'm not 100% convinced of having Remote in the name, otherwise LGTM as-is.
• the type for the limits on the options class shouldn't probably be long since we validate against ushort.MaxValue:
  

  runtime/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicOptions.cs

  Line 22 in 484a807

  public long MaxBidirectionalStreams { get; set; } = 100;

  
  

  runtime/src/libraries/System.Net.Quic/src/System/Net/Quic/Implementations/MsQuic/Interop/SafeMsQuicConfigurationHandle.cs

  Lines 53 to 56 in 01b7e73

  	if (options.MaxBidirectionalStreams > ushort.MaxValue)
  	{
  	throw new Exception("MaxBidirectionalStreams overflow.");
  	}

Additional thoughts:

• would we ever use QUIC_STREAM_START_FLAG_IMMEDIATE: Even if there is currently no data to send on the stream, MsQuic will inform the peer of the stream being opened. Without this flag, MsQuic will wait until data is queued on the stream.?
• what is the level GetParam thread safety? what happens when we call get available stream count while msquic is in the middle of processing MAX_STREAMS?

[ManickaP]

API suggestions:

• OpenBidirectionalStreamIfAvailable --> bool TryOpenBidirectionalStream(out stream)
  However this API is sync and if I get it right we still should wait for START_COMPLETE event, which will happen asynchronously (I think, I have to test it first). So should we block on the event? Make the method async? Or ignore it since we know it will happen and it will happen soon?

• I'm not 100% sure how would we properly implement WaitForAvailableBidirectionalStreamsAsync though. We can wait with reservation, or not wait at all if I get the msquic right.

[scalablecory]

Triage:

The MVP here is something like scenario 3 outlined in #32079 (comment)

We should consider if we need the other APIs -- if we suspect scenario 3 will be rare, or too difficult for the average QUIC user, then we should think about adding easier APIs.

[geoffkizer]

I'm not 100% sure how would we properly implement WaitForAvailableBidirectionalStreamsAsync though. We can wait with reservation, or not wait at all if I get the msquic right.

How does the msquic API work here?

I assume there is some way to get notified when the peer makes more streams available, right?

[ManickaP]

I assume there is some way to get notified when the peer makes more streams available, right?

Yes, when the peer increases the MAX_STREAMS we get a notification. We can hook this up to this event. When I wrote it, I was thinking about what this method promises and whether it can deliver it, which depends on synchronizing your work on the connection.

[geoffkizer]

Small thought:

Above, we have WaitForAvailableBidirectionalStreamsAsync returning the new available stream count.

I'm not sure this is useful. I suspect you usually just care that there is an available stream. And if you care about the exact available count, you can just get this yourself. It's immediately out of date anyway, so I expect the only thing people really care about is whether it's > 0 or not.

That said, I do think WaitForAvailableBidirectionalStreamsAsync needs to handle connection shutdown gracefully. So I think it should return a bool. True = streams are available. False = connection is shutting down and no more streams will ever be available. Thoughts?

[scalablecory]

Above, we have WaitForAvailableBidirectionalStreamsAsync returning the new available stream count.

Above proposal returns the number of new streams available, not total count available. it's so you can do a very simple semaphore.Release(x) without anything being out of date.

The confusion over what the count is here tells me it would need either a name change or do as you propose. We could move the semaphore inside of QuicConnection, I guess. We'll want to think if this would limit any designs that might have used a different mechanism for queuing things.

handle connection shutdown gracefully

Bool seems fine. if we keep it returning the new stream count, it could just return 0 to indicate end of connection.