Deadline Bounded StreamSend

[Johan511]

Currently, there is no API to request MsQuic to be able to request that a particular StreamSend should either be fulfilled within a deadline or not be sent all together.

Why is this useful?
Real time and low latency applications which also send large amounts of data such as live video streaming and interactive gaming,
These applications often generate blocks of data periodically and need the block to be delivered before the next time slot (read as deadline).

In low latency use cases especially when there are variable network conditions (prevalent in the modern landscape of wireless and mobile) it can be extremely detrimental to latency if we queue in a block which can not be sent within the deadline on to the wire as it blocks that send of other data until it is drained (because we do not have stream abort/reset or stream priorities below the transport layer)

The feature request can be broken into 2 major parts

• We need some logic at the transport layer to make sure that MsQuic checks if the data can be sent within the deadline.
• An API for the send side application to set the deadline and receive a callback when the it has been guessed that the data will not be sent as the deadline is likely not going to be satisfied.

This feature request does not propose

• Any negotiation of deadlines between sender and receiver, we assume that this is done at the application layer and the deadline is enforced only by the sender for latency reasons.
• Any explicit signal to the receiver that a block of data will not be sent because of the deadline might not be fulfilled. This is simply because I do not wish to formally define what the signal should be. There has been some work in this area. Read, please note that this is meant for MP-QUIC but there is no reason we should not have such a feature. There is also similar work for QUIC but it seems to have been abandoned, read.

Some alternatives which might seem like a good alternative but I believe are not

• Have a timer at the application layer and abort the stream: This works only if the data from the stream has not been queued into the wire by the time abort occurs, when we have shorter deadlines such as ~100ms which would be expected in low latency and real time applications we also have much smaller blocks which are going to be less than the congestion window and readily queued onto the wire almost instantaneously.
• Use a unreliable transport protocol: The fact that we are dropping blocks of data when they can not be sent might sound like we do not need reliability, but I would argue that is not the case. Consider the use of SVC (Scalable Video Codec) where we encode a raw video stream into hierarchical base and enhancement layers. Assume we have 4 layers in total, if we receive only Layer 1 before the deadline, we decode it and present a 480p video to the user, if we receive only Layer 1 and Layer 2 before the deadline, we decode them together and present a 720p video and so on. For a lack of a better term in my vocabulary I am going to term it as "tolerant to unreliability at the block level", the application is okay with receiving 2 out of the 5 layers, but not okay with receiving a corrupted layer which would be possible in case of using datagrams (assume each block few MTUs in size)

Proposed solution

Before diving into a proposed solution, I would like to clarify that is solution is for deadlines for each StreamSend call, and not for the stream all together. The stream can live for eternity but we enforce a condition on each StreamSend that it must be sent within the deadline.
Once we have a solution which enforces a deadline on each StreamSend call, it becomes easier to have a deadline for a stream all together by enforcing the deadline on all StreamSend calls on that Stream.

We also decide to either send the data in entirety or not at all, we will not be sending the data partially.

API for sender application

Introduce a new API in QUIC_API_TABLE named StreamSendWithDeadline and the following signature

typedef
_IRQL_requires_max_(DISPATCH_LEVEL)
QUIC_STATUS
(QUIC_API * QUIC_STREAM_SEND_WITH_DEADLINE_FN)(
    _In_ _Pre_defensive_ HQUIC Stream,
    _In_reads_(BufferCount) _Pre_defensive_
        const QUIC_BUFFER* const Buffers,
    _In_ uint32_t BufferCount,
    _In_ QUIC_SEND_FLAGS Flags,
    _In_opt_ void* ClientSendContext,
    _In_ TimeDiff TimeToDeadlineInMilliseconds, // New Field
    _In_ QUIC_DEADLINE_SEND_FLAGS DeadlineSendFlags // New Field
    );

If the current time is T0, this API sets deadline as T0 + TimeToDeadlineInMilliseconds, TimeDiff would be equivalent to decltype(operator-(std::chrono::time_point, std::chrono::time_point), we can instead use size_t or time_t

What happens if MsQuic guesses that the block can not be sent within the deadline?
This would depend on the DeadlineSendFlags set

typedef enum QUIC_DEADLINE_SEND_FLAGS {
   QUIC_DEADLINE_SEND_FLAG_CALLBACK, // Default 
   QUIC_DEADLINE_SEND_FLAG_SILENT,
} QUIC_DEADLINE_SEND_FLAGS;

In case of QUIC_DEADLINE_SEND_FLAG_CALLBACK,
There is a QUIC_STREAM_EVENT callback sent with the following type QUIC_STREAM_EVENT_DEADLINE_POSSIBLY_CAN_NOT_BE_SATISFIED (please suggest a better name) with the following event type

struct {
   void* ClientSendContext;
   TimePoint (*GetCurrentMsTime)(void);
   TimeDiff HowManyMsBeforeTheReceiverReceivesTheData
} DEADLINE_POSSIBLY_CAN_NOT_BE_SATISFIED;

If the application returns QUIC_STATUS_SUCCESS from the callback MsQuic decides to not send the data
If the application returns QUIC_STATUS_CONTINUE from the callback MsQuic decides to continue with sending the data even if the deadline is going to be broken.

Calculating HowManyMsBeforeTheReceiverReceivesTheData

This assumes we have a very good Bandwidth and RTT estimate
and computes it using the following simple hueristic

If there exists a TimeToDeadlineInMilliseconds then at the first time a frame is written:
    ExpectedDelay = (RTT / 2) + (BytesInFlight + DataSize) / ExpectedBandwidth
    // NOTE: BytesInFlight includes frames which are queued to be yeeted into flight
    if (CurrentTime + ExpectedDelay > Deadline):
         RetVal = IndicateEvent()
         If (RetVal == QUIC_STATUS_SUCCESS):
             Dequeue the buffer from the `SendRequests` list

     TimeToDeadlineInMilliseconds = std::nullopt

Additional context

No response

[nibanks]

A few comments:

An API for the send side application to set the deadline and receive a callback when the it has been guessed that the data will not be sent as the deadline is likely not going to be satisfied.

How is MsQuic supposed to "guess"? Is this deadline based on just putting the data on the wire, or based on an expectation of the one-way delay to the peer, or the round trip time for an ACK? What about loss/retransmits? What about being blocked on flow control or congestion control? Just because we're blocked now doesn't mean things couldn't open in the future.

I think this feature is a good ask, but we must be very crisp on the behavior here. Worst case, MsQuic can/should always come back if we run out of time and give a callback to the app for this.

I would like to clarify that is solution is for deadlines for each StreamSend call, and not for the stream all together

You can't cancel a single send, only the stream. So, if we went with this model, if a single send doesn't meet the deadline, the stream would be shutdown from there on an you'd have to use a new one. Is that what you're looking for (I assume not)?

typedef enum QUIC_DEADLINE_SEND_FLAGS {
   QUIC_DEADLINE_SEND_FLAG_CALLBACK, // Default 
   QUIC_DEADLINE_SEND_FLAG_SILENT,
} QUIC_DEADLINE_SEND_FLAGS;

You must have the callback, because to shut down a stream, an error code must be provided. The error code would be the mechanism for the app to provide this.

This assumes we have a very good Bandwidth and RTT estimate

Do you really want to use RTT, or do you want to use a one-way delay (not actually standardized today).

[Johan511]

Let me just add some definitions to make conversations less confusing

DeadlineTimePoint: Server side epoch time before which we want the send to be completed
DeadlineTimeDiff: How many ms to deadline as set by the TimeToDeadlineInMilliseconds parameter in the StreamSendWithDeadline function

How is MsQuic supposed to "guess"?

A very naive idea is to use ExpectedDelay = (RTT / 2) + (BytesInFlight + DataSize) / ExpectedBandwidth and check if DeadlineTimePoint is before CurrentTime + ExpectedDelay

Is this deadline based on just putting the data on the wire, or based on an expectation of the one-way delay to the peer

T0: Application calls StreamSendWithDeadline with DeadlineTimeDiff
T0 + DeadlineTimeDiff = DeadlineTimePoint

I believe, we need to include one way delay and bandwidth estimate into the calculation of the "guess"

or the round trip time for an ACK?

I prefer full block being received by the client as our "satisfaction condition" (the event which we want to occur before DeadlineTimePoint) and not server receiving ACK, to make a good guess we want to minimize variance in what we are measuring.
T1: Block is fully received for client
T2: All data frames which have transmitted the block have been ACKd

I believe having a reliable guess for T1 is easier than T2 because the client might decide to wait to accumulate multiple ACKs

What about loss/retransmits?

This is a tough problem I have been really hoping that the Bandwidth estimate from the CongestionControl protocol will just magically solve (which is probably too much to hope for). Unfortunately I have to admit that I have no concrete solution for this yet, except for possibly using full block being ACKd as the satisfaction condition which would require us to estimate T2 which as mentioned previously isn't something I am in favour of.

What about being blocked on flow control or congestion control? Just because we're blocked now doesn't mean things couldn't open in the future.

I hadn't thought about this, does something like a TryToSendLater flag work and we remove the send request from the queue only after DeadlineTimePoint

You can't cancel a single send, only the stream. So, if we went with this model, if a single send doesn't meet the deadline, the stream would be shutdown from there on an you'd have to use a new one. Is that what you're looking for (I assume not)?

I like this feature to be as simple and unsurprising as possible if we were resetting the stream, what happens to the other objects queued to be sent? We will need to return them to the application who will probably create another stream and queue the objects onto it which sounds like just an unnecessary amount of effort which I would like to prevent by dequeuing the send request. MsQuic trying to create a new stream without an explicit instruction from the application isn't something I am fond of either.

You must have the callback, because to shut down a stream, an error code must be provided. The error code would be the mechanism for the app to provide this.

If this is a critique on there existing an QUIC_DEADLINE_SEND_FLAG_SILENT, I would like to defend its existence. I believe MsQuic should not be calling callbacks for events the application is not interested in. Ideally I would like to see a "subscribe" (something akin to how we do the network statistics callback) for events, on account of the fact that we do not have such an API I am suggesting QUIC_DEADLINE_SEND_FLAG_[SILENT, CALLBACK] to show interest in QUIC_STREAM_EVENT_DEADLINE_POSSIBLY_CAN_NOT_BE_SATISFIED events.

Do you really want to use RTT, or do you want to use a one-way delay (not actually standardized today).

Path->OneWayDelay or Path->OneWayDelayLatest would be what I need

To add, this feature request is going to be a good amount of research work on trying to get a good guess and analysing which methodology provides us a good guess can not really be done unless we actually implement and test different guessing models.