Support Scatter Gather Buffers on the client to copy data directly to application provided buffers

[srsubra]

Describe the feature you'd like supported

Currently, MsQuic with XDP provides the application level callbacks using internal buffers. This requires a mem copy from the MsQuic Internal buffers to the target application buffer. For large blob, allocating this memory and performing a memory copy

Proposed solution

Provide an option where the user can specify the Target buffer address where the data needs to be copied to and MsQuic core layer can automatically copy the data to the target buffer.

Additional context

No response

[guhetier]

Hi!

Here is how we see how the implementation would look like:

• Probably a new flag "QUIC_STREAM_OPEN_FLAG_USE_EXTERNAL_RECEIVE_BUFFERS" to opt-in the feature, when opening the stream
• A new API function along the lines of "StreamProvideReceiveBuffers(stream, nb_buff, QUIC_BUFFERS* buff)"
  • The user can call this on an open stream to provide a list of buffers
  • The user would call it before starting the stream and may call it again later (on a receive callback, etc...) if it needs to provide more buffers

MsQuic would write the received data in the buffers in the order they are provided. A buffer would never be internally allocated or re-used: if MsQuic runs out of provided buffer space for receiving data, the stream would be failed.

The user would need to ensure the buffer remain valid until:

• either the stream is closed, and no more data can be received
• or a "receive" notification indicates all the bytes of the buffer
  • the user needs to keep track of the total number of bytes to know this
  • a "receive" notification could indicate only part of a buffer, multiple buffers, etc... depending on how the data is received

Does it sound like this would work for you?

[nibanks]

Probably a new flag "QUIC_STREAM_OPEN_FLAG_USE_EXTERNAL_RECEIVE_BUFFERS" to opt-in the feature, when opening the stream

I'm not convinced we need this for a couple reasons:

1. This doesn't work for peer started streams. You never open them locally.
2. I think it's enough to say calling StreamProvideReceiveBuffers opts into the feature. You don't need two signals for the same thing.

So, I'm leaning towards only the new API.

[guhetier]

Good point for the peer started stream.

I was considering the flag because we currently initialize buffer management during the stream initialization - we may have to move the recvBuffer initialization later or change its mode on the fly.

For peer started stream, we may have to look at it closer: I think the stream is started before the user is notified, so data could be buffered before external buffers are provided (which we would then need to copy).

EDIT: Actually, it may all be in the work queue which should avoid most of the problem.

[srsubra]

"MsQuic would write the received data in the buffers in the order they are provided. A buffer would never be internally allocated or re-used: if MsQuic runs out of provided buffer space for receiving data, the stream would be failed."
For our use case, we know the size of the blob data since that is fixed. But the part of the payload can have variable length data depending on what fields are populated and serialized.

can we do the following?
If the total payload size is N bytes out which the user provides a buffer for X bytes, then the first X bytes are copied to the user provided buffer, the remaining N-X bytes can be retained in the QUIC buffer. The callback can then be invoked to the application

[nibanks]

can we do the following?
If the total payload size is N bytes out which the user provides a buffer for X bytes, then the first X bytes are copied to the user provided buffer, the remaining N-X bytes can be retained in the QUIC buffer. The callback can then be invoked to the application

That would be much more complicated.

I would rather have a model where the app can just continue to supply more buffer space if you start running out. But there are some edge cases that might complicated this model too.

[srsubra]

Nick, you already have the data in MsQuic buffers right? We are just copying a section of it to the blob address. So why can't the rest of the data in the ring buffer be returned to the application in the callback ?

[guhetier]

If the total payload size is N bytes out which the user provides a buffer for X bytes, then the first X bytes are copied to the user provided buffer, the remaining N-X bytes can be retained in the QUIC buffer. The callback can then be invoked to the application

I agree with Nick here, a hybrid model would probably be confusing and less flexible: the buffer ownership would depend on the buffer and it would allow the app to provide buffers only for the first X bytes.

The approach we currently have should let you provide a new buffer in the receive callback if you start running out of space.
However, the max receivable byte number we advertise is limited by the size of the buffer provided so if you expect a header + a variable length, enough buffer size should be provided upfront to avoid a second round trip.

[guhetier]

you already have the data in MsQuic buffers right? We are just copying a section of it to the blob address. So why can't the rest of the data in the ring buffer be returned to the application in the callback ?

In the current model, we have the data in a MsQuic buffer - and we are providing a pointer to this buffer directly to the app.
In the new model, we would put the data directly in the application buffer to save a copy. MsQuic would no longer allocate buffers at all after the payload is decrypted.

[guhetier]

@srsubra Would this proposal address your need?
If not, we can have a chat about the exact scenario so we can find a good solution that is also generic enough as a MsQuic API.