UPDATE:

A couple of things:

1. API update: we don't need to add a new EXECUTION_CONFIG_ALL flag, the current design should be interpreted as, if the QTIP flag is set, we ALLOW connections to send over QTIP and Listeners to accept QTIP connections. Bottom line, don't change anything with the global flags / state.

2. In order to implement the behavior as specified in this issue, we ADD a new QUIC_SETTINGS parameter that specifies the option to use or don't use QTIP at the Connection/Listener layer. That means we need to update the Connection creation code and the Listener creation code to call into the Datapath layer to apply these state updates. If a Connection is created and we tell it to use QTIP, but the global EXECUTION_CONFIG_QTIP flag is set to OFF, then throw an error or something.

Regarding (2), I think it should be a separate SetParam parameter for listeners and connections, instead of in the QUIC_SETTINGS struct. Putting it in the QUIC_SETTINGS means that this niche feature is visible to many more users of MsQuic, and may encourage more to use it. My understanding is that it's a temporary solution to a specific Azure limitation. I'd rather not immortalize it in the QUIC_SETTINGS struct.

Regarding (2), I think it should be a separate SetParam parameter for listeners and connections, instead of in the QUIC_SETTINGS struct. Putting it in the QUIC_SETTINGS means that this niche feature is visible to many more users of MsQuic, and may encourage more to use it. My understanding is that it's a temporary solution to a specific Azure limitation. I'd rather not immortalize it in the QUIC_SETTINGS struct.

Great point! I agree.

Sounds good, so here's the new design:

Added 2 new Set/Get params. 1 for CONN, 1 for listener.

For now, we still keep the global EXECUTION_CONFIG_QTIP flag.
The idea here is that if the app did not explicitly set the CONN param for QTIP, then we will inherit what the global execution param is.

For the listener, the idea is that we will always support QTIP underneath the hood, but depending on the listener flag, we decide whether we care to bubble up any QTIP updates to the QUIC layer.

And again, if the app did not explicitly set the listener param, we will just inherit what the global execution config flag was.

UPDATE:

Here's the new design:
When an app configures MsQuic to use QUIC_EXECUTION_CONFIG_FLAG_QTIP ,
all listeners will be able to handle QUIC and QTIP traffic, and all client Connections created will be by default using QTIP.

However, you can use Connection setparams to control whether a client connection uses QTIP or QUIC. You can do this regardless of the state of QUIC_EXECUTION_CONFIG_FLAG_QTIP. You can not modify the Connection setparams for QTIP if the Connection is a part of a listener.