When to send the SETTINGS frame #2945
Comments
|
My two cents go to requiring the frame to be sent immediately after the handshake. IIRC, we discussed the issue long ago, and IIRC the consensus was that clients "can" wait for the server's SETTINGS frame before sending requests. Doing so improves the efficiency of QPACK for example, assuming that the server supports the use of dynamic table. Generally speaking, I think it is preferable to be able to learn about the capabilities of the peer at the earliest moment. OTOH, I do not think there is a compelling reason to postpone sending SETTINGS frame. |
|
I agree with you and I vote for sending SETTINGS as soon as possible right after handshake. I want to warn you though that this does not prevent an implementor to chose something different since for how the protocol is designed -- after we removed waiting for receiving SETTINGS to send a request -- everything will keep working. As you said it would just be in a non-optimal state: i.e. without the ability to use the QPACK dynamic table and without GOAWAY. This is in fact something we never thoroughly test at interop since we only send a single request on a single stream and stop there. As a sad/clowny story, we had a bug once where both endpoints would fail at reading from all unidirectional streams. It was so subtle that everything kept working fine, just no compression and no graceful shutdown. |
|
I'm ok with SHOULD send immediately, but as Luca said, I don't think we can enforce this any more than we already are, so I believe implementations should function relatively well without receiving SETTINGS. The only way to force people to send SETTINGS immediately would be to make them part of the handshake(ie: put them in transport parameters). There's a somewhat related issue about 0-RTT and SETTINGS: #2790 |
|
Here's a wild thought that probably won't help; if we go ahead with removing priorities from H3 then the only defined SETTINGS parameters relate to HTTP headers: HTTP_MAX_HEADER_LIST_SIZE Since we have control streams to manage QPACK, a total design change would make the above parameters part of the head of the encoder or decoder stream. This affects 0-rtt considerations and likely has other ripple effects. In short: kill SETTINGS - they are different enough from H2 settings anyway. |
You're making me nervous. |
|
I agree that having SETTINGS available as early as possible is best. Even though I'm not a huge fan of the asymmetry, moving the server's SETTINGS to the server's TLS EncryptedExtensions would guarantee that clients always have them before sending their request, and make TLS session tickets easier to reason about (cc @nharper - I think this was your idea). |
|
I think it was @davidben's idea to put SETTINGS in EncryptedExtensions, but I'm a huge fan of that idea. If we move SETTINGS to EE, this makes it much easier for the client to handle NewSessionTickets, especially for a QUIC stack designed for multiple applications (not just HTTP/3). With SETTINGS in EE, the QUIC layer has everything it needs when it wants to cache a NST for 0-RTT resumption, whereas if it needs to wait for a SETTINGS frame sometime later in application data, it both needs to consult the application layer when it saves a NST, and the answer from the application layer could be "please wait, I don't have all of the bits I need yet". |
|
@LPardue : what about extensions that use SETTINGS? I don't think you want to completely eliminate them. Plus what @dtikhonov said
|
|
I think some of the H3 "SETTINGS for extension story is pretty broke" because of the removal of SETTINGS ack and other constraints. Theres like 1 adopted H2 extension that uses settings (I'm sure theres lots of private ones too). I'm pretty sure we could replace SETTINGS_ENABLE_CONNECT with a standalone H3 control stream frame without much effort. I'm not advocating strongly for killing SETTINGS just lobbing some grenades, |
|
One advantage with taking on the complexity of SETTINGS now is that we have something we can (speaking hopefully) rely on in future. Aside from the justifiable nervousness at distributing things, having separate mechanisms means that we don't have any infrastructure for doing things like the crazy CONNECT protocol thing. |
|
Oh I agree with that, let's try and nail down the design of SETTINGS. I just think there'll be some surprises when someone comes to try a semantic extension that requires negotiation and finds that SETTINGS wont support them. Simple extensions like "I accept new thing" can continue to be SETTINGS and will benefit from the 0-RTT framework we set out. |
Yeah. We have seen interop fail between some endpoints due to one side expecting SETTINGS before sending the request (response). I would prefer sending the settings parameters using EE, if we are to also require the server to wait for SETTINGS before sending anything. That would make the protocol design simpler, because transmission of any frames (expect the settings parameters) can be based on the knowledge of the peer. Admittedly, I am referring to our discussion at IETF 103. |
|
I discussed this with @nharper and @DavidSchinazi today and I think we concluded placing it in transport parameters would be simpler than EE. I'm happy to do it there and that would make it required(or if it's not present, it's clear the peer is fine with the defaults). If so, the H3 feature negotiation would work identically to the QUIC transport feature negotiation, so reasoning about it should be straightforward. |
|
@kazuho, Are you suggesting an asymmetry? That is, the client uses a frame and the server uses EncryptedExtensions? Or do we pack this into transport parameters? I can see the appeal of that. As information that relates to the connection, this sort of thing naturally rolls up into the handshake. And the client is the only one with an acute need to have the associated state persisted beyond the lifetime of the connection. The server might still suffer head-of-line blocking for client settings, but we've already established that the need is much less severe. I have something of a dramatic suggestion. It's one that was thorough poo-pooed when I suggested it the first time. If Lucas can do the crazy-talk thing, so can I. We can move the TLS session ticket up two layers and send it in h3. Then we can bind whatever state we like into it. And we get nice deterministic ordering if we want. That's complicated for a bunch of reasons, but if you look at the implementation of tickets, I think that you end up writing most of that complexity into the stack anyway. I've only implemented the transport-layer interaction with tickets and it's already pretty thorny. I realize that this opens a lot of awkward questions, but it might be worth the effort to address them. |
|
I was thinking of asymmetry (client sends SETTINGS in an H3 frame, server sends them in TP in EE), though collapsing them entirely to TP and half a round trip earlier has some appeal. (My biggest concern with that is the client's SETTINGS are sent in the clear.) In thinking about how to implement #2790 (and how to specify/clarify some of that language), there's quite a bit of interaction between the H3 layer and TLS layer. I see how it's tempting to move the TLS session ticket to an H3 frame. However, this means that any application that wants to use QUIC for its transport has to provide some mechanism to convey session tickets (otherwise that application can't use 0-RTT), which seems like a large downside. Having everything at one layer for handling session tickets sounds like it would work best, and I think the transport layer (instead of the application layer) is the right place for that. The server's transport parameters already need to be remembered by both peers for 0-RTT resumption, so we're stuck with the transport-layer interaction with tickets. If we add an "application layer parameters" field to transport parameters, we can put the server's SETTINGS in there (and possibly the client's), and we get all of the logic for remembering SETTINGS for free. Also all of the ordering problems between SETTINGS and the NewSessionTicket go away. |
|
The asymmetry continues to bother me. I don't want these in the clear either. It seems like moving to transport parameters really only has the effect of forcing head-of-line blocking. I'm not that enthusiastic about that.
That's right. The interaction needs to be there anyway. The question is more to what extent we try to hide that interaction. The ticket elevation was mostly just a request for consideration than it was any serious suggestion. Though it does neatly address the problem of which protocols get to use 0-RTT - because only those that bother to design a mechanism get to use it. That's a neat property. |
What I have (or had) in my mind is asymmetry. IIRC, there are two issues with the client sending the H3 settings parameters in TP. One is that it goes in clear. The other is that it would happen before the protocol is determined by ALPN. These are the reasons why we decided not to use TP for this purpose. I am fine with sticking to this previous conclusion (although I am perfectly fine with the client using TP for sending the settings parameters too). |
|
The asymmetry between having client send SETTINGS in an H3 frame and a server sending SETTINGS in TP doesn't bother me because those settings serve different purposes. The client's SETTINGS serves solely to indicate H3 configuration for the connection on which it was sent. The server's SETTINGS serves both that purpose and to indicate H3 configuration that will be accepted for early data sent on a future connection. We already have an asymmetry in the purpose of the two SETTINGS frames. Why not consider the two use cases for the server's SETTINGS separately? The second use case (use these settings for early data on a future connection) would be better served if it were more tightly coupled to other information sent to the client about what to do on that future connection. That could be done by sending it in a NewSessionTicket extension; it could also be done by putting it in the TLS handshake (e.g. in EncryptedExtensions or in transport parameters). If we consider the use cases completely separately, then we could have the server send an H3 SETTINGS frame for the current connection and put settings in NST or EE for use for the future connection. This leads to duplication of data sent (if n NSTs are sent, the settings are sent n+1 times), although it allows the server to specify different parameters for the current connection vs the future ones. Using EE instead of NST reduces the duplication when multiple NSTs are sent. A design of both client and server sending SETTINGS as an H3 frame to apply for the current connection (and only that connection) meets the requirements for symmetry. The requirement for allowing non-default SETTINGS to apply in early data is solved by having the server signal the SETTINGS for early data of a future connection via some other mechanism, either in the NST or in EE. Is putting server SETTINGS in both an H3 frame and EE preferable to putting them just in EE? |
|
Currently you can only send SETTINGS once, so allowing them to be sent n+1 times means they all have to be identical, which is not only duplication of data, but also the client should check that they're identical. Which is doable, but seems unfortunate. At some point, it was suggested that the only data that ever needed to be sent post-handshake was NST. IF that was true, we wouldn't need CRYPTO frames in the 1 RTT packets, which might be a simplification and the elevation of NST might be a net win. Given where we're at, are people interested in any of these larger changes? It seems like the simplest change would be that any sent SETTINGS frame replaces any SETTINGS used for early data. I believe this ensures long term consistency and is the minimal change that would 'fix' this issue? |
|
The strawman of sending SETTINGS in NST (resulting in them being sent n+1 times) has only the SETTINGS frame in H3 apply to that connection. The ones in NST could be different since they only apply to early data on a new connection. |
|
IIRC, we stopped using extensions of NST in draft-12. The change was welcomed by some, because certain TLS stacks do not provide access to NST extensions. My weak preference goes to avoiding cross-layer optimizations unless there is strong benefit. Because cross-layer optimizations introduces the API concerns. Therefore, I prefer to not send session tickets using H3 frames, or to not send H3 settings using NST. |
janaiyengar
added
the
design
|
In the interest of moving forward with a solution that we have some experience with and resolving both #2945 and #2790 I(and some of the editors) suggest we say the server MUST send SETTINGS if it wants to do 0-RTT and the client SHOULD/MUST not attempt 0-RTT if it hasn't yet received SETTINGS. This would motivate servers to sent SETTINGS immediately and removes any possibility of the client and server state getting out of sync, as @nharper detailed in #2790 I find some of the other ideas presented here appealing, but they all require substantial changes and none of them are obviously better in my opinion. |
|
I agree on making progress. However, maybe I'm I'm failing to see how the suggestions address the points that @kazuho captured when making this issue. If we ignore 0-RTT, client and server sending of SETTINGS in the simple 1-RTT case can still end in a bit of a deadlock. |
|
Requiring that the server send SETTINGS as soon as possible doesn't solve the fact that they can be reordered in flight and arrive after session tickets, requiring the client to process session tickets asynchronously. I agree that requiring SETTINGS to be sent early is a good idea, but I'm not sure it's enough. |
|
I agree with David: requiring SETTINGS be sent early doesn't solve the reordering issue w.r.t. NewSessionTickets. |
|
My understanding is there is currently a correctness issue in regards to NST and SETTINGS, but there is no actual deadlock if a server sends SETTINGS really late or not at all(and doesn't do 0-RTT). A client can decide to wait until SETTINGS are received before making a request in order to use the QPACK dynamic table, but it comes with the risk that you delay requests for an unknown period of time. If implementations send requests as soon as possible and make optimal decisions given what they know at the time, then there's no deadlock, correct? |
|
The server might wait until it receives the client SETTINGS in order to make use of the dynamic table in the response. If the client doesn't want to send SETTINGS, the connection might idle time out. |
|
That seems unwise in the extreme. SETTINGS was designed to be non-blocking. The advice here seems simple enough:
corollary (that I didn't think needed to be stated explicitly, but we can do that): don't condition your SETTINGS on the value from your peer
I think that addresses concerns about resumption as well as 0-RTT. I agree that this is not great from an implementation perspective, but I'm fairly sure that we can all use |
|
@LPardue writes:
That's not the sort of bug that would go unnoticed for long. Implementing a server that way would be, to borrow @martinthomson's euphemism, unwise. QPACK is designed with the understanding that QPACK can be used before the SETTINGS frame is received. I suggest we remove the QPACK dynamic table aspect from this discussion. |
|
Yes it would be unwise because there's a good enough fall back, use static compression. Pending priority removal from the spec, the only thing SETTINGS holds is header-related parameters. Removing the QPACK dynamic table aspect from the discussion means that we are discussing only SETTINGS_MAX_HEADER_LIST_SIZE. We discussed the usefulness of that parameter on #2516, with some happy to support its removal; but we decided in London to keep it. So all we can talk about for SETTINGS is in terms of extensions, known or unknown. Sending SETTINGS isn't blocked on the peer but is that true for how an application acts upon those SETTINGS? E.g. Should a server that implements an extension and advertises it in its SETTINGS block until it receives a SETTINGS from its peer? That depends on the nature of the extension. So on this axiom, I'd be happy with text that says an endpoint MUST send SETTINGS before all other activity, along with some guidance in the "Extension to HTTP/3" section that describes the risks in blocking on reception of SETTINGS. |
|
I've uploaded #2958 as a potential way to resolve this issue by moving the server's SETTINGS to transport parameters. |
|
I really don't like #2958. It's added complexity and head-of-line blocking to solve an issue that I don't agree is worth solving. |
|
I'm not a big fan of the using transport parameters because of asymmetry. I prefer that we simply say that SETTINGS SHOULD be sent asap. There are good enough reasons for implementations to be doing this anyways. |
|
@martinthomson I'm probably just confused, but how does #2958 add head-of-line blocking? @janaiyengar Saying that implementations should send the SETTINGS frame ASAP doesn't address the problems the asymmetry is trying to solve. Perhaps QUIC does not wish to solve the problem at all, but it's not the case that "simply" sending SETTINGS ASAP fixes things. There's also a lot of hidden complexity to the status quo:
Some of these (e.g. the last one) aren't fixed by #2958, but they at least reduce to transport parameters, and the worst of it (ordering problems) is gone. (For completeness, there's a third option, which is to not stick the SETTINGS frame into the ticket at all. This fixes everything, but it means you get your SETTINGS frame slightly later.) QUIC is already quite invasive when it comes to the QUIC/TLS interface. It complicates the whole ecosystem around QUIC and compromises QUIC's ability to pick up TLS changes. (Did everyone remember their QUIC/TLS record-layer interfaces need to express record-level padding for ESNI? It's not obvious at first glance.) That ability is critical; if TLS over QUIC is ever unable to meet TLS over TCP's baseline as we evolve it with ESNI, post-quantum algorithms, etc., browsers will need to disable QUIC to avoid downgrade attacks. Let's try to trim away invasive dependencies when it doesn't buy much. Remember, complexity at component boundaries costs much more than complexity within a component. |
|
I think that's overstating the complexities. If you don't do this, you get vanilla h3. That's the fallback position. It's almost like you are arguing for that, because most of these relate directly to that point. If you do bind SETTINGS to tickets, you get to use nice h3 features like QPACK in 0-RTT and after resumption. The question we're grappling with is not whether to have the feature (I hope; if we are please open an issue so we can have that discussion), but more how to most cleanly implement that coupling. Putting SETTINGS in the transport parameters doesn't fix the problem, it just moves it around. The asymmetry it adds doesn't do anything to remove any asymmetries elsewhere. |
|
It's not really optional right now. This bit here is a problem: "The server MAY omit settings from its SETTINGS frame which are unchanged from the initial value". That means, if the client doesn't remember the SETTINGS information, due to, say, packet ordering, it won't apply this SETTINGS frame delta correctly. Having both sides disagree on how to interpret a SETTINGS frame is rather poor. Moreover, this mismatch is sticky. If you get out of a sync once, you stay out of sync as you renew tickets. To that end, I guess there's yet another option: strike that sentence. The server must encode the SETTINGS de novo, and the client can start with either the initial value or the cached value. That ambiguity is odd, but the client already starts with the initial value at 1-RTT, so I think that's less of a nuisance. Putting SETTINGS in the transport parameters gives it an ordering: it's guaranteed to be processed before NewSessionTicket. (And HTTP requests[*], for that matter, so it's an improvement in the 1-RTT case too.) [*] More accurately, it's guaranteed to be processed before 1-RTT HTTP requests, but 0-RTT HTTP requests get the remembered one, so you still get to rely on some form of SETTINGS. |
I do not think that works, because an endpoint cannot rely on a property unless that property is enforced. To paraphrase, it needs to be a MUST to be meaningful. |
|
It seems that the argument against #2958 is asymmetry. Then, I might have a suggestion. How about defining a QUIC frame for exchanging application-protocol settings, that is transmitted using Handshake packets? That would make the protocol design symmetric, at the same time guaranteeing delivery of the settings prior to application data. PS. And then, we could also state in H3 that servers cannot update settings when 0-RTT is used. That would totally eliminate the synchronization issue of settings. |
|
The problem with the Handshake is that the client is sending it prior to the ALPN selection, which means it has to send the settings for each protocol it's proposing. There are a couple interwoven issues here. Let's try to untangle them:
|
|
I think the proposal for moving it into the handshake only affects the server SETTINGS frame (that's the asymmetry that people are unhappy about), so it's fine. At the time the server sends it, it has decided on ALPN. You do raise a good point about the ALPN dependency, however. That's a further complication to the QUIC/TLS interface: as long as we remember the server SETTINGS frame across 0-RTT, the QUIC/TLS interface must allow the server to pass the SETTINGS frame into TLS after TLS negotiates ALPN and before TLS sends tickets, likely in the ALPN callback. This happens independent of when the SETTINGS frame is sent (status quo, handshake, or something else). The only way to avoid that complexity is to stop remembering the SETTINGS frame across 0-RTT.
I think your two issues are still interwoven. We can't do the second bullet point without the first, otherwise the client and server will get out of sync. But, yeah, this is the "yet another option" I suggested above.
|
I agree with you about the complexity of that, but I doubt giving up remembered settings in 0-RTT is a price we're willing to pay. I feel like it's marginally better to have something post-handshake (NST) depend on the application (even if you get it during the handshake processing) than to have the transport parameters depend on application data. We've currently been able to toe the line that nothing in the server's transport parameters depends on processing the client's transport parameters; ideally I'd consider that to include the rest of the Client Hello. |
|
The higher-order issue in my mind is that we don't have a general mechanism to do an "application handshake" as part of the crypto+transport handshake, which is a shame. If we had that, we might be able to use it simply for exchanging SETTINGS. We could add an application blob to the transport parameters (as proposed in #2958, but in both directions, listing one per ALPN) to provide this mechanism, but of course, it'll be in cleartext in the client->server direction. #2958 resolves this encryption issue by having SETTINGS in transport parameters in the server->client direction and in a SETTINGS frame in the client->server direction, which is strange IMO. It generalizes one direction but leaves the other one specific to HTTP/3... it's not the asymmetry in the mechanisms that troubles me, it's the asymmetry in the generality of the mechanisms. A future app may be able and willing to use cleartext transport params for its handshake, but I suspect having SETTINGS be in cleartext won't be agreeable to most (I'm less worried about QPACK, and more about future HTTP/2 and HTTP/3 extensions). Given that, I find myself wandering back to something that's basically what we have right now but with the normative additions in #2972. @davidben @nharper: Do the normative additions in #2972 make the potential API complications simpler? |
|
I'm not sure I quite buy the generality argument. #2958 means the future app (as well as the current one!) can reliably get server parameters delivered before application data with no RTT cost. It's true that doesn't solve client parameters, but we otherwise solve neither direction. For HTTP and other client-speaks-(mostly)-first protocols, the server parameters are the ones where ordering is more sensitive anyway, so it makes sense to focus on it. Anyway, #2972 isn't as clean as #2958, but it does at least address the showstopping portion of this issue. |
|
I think I'd like to take a two-pronged approach, then. For now, let's merge #2972 once folks think it's ready. Let's separately have a discussion about whether and how transport should provide this capability; if we decide to add such a capability to QUIC, HTTP can use it. |
|
Thanks for filing another issue @MikeBishop I think that's a good approach to move forward. The current text is insufficient and/or incorrect and we should fix it. Your PR does that. But I think we should continue this discussion. One additional point about Settings is that because they're delivered after the client obtains 1-RTT keys, so there is some complexity on the client side(and/or potentially the server) to wait for the right moment to send the initial batch of requests. In particular, if the client starts sending requests immediately upon receiving 1-RTT keys(when 0-RTT is not attempted), QPACK will never be used for those requests. I think we should call that out in the HTTP draft, but I'll note that solving it may require a fair bit of cross-layer knowledge. As a related point, if we really want to require sending SETTINGS immediately on the server side, then transport params is the only way I can think of to truly require it. |
|
I think @janaiyengar frames this issue well, that we don't currently have an application handshake embedded in the crypto+transport handshake. #2958 is close to providing a general mechanism for an application handshake embedded in the crypto+transport handshake. The change necessary to that PR to provide all of the application handshake would be to allow specifying different application_layer_parameters for different alpns. The caveat is that the client-sent application layer params are sent in the clear, but this might not be an issue. (For example, transport params are sent in the clear by the client even though they could be delayed a round-trip to have them encrypted, and an application could make the analysis that the client-sent part of its handshake is not sensitive information that needs to be encrypted because it is similar to transport params.) Since the application handshake mechanism in #2958 doesn't impose any restrictions on how the application uses it, we can use it in H3 only for the server->client direction. Having just half of the embedded application handshake seems better than not using it at all. In addition to the benefits for #2790, sending the server's SETTINGS in the handshake means both sides can know that it comes before any server-sent application data (and before client-sent application data on non 0-RTT connections). Regardless of the outcome on this issue, I think #2972 to send complete SETTINGS instead of only the ones that have changed is a good idea, and we can merge that one while still figuring out what to do here. |
It seems that there are disagreement among the implementors, regarding when the SETTINGS frame need to be sent.
Some argue that it needs to be sent right after the handshake, even if the SETTINGS frame would contain no parameters. Some argue that it can be delayed, until the need to send something on the control stream arises.
I think we should clarify the expected behavior.
FWIW, existing text that refers to the topic are:
section 3.3:
section 6.2.1:
section 7.2.5:
cc @DavidSchinazi @LPardue @rmarx
The text was updated successfully, but these errors were encountered: