Define terms for application actions #2857

MikeBishop · 2019-06-27T23:14:06Z

This consists of two related commits:

Defines the minimal set of operations which a QUIC implementation MUST expose to the application, introducing terms for them which don't rely on the specific wire formatting
Changes HTTP/3 to use these terms instead of referring to specific QUIC frames

Note that this doesn't remove all references to QUIC frames, but the remaining references are explanatory -- they describe what will happen at the transport layer after taking a particular action, rather than being part of the normative text. (But if you find one of these you think needs to go, please include that in this review.)

This is a design, because exposing this functionality is mandatory. I'm not sure how you would previously have created a usable QUIC implementation without these operations, but you weren't previously required to have them all.

Fixes #2805, closes #2677.

martinthomson

Thanks for doing this Mike, I have quite a few comments.

martinthomson · 2019-06-28T00:10:38Z

draft-ietf-quic-http.md

-responses as a result of having their request terminated abruptly, though
-clients can always discard responses at their discretion for other reasons.
+not been sent and received. When this is true, a server MAY abort reading the
+receiving part of the request stream with error code HTTP_EARLY_RESPONSE, send a


Do you think that we can just say "abort reading the request stream" and drop the "receiving part of" piece on the basis that "abort reading" can only apply to the receiving part?

martinthomson · 2019-06-28T00:11:17Z

draft-ietf-quic-http.md

-HTTP_REQUEST_CANCELLED ({{http-error-codes}}).  When the client cancels a
-response, it indicates that this response is no longer of interest.
-Implementations SHOULD cancel requests by aborting both directions of a stream.
+Clients can cancel requests by abruptly terminating the sending part of the


"abruptly terminating" or just "resetting" ?

martinthomson · 2019-06-28T00:12:53Z

draft-ietf-quic-http.md

-The following error codes are defined for use in QUIC RESET_STREAM frames,
-STOP_SENDING frames, and CONNECTION_CLOSE frames when using HTTP/3.
+The following error codes are defined for use when abruptly terminating the
+sending part of streams, aborting reading of the receiving part of streams, or


As above regarding reference to "parts".

martinthomson · 2019-06-28T00:13:34Z

draft-ietf-quic-http.md

-rely on QUIC PADDING frames or employ the reserved frame and stream types
-discussed in {{frame-grease}} and {{stream-grease}}.
+either rely on transport-layer padding or employ the reserved frame and stream
+types discussed in {{frame-grease}} and {{stream-grease}}.


Aside: we should probably have a conversation about the trade-offs with padding. Transport-layer padding is in some ways superior, like in how it can be granular to a single byte, which reserved streams and frames cannot do. But PADDING frames at the transport aren't retransmitted when lost, which could lead to some interesting attacks if packets can be suppressed by an attacker. If retransmissions aren't padded, boom.

martinthomson · 2019-06-28T00:19:52Z

draft-ietf-quic-transport.md

+
+On the sending part of a stream:
+
+- Attempt to write data, understanding when stream flow control credit


List construction here implies continuation of a sentence, but the capital letter doesn't fit and there is no joiner.

I think that you want:

On the sending part of a stream, application protocols need to be able to:

write data, understanding when .... ({{frame-stop-sending}});

end the stream, ....; and

reset the stream ... was already ended.

martinthomson · 2019-06-28T00:24:26Z

draft-ietf-quic-transport.md

+any implementation of this version of QUIC MUST expose the ability to perform
+the operations described in this section on a QUIC connection.
+
+When implementing the client role:


Same list construction comment as above.

martinthomson · 2019-06-28T00:25:36Z

draft-ietf-quic-transport.md

+
+When implementing the client role:
+
+- Open a connection, which begins the exchange described in {{handshake}}


Add: to enable 0-RTT, and to be informed when 0-RTT has been either accepted or rejected by a server.

martinthomson · 2019-06-28T00:26:46Z

draft-ietf-quic-transport.md

+- If Early Data is supported, embed application-controlled data in the TLS
+  resumption ticket sent to the client
+- If Early Data is supported, retrieve application-controlled data from the
+  client's resumption ticket and approve/veto accepting Early Data


and enable rejection of 0-RTT based on that information

I don't like "veto" much here. And we don't need to have an active acceptance involved.

martinthomson · 2019-06-28T00:28:18Z

draft-ietf-quic-transport.md

+In either role:
+
+- Configure minimum values for the initial number of permitted streams of each
+  type, as communicated in the transport parameters ({{transport-parameters}})


I think that there is a more general point, which basically means controlling resource allocation of all types, including streams and flow control.

The point you are looking at here - setting a minimum on the number of streams - is something HTTP needs, so I would add rather than replace.

martinthomson · 2019-06-28T00:28:58Z

draft-ietf-quic-transport.md

+- Keep a connection from silently closing, either by generating PING frames
+  ({{frame-ping}}) or by requesting that the transport send additional frames
+  before the idle timeout expires ({{idle-timeout}})
+- Cease issuing credit for new streams via MAX_STREAMS frames


See the above point about controlling resource allocation.

draft-ietf-quic-transport.md

mikkelfj · 2019-06-28T04:52:25Z

draft-ietf-quic-transport.md

+- Keep a connection from silently closing, either by generating PING frames
+  ({{frame-ping}}) or by requesting that the transport send additional frames
+  before the idle timeout expires ({{idle-timeout}})
+- Cease issuing credit for new streams via MAX_STREAMS frames


Cease and resume

The only place where this is required by HTTP is to support GOAWAY; I'd argue that the ability to stop is required, and the ability to resume is something an implementation could choose to expose.

LPardue · 2019-06-29T13:35:55Z

draft-ietf-quic-http.md

-`initial_max_uni_streams` (three being the minimum value required for the base
-HTTP/3 protocol and QPACK), and SHOULD use a value of 1,024 or greater for the
-QUIC transport parameter `initial_max_stream_data_uni`.
+Endpoints that excessively restrict the number or flow control window of these


This reads weird to me, my brain keeps thinking "or" is a typo for "of", which is untrue. I think it could read better to say

"Endpoints that excessively restrict the number of these streams or the size of each stream's flow control window will increase the chance that the remote peer reaches the limit early and becomes blocked."

LPardue · 2019-06-29T13:43:20Z

draft-ietf-quic-http.md

+unidirectional stream for the HTTP control stream plus the number of
+unidirectional streams required by mandatory extensions (three being the minimum
+number required for the base HTTP/3 protocol and QPACK), and SHOULD provide at
+least 1,024 bytes of flow control credit to each stream.


observation: the change to remove references to Transport Parameters in this paragraph could lead to some interesting unintended issues. Previously it was clear that the problems mentioned were related to small values of initial limits (imposed by TPs). One way to interpet the new text is that it is ok to advertise low values in TPs as long as you send MAX_STREAMS or MAX_STREAM_DATA.

Just wondering if that was the intent of this change, given some of the problems we've seen with these values during interop.

Good point. I personally think they're equivalent, but I believe our consensus was to require it to be in the transport parameters. I added a phrase to clarify this.

dtikhonov · 2019-07-01T14:04:57Z

draft-ietf-quic-http.md

-processed.  Servers SHOULD NOT increase the QUIC MAX_STREAMS limit after
-sending a GOAWAY frame.
+connection shutdown.  This identifier MAY be zero if no requests were processed.
+Servers SHOULD NOT permit additional transport streams after sending a GOAWAY


"transport stream" is new term not defined elsewhere in the document.

dtikhonov · 2019-07-01T14:07:17Z

draft-ietf-quic-http.md

-unnecessarily limit parallelism.
+to permit these streams to open, an HTTP/3 server SHOULD configure non-zero
+minimum values for the number of permitted streams and the initial flow control
+window for each stream. It is RECOMMENDED that at least 100 requests be


"initial flow control window for each stream" makes it sound as if each stream's flow control window could be configured independently.

dtikhonov · 2019-07-01T14:19:06Z

draft-ietf-quic-transport.md

+
+Applications also need to be informed of state changes on streams, including
+when the peer has initiated, reset, or aborted reading on a stream; when new
+data is available; and when data on the stream is blocked due to flow


when data on the stream is blocked due to flow control

Does this refer to the endpoint's or peer's inability to write?

ianswett

Thanks for doing this, it looks much better.

ianswett · 2019-07-01T14:00:10Z

draft-ietf-quic-transport.md

+
+Applications also need to be informed of state changes on streams, including
+when the peer has initiated, reset, or aborted reading on a stream; when new
+data is available; and when data on the stream is blocked due to flow


Should this be flow or congestion control? And should there be a signal when the stream is unblocked so you can resume sending?

ianswett · 2019-07-01T14:04:32Z

draft-ietf-quic-transport.md

+
+- open a connection, which begins the exchange described in {{handshake}};
+- enable 0-RTT; and
+- be informed when 0-RTT has been accepted or rejected by a server.


I'm not certain the application needs to know about 0-RTT accept/reject, but I believe it does need to know when it can start sending 1RTT data?

The client needs to know because rejection resets the state of all streams; the client has to know whether to start over.

It absolutely needs to know about this because 0-RTT rejection implies the need to start over in some cases. You can't just treat 0-RTT as "lost".

Good point. I still think the client needs to know when 1-RTT data can be sent, because it may want to send some data in 0-RTT but not other data.

But why does an application need to be able to enable 0-RTT? Isn't 0-RTT optional at transport level, and more so at the app level?

An application needs to be able to control whether 0-RTT is offered; you're correct that this phrasing implies that 0-RTT support is mandatory, which wasn't the intention.

ianswett · 2019-07-01T14:06:48Z

draft-ietf-quic-transport.md

+  resumption ticket sent to the client; and
+- if Early Data is supported, retrieve application-controlled data from the
+  client's resumption ticket and enable rejecting Early Data based on that
+  information.


Servers need to expose whether incoming data is Early Data, correct? ie: for the Early Data header.

Added to stream requirements, but now I'm not so sure that's correct. Early-Data has to be added when the server "forwards a request prior to the completion of the TLS handshake". That would mean the server really only needs a yes/no about the current state of the connection when the request is being forwarded. If the handshake has completed, it doesn't matter that the request was originally early -- it's now retroactively validated as a non-replay by the successful handshake.

Agreed, I think it's only yes/no about the current state of the connection.

An app should also be able to reject incoming connections, for example based on implementation specific criteria such as load or peer IP address.

That seems like a worthwhile feature for a QUIC implementation to expose, but doesn't seem like one we should mandate every QUIC implementation must expose.

I can agree to that.

ianswett · 2019-07-01T14:13:55Z

draft-ietf-quic-http.md

-response, it indicates that this response is no longer of interest.
-Implementations SHOULD cancel requests by aborting both directions of a stream.
+Clients can cancel requests by resetting the request stream with an error code
+of HTTP_REQUEST_CANCELLED ({{http-error-codes}}).  When the client aborts


How about "Clients can cancel requests by resetting and aborting the request stream with an error code of HTTP_REQUEST_CANCELLED"? I think that phrasing makes it clearer that you need to reset the write side and abort the read side in order to fully cancel a request?

ianswett · 2019-07-01T14:14:30Z

draft-ietf-quic-http.md

+Clients can cancel requests by resetting the request stream with an error code
+of HTTP_REQUEST_CANCELLED ({{http-error-codes}}).  When the client aborts
+reading a response, it indicates that this response is no longer of interest.
+Implementations SHOULD cancel requests by terminating both directions of a


Do we define terminating?

The terms "clean termination" and "abrupt termination" are given as synonyms for "end the stream" and "reset the stream" in -transport line 361. I should specify the termination type here.

MikeBishop · 2019-08-02T17:54:37Z

@janaiyengar and @martinthomson, I think this is mergeable from an HTTP standpoint. Since it's technically a requirements change (requiring that functionality be exposed), it probably needs to be marked for a consensus call when you think it's ready.

martinthomson · 2019-07-17T16:31:04Z

draft-ietf-quic-transport.md

+
+- open a connection, which begins the exchange described in {{handshake}};
+- enable 0-RTT; and
+- be informed when 0-RTT has been accepted or rejected by a server.


It absolutely needs to know about this because 0-RTT rejection implies the need to start over in some cases. You can't just treat 0-RTT as "lost".

martinthomson · 2019-07-17T16:32:11Z

draft-ietf-quic-transport.md

+stream, it can abort reading the stream and specify an application error code.
+
+If the stream is in the "Recv or "Size Known" states, the transport SHOULD
+signal this by sending a STOP_SENDING frame identifying that stream to prompt


I think that we can lose a few words and make this a little more readable.

Suggested change

signal this by sending a STOP_SENDING frame identifying that stream to prompt

signal this by sending a STOP_SENDING frame to prompt

martinthomson · 2019-08-05T00:06:59Z

draft-ietf-quic-transport.md

+  STOP_SENDING frame ({{frame-stop-sending}})
+
+Applications also need to be informed of state changes on streams, including
+when the peer has initiated, reset, or aborted reading on a stream; when new


List construction here implies that you can "reset reading on a stream" or that you can observe "initiation of reading on a stream". You probably need to say "when the peer has opened or reset a stream; when a peer aborts reading a stream; when new ..."

ianswett · 2019-08-06T21:37:06Z

draft-ietf-quic-transport.md

+  type, as communicated in the transport parameters ({{transport-parameters}});
+- control resource allocation of various types, including flow control and the
+  number of permitted streams of each type;
+- identify whether the handshake has completed successfully or is still ongoing


I think it'd be nice to clarify that a reason for this bullet is to know it can send data in 1-RTT.

Given that hanshake keys must now be held forever, how can it be indicated that a handshake is complete?

The phrases "handshake completes," "handshake is complete," "handshake has completed," etc. occur throughout the document. If you believe that there can be no such concept, please open a separate issue and explain in detail.

I'm not objecting to this term being defined, but I do suspect there is an issue if the now infamous PR #3121 gets merged. But, as it hasn't yet, and there is pushback, I'll see where it goes. Defining exactly what is, or might be, wrong requires a deep analysis because one peer can consider a handshake complete without the other peer agreeing, and sometimes that might be fine while in other cases not so much, like path migration.

ianswett

Some small suggestions, but looks good overall.

mikkelfj

I realize this is merged by now, but I still have a few comments.

mikkelfj · 2019-10-18T06:06:29Z

draft-ietf-quic-transport.md

+- end the stream (clean termination), resulting in a STREAM frame
+  ({{frame-stream}}) with the FIN bit set; and
+- reset the stream (abrupt termination), resulting in a RESET_STREAM frame
+  ({{frame-reset-stream}}), even if the stream was already ended.


This is unclear - what does it mean that "the stream was already ended"? I would say it is fair for an API to issue a soft error if you try to RESET (abrupt termination) after having issued a FIN request (clean termination), or vice version. Certainly I would not want a QUIC implementation tracking a stream handle indefinitely in case some app decides to reset it eventually.

"End the stream" was the previous bullet, so I think it's reasonably clear what "already ended" should mean in this context. If you look at the sending stream states, this reflects the possibility to transition from "Data Sent" (stream has ended) to "Reset Sent" by sending a RESET_STREAM.

Based on the state diagram, if an implementation was already in the "Data Recvd" state, they wouldn't send a RESET_STREAM, as they're already in a terminal state. Surfacing this through the API would be a reasonable choice. Likewise, there's no path from "Reset Sent" or "Reset Recvd" to "Data Sent"; I think attempting to write to or close a reset stream will always be an error.

So you mean to say that an application should be able to reset a stream at an arbitrary point after the stream has sent a FIN bit? If so, I don't think that is a good idea. In my view that would be an API error of invalid handle or invalid handle state. It's one thing to allow races between RESET and FIN but to require it forever is different.

Rather than having this discussion on a closed PR, let's open a separate issue.

mikkelfj · 2019-10-18T06:10:16Z

draft-ietf-quic-transport.md

+
+- open a connection, which begins the exchange described in {{handshake}};
+- enable 0-RTT; and
+- be informed when 0-RTT has been accepted or rejected by a server.


But why does an application need to be able to enable 0-RTT? Isn't 0-RTT optional at transport level, and more so at the app level?

mikkelfj · 2019-10-18T06:12:06Z

draft-ietf-quic-transport.md

+  resumption ticket sent to the client; and
+- if Early Data is supported, retrieve application-controlled data from the
+  client's resumption ticket and enable rejecting Early Data based on that
+  information.


An app should also be able to reject incoming connections, for example based on implementation specific criteria such as load or peer IP address.

mikkelfj · 2019-10-18T06:13:00Z

draft-ietf-quic-transport.md

+  type, as communicated in the transport parameters ({{transport-parameters}});
+- control resource allocation of various types, including flow control and the
+  number of permitted streams of each type;
+- identify whether the handshake has completed successfully or is still ongoing


Given that hanshake keys must now be held forever, how can it be indicated that a handshake is complete?

MikeBishop added 2 commits June 27, 2019 16:01

Required operations in Transport

4c626e3

HTTP rewords for newly-defined terms

337cb98

MikeBishop added design An issue that affects the design of the protocol; resolution requires consensus. -transport -http labels Jun 27, 2019

MikeBishop requested review from martinthomson, LPardue, janaiyengar, ianswett and dtikhonov June 27, 2019 23:14

MikeBishop added 2 commits June 27, 2019 16:15

Verb tense

b4213af

Correct top-level section

88d091a

MikeBishop force-pushed the define-terms branch from 6d4376f to 88d091a Compare June 27, 2019 23:19

Fix bulleted lists

34da6ec

martinthomson reviewed Jun 28, 2019

View reviewed changes

mikkelfj reviewed Jun 28, 2019

View reviewed changes

MikeBishop added 2 commits June 28, 2019 11:04

Review feedback

7656c73

Introduce STOP_SENDING description with 'abort reading'

36ad006

LPardue reviewed Jun 29, 2019

View reviewed changes

LPardue mentioned this pull request Jul 1, 2019

Trigger unhappy #2677

Closed

dtikhonov suggested changes Jul 1, 2019

View reviewed changes

ianswett reviewed Jul 1, 2019

View reviewed changes

Updates from review feedback

ca7b3ed

martinthomson mentioned this pull request Jul 17, 2019

Tradeoffs with padding #2905

Closed

Early-Data isn't per-stream, it's connection handshake state

a66d991

martinthomson reviewed Aug 5, 2019

View reviewed changes

ianswett reviewed Aug 6, 2019

View reviewed changes

ianswett approved these changes Aug 6, 2019

View reviewed changes

More feedback

5579d57

MikeBishop force-pushed the define-terms branch from a9aaa49 to 5579d57 Compare August 6, 2019 22:20

ianswett mentioned this pull request Aug 15, 2019

Preface QUIC frames with "QUIC" #2967

Closed

martinthomson merged commit 1ee2798 into master Aug 16, 2019

juliens mentioned this pull request Oct 12, 2019

Draft 23 juliens/quic-go#1

Closed

28 tasks

kazuho mentioned this pull request Oct 16, 2019

HTTP/3 has less concurrency than HTTP/2 #2787

Closed

mikkelfj reviewed Oct 18, 2019

View reviewed changes

MikeBishop mentioned this pull request Oct 28, 2019

Transport API: Reset after FIN? #3163

Closed

MikeBishop deleted the define-terms branch March 18, 2020 19:42


		On the sending part of a stream:

		- Attempt to write data, understanding when stream flow control credit


		When implementing the client role:

		- Open a connection, which begins the exchange described in {{handshake}}

	signal this by sending a STOP_SENDING frame identifying that stream to prompt
	signal this by sending a STOP_SENDING frame to prompt

Define terms for application actions #2857

Define terms for application actions #2857

Conversation

MikeBishop commented Jun 27, 2019

martinthomson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ianswett left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MikeBishop commented Aug 2, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ianswett left a comment

Choose a reason for hiding this comment

mikkelfj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment