From 4f6ceaac46c94bd093a38e39f86b79ac0b2e37d7 Mon Sep 17 00:00:00 2001 From: Jana Iyengar Date: Fri, 2 Nov 2018 21:47:30 +0800 Subject: [PATCH 1/7] Rework of stream sections --- draft-ietf-quic-transport.md | 501 +++++++++++++++++------------------ 1 file changed, 242 insertions(+), 259 deletions(-) diff --git a/draft-ietf-quic-transport.md b/draft-ietf-quic-transport.md index 6a2f146890..8ba851639c 100644 --- a/draft-ietf-quic-transport.md +++ b/draft-ietf-quic-transport.md @@ -193,7 +193,9 @@ Server: Endpoint: -: The client or server end of a connection. +: An entity that can participate in a QUIC conversation by generating, + receiving, and fully processing QUIC packets. There are only two types of + endpoint in QUIC: client and server. Stream: @@ -216,6 +218,10 @@ QUIC packet: QUIC is a name, not an acronym. +Application: + +: An entity that uses QUIC to send and receive data. + ## Notational Conventions @@ -240,17 +246,16 @@ x (*) ... # Streams {#streams} -Streams in QUIC provide a lightweight, ordered byte-stream abstraction. + -There are two basic types of stream in QUIC. Unidirectional streams carry data -in one direction: from the initiator of the stream to its peer; -bidirectional streams allow for data to be sent in both directions. Different -stream identifiers are used to distinguish between unidirectional and -bidirectional streams, as well as to create a separation between streams that -are initiated by the client and server (see {{stream-id}}). +Streams in QUIC provide a lightweight, ordered byte-stream abstraction to an +application. An alternative view of QUIC streams is as an elastic "message" +abstraction, similar to the way ephemeral streams are used in SST +{{?SST=DOI.10.1145/1282427.1282421}}, which may be a more appealing description +for some applications. -Either type of stream can be created by either endpoint, can concurrently send -data interleaved with other streams, and can be cancelled. +QUIC does not provide any means of ensuring ordering between octets on different +streams. Streams can be created by sending data. Other processes associated with stream management - ending, cancelling, and managing flow control - are all designed to @@ -258,152 +263,95 @@ impose minimal overheads. For instance, a single STREAM frame ({{frame-stream}}) can open, carry data for, and close a stream. Streams can also be long-lived and can last the entire duration of a connection. -Stream offsets allow for the bytes on a stream to be placed in order. An -endpoint MUST be capable of delivering data received on a stream in order. -Implementations MAY choose to offer the ability to deliver data out of order. -There is no means of ensuring ordering between bytes on different streams. +QUIC allows for an arbitrary number of streams to operate concurrently and for +an arbitrary amount of data to be sent on any stream, subject to flow control +constraints (see {{flow-control}}). -Streams are individually flow controlled, allowing an endpoint to limit memory -commitment and to apply back pressure. The creation of streams is also flow -controlled, with each peer declaring the maximum stream ID it is willing to -accept at a given time. +## Stream Types and Identifiers {#stream-id} -An alternative view of QUIC streams is as an elastic "message" abstraction, -similar to the way ephemeral streams are used in SST -{{?SST=DOI.10.1145/1282427.1282421}}, which may be a more appealing description -for some applications. + +Streams can be unidirectional or bidirectional. Unidirectional streams carry +data in one direction: from the initiator of the stream to its peer. +Bidirectional streams allow for data to be sent in both directions. Streams can +be created by either endpoint, can concurrently send data interleaved with other +streams, and can be cancelled. Any stream can be initiated by either enndpoint. -## Stream Identifiers {#stream-id} - -Streams are identified by an unsigned 62-bit integer, referred to as the Stream -ID. Stream IDs are encoded as a variable-length integer (see -{{integer-encoding}}). The least significant two bits of the Stream ID are used -to identify the type of stream (unidirectional or bidirectional) and the -initiator of the stream. +Streams are identified within a connection by a numeric value, referred to as +the Stream ID. Stream IDs are unique to a stream: a QUIC endpoint MUST NOT +reuse a Stream ID within a connection. Stream IDs are encoded as a +variable-length integer (see {{integer-encoding}}). The least significant bit (0x1) of the Stream ID identifies the initiator of the -stream. Clients initiate even-numbered streams (those with the least -significant bit set to 0); servers initiate odd-numbered streams (with the bit -set to 1). Separation of the stream identifiers ensures that client and server -are able to open streams without the latency imposed by negotiating for an -identifier. - -The second least significant bit (0x2) of the Stream ID differentiates between -unidirectional streams and bidirectional streams. Unidirectional streams always -have this bit set to 1 and bidirectional streams have this bit set to 0. - -The two type bits from a Stream ID therefore identify streams as summarized in -{{stream-id-types}}. - -| Low Bits | Stream Type | -|:---------|:---------------------------------| -| 0x0 | Client-Initiated, Bidirectional | -| 0x1 | Server-Initiated, Bidirectional | -| 0x2 | Client-Initiated, Unidirectional | -| 0x3 | Server-Initiated, Unidirectional | +stream. Client-initiated streams have even-numbered Stream IDs (with the bit +set to 0), and server-initiated streams have odd-numbered Stream IDs (with the +bit set to 1). + +The second least significant bit (0x2) of the Stream ID distinguishes between +bidirectional streams (with the bit set to 0) and unidirectional streams (with +the bit set to 1). + +The least significant two bits from a Stream ID therefore identify a stream as +one of four types, as summarized in {{stream-id-types}}. + +| Bits | Stream Type | +|:-----|:---------------------------------| +| 0x0 | Client-Initiated, Bidirectional | +| 0x1 | Server-Initiated, Bidirectional | +| 0x2 | Client-Initiated, Unidirectional | +| 0x3 | Server-Initiated, Unidirectional | {: #stream-id-types title="Stream ID Types"} -The first bidirectional stream opened by the client is stream 0. - -A QUIC endpoint MUST NOT reuse a Stream ID. Streams of each type are created in -numeric order. Streams that are used out of order result in opening all -lower-numbered streams of the same type in the same direction. - +Within each type, streams are created with numerically increasing Stream IDs. A +Stream ID that is used out of order results in all lower-numbered streams of +that type also being opened. -## Stream Concurrency {#stream-concurrency} - -QUIC allows for an arbitrary number of streams to operate concurrently. An -endpoint limits the number of concurrently active incoming streams by limiting -the number of streams of each type (see {{stream-limit-increment}}). - -The stream limit is specific to each endpoint and applies only to the peer that -receives the setting. That is, the client limits the number of streams of each -type the server can initiate, and the server limits the number of streams the -client can initiate. Each endpoint can respond on streams initiated by the -other peer, regardless of whether it is permitted to initiate new streams. - -Endpoints MUST NOT exceed the limit set by their peer. An endpoint that -receives a STREAM frame with an ID greater than the limit it has sent MUST treat -this as a stream error of type STREAM_LIMIT_ERROR ({{error-handling}}). - -A receiver cannot renege on an advertisement; that is, once a receiver -advertises a stream limit using the MAX_STREAMS frame, advertising a smaller -limit has no effect. A receiver MUST ignore any MAX_STREAMS frame that does -not increase the stream limit. +The first bidirectional stream opened by the client has a Stream ID of 0. ## Sending and Receiving Data -Endpoints uses streams to send and receive data. Endpoints send STREAM frames, -which encapsulate data for a stream. STREAM frames carry a flag that can be used -to signal the end of a stream. +Endpoints use STREAM frames to encapsulate data sent by an application on a +stream ({{frame-stream}}). An endpoint uses the stream ID and offset fields in a +received STREAM frame to place the contained data in order within the stream. -Streams are an ordered byte-stream abstraction with no other structure that is -visible to QUIC. STREAM frame boundaries are not expected to preserved when data -is transmitted, when data is retransmitted after packet loss, or when data is -delivered to the application at the receiver. +Endpoints MUST be able to deliver stream data to an application as an ordered +byte-stream. Delivering an ordered byte-stream requires that an endpoint buffer +any data that is received out of order, up to the advertised flow control limit. -When new data is to be sent on a stream, a sender MUST set the encapsulating -STREAM frame's offset field to the stream offset of the first byte of this new -data. The first byte of data on a stream has an offset of 0. An endpoint is -expected to send every stream byte. The largest offset delivered on a stream -MUST be less than 2^62. +QUIC makes no specific allowances for delivery of stream data out of +order. However, implementations MAY choose to offer the ability to deliver data +out of order to a receiving application. -QUIC makes no specific allowances for partial reliability or delivery of stream -data out of order. Endpoints MUST be able to deliver stream data to an -application as an ordered byte-stream. Delivering an ordered byte-stream -requires that an endpoint buffer any data that is received out of order, up to -the advertised flow control limit. +An endpoint could receive data for a stream at the same stream offset multiple +times; data that has already been received can be discarded. The data at a +given offset MUST NOT change if it is sent multiple times; an endpoint MAY treat +receipt of different data at the same offset within a stream as a connection +error of type PROTOCOL_VIOLATION. -An endpoint could receive the same bytes multiple times; bytes that have already -been received can be discarded. The value for a given byte MUST NOT change if -it is sent multiple times; an endpoint MAY treat receipt of a changed byte as a -connection error of type PROTOCOL_VIOLATION. +Streams are an ordered byte-stream abstraction with no other structure that is +visible to QUIC. STREAM frame boundaries are not expected to be preserved when +data is transmitted, when data is retransmitted after packet loss, or when data +is delivered to the application at a receiver. An endpoint MUST NOT send data on any stream without ensuring that it is within -the data limits set by its peer. Flow control is described in detail in +the flow control limits set by its peer. Flow control is described in detail in {{flow-control}}. -## Stream Prioritization +## Stream Prioritization {#stream-prioritization} -Stream multiplexing has a significant effect on application performance if -resources allocated to streams are correctly prioritized. Experience with other -multiplexed protocols, such as HTTP/2 {{?HTTP2}}, shows that effective -prioritization strategies have a significant positive impact on performance. +Stream multiplexing can have a significant effect on application performance if +resources allocated to streams are correctly prioritized. QUIC does not provide frames for exchanging prioritization information. Instead it relies on receiving priority information from the application that uses QUIC. -Protocols that use QUIC are able to define any prioritization scheme that suits -their application semantics. A protocol might define explicit messages for -signaling priority, such as those defined in HTTP/2; it could define rules that -allow an endpoint to determine priority based on context; or it could leave the -determination to the application. A QUIC implementation SHOULD provide ways in which an application can indicate the relative priority of streams. When deciding which streams to dedicate -resources to, QUIC SHOULD use the information provided by the application. -Failure to account for priority of streams can result in suboptimal performance. - -Stream priority is most relevant when deciding which stream data will be -transmitted. Often, there will be limits on what can be transmitted as a result -of connection flow control or the current congestion controller state. - -Giving preference to the transmission of its own management frames ensures that -the protocol functions efficiently. That is, prioritizing frames other than -STREAM frames ensures that loss recovery, congestion control, and flow control -operate effectively. - -CRYPTO frames SHOULD be prioritized over other streams prior to the completion -of the cryptographic handshake. This includes the retransmission of the second -flight of client handshake messages, that is, the TLS Finished and any client -authentication messages. - -STREAM data in frames determined to be lost SHOULD be retransmitted before -sending new data, unless application priorities indicate otherwise. -Retransmitting lost stream data can fill in gaps, which allows the peer to -consume already received data and free up the flow control window. +resources to, the implementation SHOULD use the information provided by the +application. Failure to account for priority of streams can result in +suboptimal application performance. # Stream States: Life of a Stream {#stream-states} @@ -725,102 +673,92 @@ data or a RST_STREAM frame has been received for the stream - that is, the stream is in any state other than "Recv" or "Size Known" - sending a STOP_SENDING frame is unnecessary. +An endpoint that wishes to terminate both directions of a bidirectional stream +can terminate one direction by sending a RST_STREAM, and it can encourage prompt +termination in the opposite direction by sending a STOP_SENDING frame. + # Flow Control {#flow-control} -It is necessary to limit the amount of data that a sender may have outstanding -at any time, so as to prevent a fast sender from overwhelming a slow receiver, -or to prevent a malicious sender from consuming significant resources at a -receiver. To this end, QUIC employs a credit-based flow-control scheme similar -to that in HTTP/2 {{?HTTP2}}. A receiver advertises the number of bytes it is -prepared to receive on a given stream and for the entire connection. This leads -to two levels of flow control in QUIC: +It is necessary to limit the amount of data that a receiver may have to buffer, +so as to prevent a fast sender from overwhelming a slow receiver, or to prevent +a malicious sender from consuming a large amount of memory at a receiver. To +enable a receiver to limit memory commitment to a connection and to apply back +pressure on the sender, streams are flow controlled both individually and as an +aggregate. A QUIC receiver controls the maximum amount of data the sender can +send on a stream at any time, as described in {{data-flow-control}} and +{{fc-credit}} + +Similarly, to limit concurrency within a connection, a QUIC endpoint controls +the maximum number of streams that its peer can initiate at any time, as +described in {{controlling-concurrency}}. + +Data sent in CRYPTO frames is not flow controlled in the same way as stream +data. QUIC relies on the cryptographic protocol implementation to avoid +excessive buffering of data, see {{QUIC-TLS}}. The implementation SHOULD +provide an interface to QUIC to tell it about its buffering limits so that there +is not excessive buffering at multiple layers. + + +## Data Flow Control {#data-flow-control} + +QUIC employs a credit-based flow-control scheme similar to that in HTTP/2 +{{?HTTP2}}, where a receiver advertises the number of bytes it is prepared to +receive on a given stream and for the entire connection. This leads to two +levels of data flow control in QUIC: * Stream flow control, which prevents a single stream from consuming the entire - receive buffer for a connection. + receive buffer for a connection by limiting the amount of data that can be + sent on any stream. * Connection flow control, which prevents senders from exceeding a receiver's - buffer capacity for the connection. - -A data receiver sets initial credits for all streams by sending transport -parameters during the handshake ({{transport-parameters}}). + buffer capacity for the connection, by limiting the total bytes of stream data + sent in STREAM frames on all streams. -A data receiver sends MAX_STREAM_DATA or MAX_DATA frames to the sender to -advertise additional credit. MAX_STREAM_DATA frames send the maximum absolute -byte offset of a stream, while MAX_DATA frames send the maximum of the sum of -the absolute byte offsets of all streams. +A receiver sets initial credits for all streams by sending transport parameters +during the handshake ({{transport-parameters}}). A receiver sends +MAX_STREAM_DATA ({{frame-max-stream-data}}) or MAX_DATA ({{frame-max-data}}) +frames to the sender to advertise additional credit. A receiver advertises credit for a stream by sending a MAX_STREAM_DATA frame -with the Stream ID set appropriately. A receiver could use the current offset of -data consumed to determine the flow control offset to be advertised. A receiver -MAY send MAX_STREAM_DATA frames in multiple packets in order to make sure that -the sender receives an update before running out of flow control credit, even if -one of the packets is lost. - -Connection flow control is a limit to the total bytes of stream data sent in -STREAM frames on all streams. A receiver advertises credit for a connection by -sending a MAX_DATA frame. A receiver maintains a cumulative sum of bytes -received on all contributing streams, which are used to check for flow control -violations. A receiver might use a sum of bytes consumed on all streams to -determine the maximum data limit to be advertised. - -A receiver MAY advertise a larger offset at any point by sending MAX_STREAM_DATA -or MAX_DATA frames. A receiver cannot renege on an advertisement; that is, once -a receiver advertises an offset, advertising a smaller offset has no effect. A -sender MUST therefore ignore any MAX_STREAM_DATA or MAX_DATA frames that do not -increase flow control limits. +with the Stream ID set appropriately. A MAX_STREAM_DATA frame indicates the +maximum absolute byte offset of a stream. A receiver could use the current +offset of data consumed to determine the flow control offset to be advertised. +A receiver MAY send MAX_STREAM_DATA frames in multiple packets in order to make +sure that the sender receives an update before running out of flow control +credit, even if one of the packets is lost. + +A receiver advertises credit for a connection by sending a MAX_DATA frame, which +indicates the maximum of the sum of the absolute byte offsets of all streams. A +receiver maintains a cumulative sum of bytes received on all contributing +streams, which is used to check for flow control violations. A receiver might +use a sum of bytes consumed on all streams to determine the maximum data limit +to be advertised. + +A receiver can advertise a larger offset by sending MAX_STREAM_DATA or MAX_DATA +frames at any time during the connection. A receiver cannot renege on an +advertisement however. That is, once a receiver advertises an offset, +advertising a smaller offset has no effect. A sender MUST therefore ignore any +MAX_STREAM_DATA or MAX_DATA frames that do not increase flow control limits. A receiver MUST close the connection with a FLOW_CONTROL_ERROR error -({{error-handling}}) if the peer violates the advertised connection or stream +({{error-handling}}) if the sender violates the advertised connection or stream data limits. -A sender SHOULD send STREAM_DATA_BLOCKED or DATA_BLOCKED frames to indicate it -has data to write but is blocked by flow control limits. These frames are -expected to be sent infrequently in common cases, but they are considered useful -for debugging and monitoring purposes. - -A similar method is used to control the number of open streams (see -{{stream-limit-increment}} for details). - +If a sender runs out of flow control credit, it will be unable to send new data +and is considered blocked. A sender SHOULD send STREAM_DATA_BLOCKED or +DATA_BLOCKED frames to indicate it has data to write but is blocked by flow +control limits. These frames are expected to be sent infrequently in common +cases, but they are considered useful for debugging and monitoring purposes. -## Handling of Stream Cancellation +A sender sends a single STREAM_DATA_BLOCKED or DATA_BLOCKED frame only once when +it reaches a data limit. A sender SHOULD NOT send multiple STREAM_DATA_BLOCKED +or DATA_BLOCKED frames for the same data limit, unless the original frame is +determined to be lost. Another STREAM_DATA_BLOCKED or DATA_BLOCKED frame can be +sent after the data limit is increased. -There are some edge cases which must be considered when dealing with stream and -connection level flow control. Given enough time, both endpoints must agree on -flow control state. If one end believes it can send more than the other end is -willing to receive, the connection will be torn down when too much data arrives. -Conversely if a sender believes it is blocked, while endpoint B expects more -data can be received, then the connection can be in a deadlock, with the sender -waiting for a MAX_STREAM_DATA or MAX_DATA frame which will never come. -On receipt of a RST_STREAM frame, an endpoint will tear down state for the -matching stream and ignore further data arriving on that stream. This could -result in the endpoints getting out of sync, since the RST_STREAM frame may have -arrived out of order and there could be more data in flight. The data sender -would have counted the data against its connection level flow control budget, -but a receiver that has not received these bytes would not know to include them -as well. The receiver must learn the number of bytes that were sent on the -stream to make the same adjustment in its connection flow controller. - -To ensure that endpoints maintain a consistent connection-level flow control -state, the RST_STREAM frame ({{frame-rst-stream}}) includes the largest offset -of data sent on the stream. On receiving a RST_STREAM frame, a receiver -definitively knows how many bytes were sent on that stream before the RST_STREAM -frame, and the receiver MUST use the final offset to account for all bytes sent -on the stream in its connection level flow controller. - -RST_STREAM terminates one direction of a stream abruptly. Whether any action or -response can or should be taken on the data already received is application -specific. - -For a bidirectional stream, RST_STREAM has no effect on data flow in the -opposite direction. The RST_STREAM sender can send a STOP_SENDING frame to -encourage prompt termination. Both endpoints MUST maintain state for the stream -in the unterminated direction until that direction enters a terminal state, or -either side sends CONNECTION_CLOSE. - - -## Data Limit Increments {#fc-credit} +## Flow Credit Increments {#fc-credit} This document leaves when and how many bytes to advertise in a MAX_STREAM_DATA or MAX_DATA frame to implementations, but offers a few considerations. These @@ -831,29 +769,48 @@ larger resource commitments at the receiver. Thus there is a trade-off between resource commitment and overhead when determining how large a limit is advertised. -A receiver MAY use an autotuning mechanism to tune the frequency and amount that -it increases data limits based on a round-trip time estimate and the rate at -which the receiving application consumes data, similar to common TCP -implementations. +A receiver MAY use an autotuning mechanism to tune the frequency and amount of +advertised additional credit. -If a sender runs out of flow control credit, it will be unable to send new -data. That is, the sender is blocked. A blocked sender SHOULD send a -STREAM_DATA_BLOCKED or DATA_BLOCKED frame. A receiver uses these frames for -debugging purposes. A receiver MUST NOT wait for a STREAM_DATA_BLOCKED or -DATA_BLOCKED frame before sending MAX_STREAM_DATA or MAX_DATA, since doing so -will mean that a sender will be blocked for an entire round trip and the peer -might never send a STREAM_DATA_BLOCKED or DATA_BLOCKED frame. +If a sender runs out of flow control credit, it will be unable to send new data +and is considered blocked. It is generally considered best to not let the +sender go into quiescence. To avoid blocking a sender, and to reasonably +account for the possibility of loss, a receiver should send a MAX_DATA or +MAX_STREAM_DATA frame at least two round trips before it expects the sender to +get blocked. -It is generally considered best to not let the sender go into quiescence if -avoidable. To avoid blocking a sender, and to reasonably account for the -possibility of loss, a receiver should send a MAX_DATA or MAX_STREAM_DATA frame -at least two round trips before it expects the sender to get blocked. + -A sender sends a single STREAM_DATA_BLOCKED or DATA_BLOCKED frame only once when -it reaches a data limit. A sender SHOULD NOT send multiple STREAM_DATA_BLOCKED -or DATA_BLOCKED frames for the same data limit, unless the original frame is -determined to be lost. Another STREAM_DATA_BLOCKED or DATA_BLOCKED frame can be -sent after the data limit is increased. +A receiver MUST NOT wait for a STREAM_DATA_BLOCKED or DATA_BLOCKED frame before +sending MAX_STREAM_DATA or MAX_DATA, since doing so will mean that a sender will +be blocked for at least an entire round trip, and potentially for longer if the +peer chooses to not send STREAM_DATA_BLOCKED or DATA_BLOCKED frames. + + +## Handling Stream Cancellation {#stream-cancellation} + +Given enough time, both endpoints must agree on flow control state, to avoid +flow control violations or deadlock. + +On receipt of a RST_STREAM frame, an endpoint will tear down state for the +matching stream and ignore further data arriving on that stream. If a +RST_STREAM frame is reordered with stream data for the same stream, the +receiver's estimate of the number of octets received on that stream can be lower +than the sender's estimate of the number sent. As a result, the two endpoints +could disagree on the number of octets that count towards connection flow +control. + +To remedy this issue, a RST_STREAM frame ({{frame-rst-stream}}) includes the +final offset of data sent on the stream. On receiving a RST_STREAM frame, a +receiver definitively knows how many bytes were sent on that stream before the +RST_STREAM frame, and the receiver MUST use the final offset to account for all +bytes sent on the stream in its connection level flow controller. + +RST_STREAM terminates one direction of a stream abruptly. For a bidirectional +stream, RST_STREAM has no effect on data flow in the opposite direction. Both +endpoints MUST maintain flow control state for the stream in the unterminated +direction until that direction enters a terminal state, or until one of the +endpoints sends CONNECTION_CLOSE. ## Stream Final Offset {#final-offset} @@ -865,45 +822,48 @@ data carried in a STREAM frame marked with a FIN flag, or 0 in the case of incoming unidirectional streams. An endpoint will know the final offset for a stream when the receive stream -enters the "Size Known" or "Reset Recvd" state. +enters the "Size Known" or "Reset Recvd" state ({{stream-states}}). An endpoint MUST NOT send data on a stream at or beyond the final offset. Once a final offset for a stream is known, it cannot change. If a RST_STREAM or -STREAM frame causes the final offset to change for a stream, an endpoint SHOULD -respond with a FINAL_OFFSET_ERROR error (see {{error-handling}}). A receiver -SHOULD treat receipt of data at or beyond the final offset as a -FINAL_OFFSET_ERROR error, even after a stream is closed. Generating these -errors is not mandatory, but only because requiring that an endpoint generate -these errors also means that the endpoint needs to maintain the final offset -state for closed streams, which could mean a significant state commitment. - - -## Flow Control for Cryptographic Handshake {#flow-control-crypto} - -Data sent in CRYPTO frames is not flow controlled in the same way as STREAM -frames. QUIC relies on the cryptographic protocol implementation to avoid -excessive buffering of data, see {{QUIC-TLS}}. The implementation SHOULD -provide an interface to QUIC to tell it about its buffering limits so that there -is not excessive buffering at multiple layers. - +STREAM frame is received indicating a change in the final offset for the stream, +an endpoint SHOULD respond with a FINAL_OFFSET_ERROR error (see +{{error-handling}}). A receiver SHOULD treat receipt of data at or beyond the +final offset as a FINAL_OFFSET_ERROR error, even after a stream is closed. +Generating these errors is not mandatory, but only because requiring that an +endpoint generate these errors also means that the endpoint needs to maintain +the final offset state for closed streams, which could mean a significant state +commitment. + +## Controlling Concurrency {#controlling-concurrency} + +An endpoint controls concurrency by limiting the total number of incoming +streams. An initial value is set in the transport parameters (see +{{transport-parameter-definitions}}) and subsequently increments are advertised +using MAX_STREAMS frames ({{frame-max-streams}}). Separate limits apply to +unidirectional and bidirectional streams. -## Stream Limit Increment {#stream-limit-increment} +Endpoints MUST NOT exceed the limit set by their peer. An endpoint that +receives a STREAM frame with a stream ID exceeding the limit it has sent MUST +treat this as a stream error of type STREAM_LIMIT_ERROR ({{error-handling}}). -An endpoint limits the number of concurrently active incoming streams. An -initial value is set in the transport parameters (see -{{transport-parameter-definitions}}) and is subsequently increased by -MAX_STREAMS frames (see {{frame-max-streams}}). Separate limits apply to -bidirectional and unidirectional streams. +A receiver cannot renege on an advertisement. That is, once a receiver +advertises a stream limit using the MAX_STREAMS frame, advertising a smaller +limit has no effect. A receiver MUST ignore any MAX_STREAMS frame that does not +increase the stream limit. As with stream and connection flow control, this document leaves when and how -many streams to make available to a peer via MAX_STREAMS to implementations. +many streams to advertise to a peer via MAX_STREAMS to implementations. Implementations might choose to increase limits as streams close to keep the number of streams available to peers roughly consistent. -The STREAMS_BLOCKED frame ({{frame-streams-blocked}}) signals that a new stream -could not be created. Implementations can use this as a signal to the peer to -send a MAX_STREAMS frame with a larger limit. +An endpoint that is unable to open a new stream due to the peer's limits SHOULD +send a STREAMS_BLOCKED frame ({{frame-streams-blocked}}). This signal is +considered useful for debugging. An endpoint SHOULD NOT wait to receive this +signal before advertising additional credit, since doing so will mean that the +peer will be blocked for at least an entire round trip, and potentially for +longer if the peer chooses to not send STREAMS_BLOCKED frames. # Connections {#connections} @@ -2762,6 +2722,28 @@ streams as necessary in outgoing packets without losing transmission efficiency to underfilled packets. +## Frame Priority {#frame-priority} + +Often, there will be limits on what can be transmitted as a result of connection +flow control or the current congestion controller state. + +Giving preference to the transmission of its own management functions ensures +that a protocol functions efficiently. That is, prioritizing frames other than +STREAM frames ensures that loss recovery, congestion control, and flow control +operate effectively. + +CRYPTO frames SHOULD be prioritized over STREAM frames prior to the completion +of the cryptographic handshake. This includes the retransmission of the second +flight of client handshake messages, that is, the TLS Finished and any client +authentication messages. + +STREAM data in frames determined to be lost SHOULD be retransmitted before +sending new data, unless priorities specified by the application indicate +otherwise (see {{stream-prioritization}}). Retransmitting lost stream data can +fill in gaps, which allows the peer to consume already received data and free up +the flow control window. + + ## Packet Processing and Acknowledgment {#processing-and-ack} A packet MUST NOT be acknowledged until packet protection has been successfully @@ -4189,8 +4171,7 @@ The fields in the MAX_STREAM_DATA frame are as follows: Stream ID: -: The stream ID of the stream that is affected encoded as a variable-length - integer. +: The Stream ID of the affected stream, encoded as a variable-length integer. Maximum Stream Data: @@ -4838,9 +4819,11 @@ Stream Data: When a Stream Data field has a length of 0, the offset in the STREAM frame is the offset of the next byte that would be sent. -The first byte in the stream has an offset of 0. The largest offset delivered -on a stream - the sum of the re-constructed offset and data length - MUST be -less than 2^62. +When new data is to be sent on a stream, a sender MUST set the encapsulating +STREAM frame's offset field to the stream offset of the first byte of this new +data. The first byte in the stream has an offset of 0. The largest offset +delivered on a stream - the sum of the re-constructed offset and data length - +MUST be less than 2^62. ## CRYPTO Frame {#frame-crypto} From beea504e3d832d7358621c7e72f4172be058bf77 Mon Sep 17 00:00:00 2001 From: Jana Iyengar Date: Fri, 2 Nov 2018 22:55:11 +0800 Subject: [PATCH 2/7] rework of stream states --- draft-ietf-quic-transport.md | 147 ++++++++++++++++++----------------- 1 file changed, 76 insertions(+), 71 deletions(-) diff --git a/draft-ietf-quic-transport.md b/draft-ietf-quic-transport.md index 8ba851639c..d4acf6ae91 100644 --- a/draft-ietf-quic-transport.md +++ b/draft-ietf-quic-transport.md @@ -302,13 +302,12 @@ one of four types, as summarized in {{stream-id-types}}. | 0x3 | Server-Initiated, Unidirectional | {: #stream-id-types title="Stream ID Types"} -Within each type, streams are created with numerically increasing Stream IDs. A -Stream ID that is used out of order results in all lower-numbered streams of -that type also being opened. +Within each type, streams are expected to be created with numerically increasing +Stream IDs. A Stream ID that is used out of order results in all streams of +that type with lower-numbered stream IDs also being opened. The first bidirectional stream opened by the client has a Stream ID of 0. - ## Sending and Receiving Data Endpoints use STREAM frames to encapsulate data sent by an application on a @@ -356,10 +355,10 @@ suboptimal application performance. # Stream States: Life of a Stream {#stream-states} -This section describes the two types of QUIC stream in terms of the states of -their send or receive components. Two state machines are described: one for -streams on which an endpoint transmits data ({{stream-send-states}}); another -for streams from which an endpoint receives data ({{stream-recv-states}}). +This section describes streams in terms of their send or receive components. +Two state machines are described: one for the streams on which an endpoint +transmits data ({{stream-send-states}}), and another for streams on which an +endpoint receives data ({{stream-recv-states}}). Unidirectional streams use the applicable state machine directly. Bidirectional streams use both state machines. For the most part, the use of these state @@ -368,8 +367,9 @@ conditions for opening a stream are slightly more complex for a bidirectional stream because the opening of either send or receive sides causes the stream to open in both directions. -An endpoint can open streams up to its maximum stream limit in any order, -however endpoints SHOULD open the send side of streams for each type in order. +An endpoint can open streams up to its maximum stream limit in any order of +stream ID. However, within each type, endpoints SHOULD open the send side of +streams in numerically increasing order of stream ID. Note: @@ -425,10 +425,10 @@ data to a peer. {: #fig-stream-send-states title="States for Send Streams"} The sending part of stream that the endpoint initiates (types 0 and 2 for -clients, 1 and 3 for servers) is opened by the application or application -protocol. The "Ready" state represents a newly created stream that is able to -accept data from the application. Stream data might be buffered in this state -in preparation for sending. +clients, 1 and 3 for servers) is opened by the application. The "Ready" state +represents a newly created stream that is able to accept data from the +application. Stream data might be buffered in this state in preparation for +sending. Sending the first STREAM or STREAM_DATA_BLOCKED frame causes a send stream to enter the "Send" state. An implementation might choose to defer allocating a @@ -437,27 +437,30 @@ which can allow for better stream prioritization. The sending part of a bidirectional stream initiated by a peer (type 0 for a server, type 1 for a client) enters the "Ready" state then immediately -transitions to the "Send" state if the receiving part enters the "Recv" state. - -In the "Send" state, an endpoint transmits - and retransmits as necessary - data -in STREAM frames. The endpoint respects the flow control limits of its peer, -accepting MAX_STREAM_DATA frames. An endpoint in the "Send" state generates -STREAM_DATA_BLOCKED frames if it encounters flow control limits. - -After the application indicates that stream data is complete and a STREAM frame -containing the FIN bit is sent, the send stream enters the "Data Sent" state. -From this state, the endpoint only retransmits stream data as necessary. The -endpoint no longer needs to track flow control limits or send -STREAM_DATA_BLOCKED frames for a send stream in this state. The endpoint can -ignore any MAX_STREAM_DATA frames it receives from its peer in this state; -MAX_STREAM_DATA frames might be received until the peer receives the final -stream offset. +transitions to the "Send" state if the receiving part enters the "Recv" state +({{stream-recv-states}}). + +In the "Send" state, an endpoint transmits - and retransmits as necessary - +stream data in STREAM frames. The endpoint respects the flow control limits set +by its peer, and continues to accept and process MAX_STREAM_DATA frames. An +endpoint in the "Send" state generates STREAM_DATA_BLOCKED frames if it is +blocked from sending by stream or connection flow control limits +{{data-flow-control}}. + +After the application indicates that all stream data has been sent and a STREAM +frame containing the FIN bit is sent, the send stream enters the "Data Sent" +state. From this state, the endpoint only retransmits stream data as necessary. +The endpoint does not need to check flow control limits or send +STREAM_DATA_BLOCKED frames for a send stream in this state. MAX_STREAM_DATA +frames might be received until the peer receives the final stream offset. The +endpoint can safely ignore any MAX_STREAM_DATA frames it receives from its peer +for a stream in this state. Once all stream data has been successfully acknowledged, the send stream enters the "Data Recvd" state, which is a terminal state. From any of the "Ready", "Send", or "Data Sent" states, an application can -signal that it wishes to abandon transmission of stream data. Similarly, the +signal that it wishes to abandon transmission of stream data. Alternatively, an endpoint might receive a STOP_SENDING frame from its peer. In either case, the endpoint sends a RST_STREAM frame, which causes the stream to enter the "Reset Sent" state. @@ -474,10 +477,10 @@ enters the "Reset Recvd" state, which is a terminal state. {{fig-stream-recv-states}} shows the states for the part of a stream that receives data from a peer. The states for a receive stream mirror only some of -the states of the send stream at the peer. A receive stream doesn't track -states on the send stream that cannot be observed, such as the "Ready" state; -instead, receive streams track the delivery of data to the application or -application protocol some of which cannot be observed by the sender. +the states of the send stream at the peer. A receive stream does not track +states on the send stream that cannot be observed, such as the "Ready" state. +Instead, receive streams track the delivery of data to the application, some of +which cannot be observed by the sender. ~~~ o @@ -515,26 +518,25 @@ application protocol some of which cannot be observed by the sender. {: #fig-stream-recv-states title="States for Receive Streams"} The receiving part of a stream initiated by a peer (types 1 and 3 for a client, -or 0 and 2 for a server) are created when the first STREAM, STREAM_DATA_BLOCKED, +or 0 and 2 for a server) is created when the first STREAM, STREAM_DATA_BLOCKED, RST_STREAM, or MAX_STREAM_DATA (bidirectional only, see below) is received for -that stream. The initial state for a receive stream is "Recv". Receiving a -RST_STREAM frame causes the receive stream to immediately transition to the -"Reset Recvd". +that stream. The initial state for a receive stream is "Recv". The receive stream enters the "Recv" state when the sending part of a bidirectional stream initiated by the endpoint (type 0 for a client, type 1 for a server) enters the "Ready" state. -A bidirectional stream also opens when a MAX_STREAM_DATA frame is received. -Receiving a MAX_STREAM_DATA frame implies that the remote peer has opened the -stream and is providing flow control credit. A MAX_STREAM_DATA frame might -arrive before a STREAM or STREAM_DATA_BLOCKED frame if packets are lost or -reordered. +An endpoint opens a bidirectional stream when a MAX_STREAM_DATA frame is +received from the peer for that stream. Receiving a MAX_STREAM_DATA frame for +an unopened stream indicates that the remote peer has opened the stream and is +providing flow control credit. A MAX_STREAM_DATA frame might arrive before a +STREAM or STREAM_DATA_BLOCKED frame if packets are lost or reordered. -Before creating a stream, all lower-numbered streams of the same type MUST be -created. That means that receipt of a frame that would open a stream causes all -lower-numbered streams of the same type to be opened in numeric order. This -ensures that the creation order for streams is consistent on both endpoints. +Before creating a stream, all streams of the same type with lower-numbered +stream IDs MUST be created. This means that receipt of a frame that would open +a stream causes all streams of the same type with lower-numbered stream IDs to +be opened in numeric order. This ensures that the creation order for streams is +consistent on both endpoints. In the "Recv" state, the endpoint receives STREAM and STREAM_DATA_BLOCKED frames. Incoming data is buffered and can be reassembled into the correct order @@ -542,10 +544,10 @@ for delivery to the application. As data is consumed by the application and buffer space becomes available, the endpoint sends MAX_STREAM_DATA frames to allow the peer to send more data. -When a STREAM frame with a FIN bit is received, the final offset (see -{{final-offset}}) is known. The receive stream enters the "Size Known" state. -In this state, the endpoint no longer needs to send MAX_STREAM_DATA frames, it -only receives any retransmissions of stream data. +When a STREAM frame with a FIN bit is received, the final offset is known (see +{{final-offset}}). The receive stream enters the "Size Known" state. In this +state, the endpoint no longer needs to send MAX_STREAM_DATA frames, it only +receives any retransmissions of stream data. Once all data for the stream has been received, the receive stream enters the "Data Recvd" state. This might happen as a result of receiving the same STREAM @@ -554,8 +556,8 @@ has all stream data. Any STREAM or STREAM_DATA_BLOCKED frames it receives for the stream can be discarded. The "Data Recvd" state persists until stream data has been delivered to the -application or application protocol. Once stream data has been delivered, the -stream enters the "Data Read" state, which is a terminal state. +application. Once stream data has been delivered, the stream enters the "Data +Read" state, which is a terminal state. Receiving a RST_STREAM frame in the "Recv" or "Size Known" states causes the stream to enter the "Reset Recvd" state. This might cause the delivery of @@ -564,15 +566,15 @@ stream data to the application to be interrupted. It is possible that all stream data is received when a RST_STREAM is received (that is, from the "Data Recvd" state). Similarly, it is possible for remaining stream data to arrive after receiving a RST_STREAM frame (the "Reset Recvd" -state). An implementation is able to manage this situation as they choose. +state). An implementation is free to manage this situation as it chooses. Sending RST_STREAM means that an endpoint cannot guarantee delivery of stream data; however there is no requirement that stream data not be delivered if a RST_STREAM is received. An implementation MAY interrupt delivery of stream -data, discard any data that was not consumed, and signal the existence of the +data, discard any data that was not consumed, and signal the receipt of the RST_STREAM immediately. Alternatively, the RST_STREAM signal might be -suppressed or withheld if stream data is completely received. In the latter -case, the receive stream effectively transitions to "Data Recvd" from "Reset -Recvd". +suppressed or withheld if stream data is completely received and is buffered to +be read by the application. In the latter case, the receive stream transitions +from "Reset Recvd" to "Data Recvd". Once the application has been delivered the signal indicating that the receive stream was reset, the receive stream transitions to the "Reset Read" state, @@ -588,9 +590,11 @@ STREAM_DATA_BLOCKED ({{frame-stream-data-blocked}}), and RST_STREAM A sender MUST NOT send any of these frames from a terminal state ("Data Recvd" or "Reset Recvd"). A sender MUST NOT send STREAM or STREAM_DATA_BLOCKED after -sending a RST_STREAM; that is, in the "Reset Sent" state in addition to the -terminal states. A receiver could receive any of these frames in any state, but -only due to the possibility of delayed delivery of packets carrying them. +sending a RST_STREAM; that is, in the terminal states and in the "Reset Sent" +state. + +A receiver could receive any of these three frames in any state, due to the +possibility of delayed delivery of packets carrying them. The receiver of a stream sends MAX_STREAM_DATA ({{frame-max-stream-data}}) and STOP_SENDING frames ({{frame-stop-sending}}). @@ -598,9 +602,11 @@ STOP_SENDING frames ({{frame-stop-sending}}). The receiver only sends MAX_STREAM_DATA in the "Recv" state. A receiver can send STOP_SENDING in any state where it has not received a RST_STREAM frame; that is states other than "Reset Recvd" or "Reset Read". However there is -little value in sending a STOP_SENDING frame after all stream data has been -received in the "Data Recvd" state. A sender could receive these frames in any -state as a result of delayed delivery of packets. +little value in sending a STOP_SENDING frame in the "Data Recvd" state, since +all stream data has been received. + +A sender could receive either of these two frames in any state as a result of +delayed delivery of packets. ## Bidirectional Stream States {#stream-bidi-states} @@ -645,23 +651,22 @@ If an endpoint is no longer interested in the data it is receiving on a stream, it MAY send a STOP_SENDING frame identifying that stream to prompt closure of the stream in the opposite direction. This typically indicates that the receiving application is no longer reading data it receives from the stream, but -is not a guarantee that incoming data will be ignored. +it is not a guarantee that incoming data will be ignored. -STREAM frames received after sending STOP_SENDING are still counted toward the -connection and stream flow-control windows, even though these frames will be -discarded upon receipt. This avoids potential ambiguity about which STREAM -frames count toward flow control. +STREAM frames received after sending STOP_SENDING are still counted toward +connection and stream flow control, even though these frames will be discarded +upon receipt. A STOP_SENDING frame requests that the receiving endpoint send a RST_STREAM frame. An endpoint that receives a STOP_SENDING frame MUST send a RST_STREAM frame for that stream. An endpoint SHOULD copy the error code from the STOP_SENDING frame, but MAY use any other application error code. The endpoint that sends a STOP_SENDING frame can ignore the error code carried in any -RST_STREAM frame they receive. +RST_STREAM frame it receives. If the STOP_SENDING frame is received on a send stream that is already in the "Data Sent" state, a RST_STREAM frame MAY still be sent in order to cancel -retransmission of previously-sent STREAM frames. +retransmissions of previously-sent STREAM frames. STOP_SENDING SHOULD only be sent for a receive stream that has not been reset. STOP_SENDING is most useful for streams in the "Recv" or "Size Known" From 3389bda3ee81fa5c893a63ff022f5d15c15156f3 Mon Sep 17 00:00:00 2001 From: Jana Iyengar Date: Sat, 3 Nov 2018 13:00:50 +0700 Subject: [PATCH 3/7] Fix reference --- draft-ietf-quic-transport.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/draft-ietf-quic-transport.md b/draft-ietf-quic-transport.md index d4acf6ae91..ecb123e745 100644 --- a/draft-ietf-quic-transport.md +++ b/draft-ietf-quic-transport.md @@ -5135,10 +5135,10 @@ higher-numbered stream ID is received. Thus, on a new connection, opening stream 2000001 opens 1 million streams, as required by the specification. The number of active streams is limited by the concurrent stream limit transport -parameter, as explained in {{stream-concurrency}}. If chosen judiciously, this -limit mitigates the effect of the stream commitment attack. However, setting -the limit too low could affect performance when applications expect to open -large number of streams. +parameter, as explained in {{controlling-concurrency}}. If chosen judiciously, +this limit mitigates the effect of the stream commitment attack. However, +setting the limit too low could affect performance when applications expect to +open large number of streams. ## Explicit Congestion Notification Attacks {#security-ecn} From 1894181aa1b7913b6219936433fb483bb0da2da0 Mon Sep 17 00:00:00 2001 From: Jana Iyengar Date: Sat, 3 Nov 2018 13:07:57 +0700 Subject: [PATCH 4/7] rebase --- draft-ietf-quic-transport.md | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/draft-ietf-quic-transport.md b/draft-ietf-quic-transport.md index ecb123e745..2d7d7ebfe7 100644 --- a/draft-ietf-quic-transport.md +++ b/draft-ietf-quic-transport.md @@ -193,9 +193,7 @@ Server: Endpoint: -: An entity that can participate in a QUIC conversation by generating, - receiving, and fully processing QUIC packets. There are only two types of - endpoint in QUIC: client and server. +: The client or server end of a connection. Stream: @@ -218,10 +216,6 @@ QUIC packet: QUIC is a name, not an acronym. -Application: - -: An entity that uses QUIC to send and receive data. - ## Notational Conventions From 7f0dd505ab37f4b44271c15edb52c32abd0d73a7 Mon Sep 17 00:00:00 2001 From: Jana Iyengar Date: Sat, 3 Nov 2018 13:22:25 +0700 Subject: [PATCH 5/7] s/SHOULD/MUST --- draft-ietf-quic-transport.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/draft-ietf-quic-transport.md b/draft-ietf-quic-transport.md index 2d7d7ebfe7..cb8770bef9 100644 --- a/draft-ietf-quic-transport.md +++ b/draft-ietf-quic-transport.md @@ -778,8 +778,6 @@ account for the possibility of loss, a receiver should send a MAX_DATA or MAX_STREAM_DATA frame at least two round trips before it expects the sender to get blocked. - - A receiver MUST NOT wait for a STREAM_DATA_BLOCKED or DATA_BLOCKED frame before sending MAX_STREAM_DATA or MAX_DATA, since doing so will mean that a sender will be blocked for at least an entire round trip, and potentially for longer if the @@ -859,7 +857,7 @@ number of streams available to peers roughly consistent. An endpoint that is unable to open a new stream due to the peer's limits SHOULD send a STREAMS_BLOCKED frame ({{frame-streams-blocked}}). This signal is -considered useful for debugging. An endpoint SHOULD NOT wait to receive this +considered useful for debugging. An endpoint MUST NOT wait to receive this signal before advertising additional credit, since doing so will mean that the peer will be blocked for at least an entire round trip, and potentially for longer if the peer chooses to not send STREAMS_BLOCKED frames. From e58c99220cdfe8fb1931c2c62b29661275ea00c2 Mon Sep 17 00:00:00 2001 From: Jana Iyengar Date: Sat, 3 Nov 2018 16:08:29 +0700 Subject: [PATCH 6/7] mt comments --- draft-ietf-quic-transport.md | 123 ++++++++++++++++------------------- 1 file changed, 56 insertions(+), 67 deletions(-) diff --git a/draft-ietf-quic-transport.md b/draft-ietf-quic-transport.md index cb8770bef9..ef4439324d 100644 --- a/draft-ietf-quic-transport.md +++ b/draft-ietf-quic-transport.md @@ -240,52 +240,49 @@ x (*) ... # Streams {#streams} - - Streams in QUIC provide a lightweight, ordered byte-stream abstraction to an application. An alternative view of QUIC streams is as an elastic "message" abstraction, similar to the way ephemeral streams are used in SST {{?SST=DOI.10.1145/1282427.1282421}}, which may be a more appealing description for some applications. -QUIC does not provide any means of ensuring ordering between octets on different -streams. - Streams can be created by sending data. Other processes associated with stream management - ending, cancelling, and managing flow control - are all designed to impose minimal overheads. For instance, a single STREAM frame ({{frame-stream}}) can open, carry data for, and close a stream. Streams can also be long-lived and can last the entire duration of a connection. +Streams can be created by either endpoint, can concurrently send data +interleaved with other streams, and can be cancelled. Any stream can be +initiated by either endpoint. QUIC does not provide any means of ensuring +ordering between bytes on different streams. + QUIC allows for an arbitrary number of streams to operate concurrently and for an arbitrary amount of data to be sent on any stream, subject to flow control constraints (see {{flow-control}}). -## Stream Types and Identifiers {#stream-id} - +## Stream Types and Identifiers {#stream-id} Streams can be unidirectional or bidirectional. Unidirectional streams carry data in one direction: from the initiator of the stream to its peer. -Bidirectional streams allow for data to be sent in both directions. Streams can -be created by either endpoint, can concurrently send data interleaved with other -streams, and can be cancelled. Any stream can be initiated by either enndpoint. +Bidirectional streams allow for data to be sent in both directions. Streams are identified within a connection by a numeric value, referred to as -the Stream ID. Stream IDs are unique to a stream: a QUIC endpoint MUST NOT -reuse a Stream ID within a connection. Stream IDs are encoded as a +the stream ID. Stream IDs are unique to a stream. A QUIC endpoint MUST NOT +reuse a stream ID within a connection. Stream IDs are encoded as a variable-length integer (see {{integer-encoding}}). -The least significant bit (0x1) of the Stream ID identifies the initiator of the -stream. Client-initiated streams have even-numbered Stream IDs (with the bit -set to 0), and server-initiated streams have odd-numbered Stream IDs (with the +The least significant bit (0x1) of the stream ID identifies the initiator of the +stream. Client-initiated streams have even-numbered stream IDs (with the bit +set to 0), and server-initiated streams have odd-numbered stream IDs (with the bit set to 1). -The second least significant bit (0x2) of the Stream ID distinguishes between +The second least significant bit (0x2) of the stream ID distinguishes between bidirectional streams (with the bit set to 0) and unidirectional streams (with the bit set to 1). -The least significant two bits from a Stream ID therefore identify a stream as +The least significant two bits from a stream ID therefore identify a stream as one of four types, as summarized in {{stream-id-types}}. | Bits | Stream Type | @@ -296,17 +293,17 @@ one of four types, as summarized in {{stream-id-types}}. | 0x3 | Server-Initiated, Unidirectional | {: #stream-id-types title="Stream ID Types"} -Within each type, streams are expected to be created with numerically increasing -Stream IDs. A Stream ID that is used out of order results in all streams of -that type with lower-numbered stream IDs also being opened. +Within each type, streams are created with numerically increasing stream IDs. A +stream ID that is used out of order results in all streams of that type with +lower-numbered stream IDs also being opened. -The first bidirectional stream opened by the client has a Stream ID of 0. +The first bidirectional stream opened by the client has a stream ID of 0. ## Sending and Receiving Data -Endpoints use STREAM frames to encapsulate data sent by an application on a -stream ({{frame-stream}}). An endpoint uses the stream ID and offset fields in a -received STREAM frame to place the contained data in order within the stream. +STREAM frames ({{frame-stream}}) encapsulate data sent by an application. An +endpoint uses the Stream ID and Offset fields in STREAM frames to place data in +order. Endpoints MUST be able to deliver stream data to an application as an ordered byte-stream. Delivering an ordered byte-stream requires that an endpoint buffer @@ -317,7 +314,7 @@ order. However, implementations MAY choose to offer the ability to deliver data out of order to a receiving application. An endpoint could receive data for a stream at the same stream offset multiple -times; data that has already been received can be discarded. The data at a +times. Data that has already been received can be discarded. The data at a given offset MUST NOT change if it is sent multiple times; an endpoint MAY treat receipt of different data at the same offset within a stream as a connection error of type PROTOCOL_VIOLATION. @@ -343,11 +340,10 @@ it relies on receiving priority information from the application that uses QUIC. A QUIC implementation SHOULD provide ways in which an application can indicate the relative priority of streams. When deciding which streams to dedicate resources to, the implementation SHOULD use the information provided by the -application. Failure to account for priority of streams can result in -suboptimal application performance. +application. -# Stream States: Life of a Stream {#stream-states} +# Stream States {#stream-states} This section describes streams in terms of their send or receive components. Two state machines are described: one for the streams on which an endpoint @@ -361,9 +357,7 @@ conditions for opening a stream are slightly more complex for a bidirectional stream because the opening of either send or receive sides causes the stream to open in both directions. -An endpoint can open streams up to its maximum stream limit in any order of -stream ID. However, within each type, endpoints SHOULD open the send side of -streams in numerically increasing order of stream ID. +An endpoint MUST open streams of the same type in increasing order of stream ID. Note: @@ -426,7 +420,7 @@ sending. Sending the first STREAM or STREAM_DATA_BLOCKED frame causes a send stream to enter the "Send" state. An implementation might choose to defer allocating a -Stream ID to a send stream until it sends the first frame and enters this state, +stream ID to a send stream until it sends the first frame and enters this state, which can allow for better stream prioritization. The sending part of a bidirectional stream initiated by a peer (type 0 for a @@ -527,9 +521,7 @@ providing flow control credit. A MAX_STREAM_DATA frame might arrive before a STREAM or STREAM_DATA_BLOCKED frame if packets are lost or reordered. Before creating a stream, all streams of the same type with lower-numbered -stream IDs MUST be created. This means that receipt of a frame that would open -a stream causes all streams of the same type with lower-numbered stream IDs to -be opened in numeric order. This ensures that the creation order for streams is +stream IDs MUST be created. This ensures that the creation order for streams is consistent on both endpoints. In the "Recv" state, the endpoint receives STREAM and STREAM_DATA_BLOCKED @@ -585,10 +577,8 @@ STREAM_DATA_BLOCKED ({{frame-stream-data-blocked}}), and RST_STREAM A sender MUST NOT send any of these frames from a terminal state ("Data Recvd" or "Reset Recvd"). A sender MUST NOT send STREAM or STREAM_DATA_BLOCKED after sending a RST_STREAM; that is, in the terminal states and in the "Reset Sent" -state. - -A receiver could receive any of these three frames in any state, due to the -possibility of delayed delivery of packets carrying them. +state. A receiver could receive any of these three frames in any state, due to +the possibility of delayed delivery of packets carrying them. The receiver of a stream sends MAX_STREAM_DATA ({{frame-max-stream-data}}) and STOP_SENDING frames ({{frame-stop-sending}}). @@ -597,10 +587,8 @@ The receiver only sends MAX_STREAM_DATA in the "Recv" state. A receiver can send STOP_SENDING in any state where it has not received a RST_STREAM frame; that is states other than "Reset Recvd" or "Reset Read". However there is little value in sending a STOP_SENDING frame in the "Data Recvd" state, since -all stream data has been received. - -A sender could receive either of these two frames in any state as a result of -delayed delivery of packets. +all stream data has been received. A sender could receive either of these two +frames in any state as a result of delayed delivery of packets. ## Bidirectional Stream States {#stream-bidi-states} @@ -679,9 +667,9 @@ termination in the opposite direction by sending a STOP_SENDING frame. # Flow Control {#flow-control} -It is necessary to limit the amount of data that a receiver may have to buffer, -so as to prevent a fast sender from overwhelming a slow receiver, or to prevent -a malicious sender from consuming a large amount of memory at a receiver. To +It is necessary to limit the amount of data that a receiver could buffer, to +prevent a fast sender from overwhelming a slow receiver, or to prevent a +malicious sender from consuming a large amount of memory at a receiver. To enable a receiver to limit memory commitment to a connection and to apply back pressure on the sender, streams are flow controlled both individually and as an aggregate. A QUIC receiver controls the maximum amount of data the sender can @@ -720,8 +708,8 @@ MAX_STREAM_DATA ({{frame-max-stream-data}}) or MAX_DATA ({{frame-max-data}}) frames to the sender to advertise additional credit. A receiver advertises credit for a stream by sending a MAX_STREAM_DATA frame -with the Stream ID set appropriately. A MAX_STREAM_DATA frame indicates the -maximum absolute byte offset of a stream. A receiver could use the current +with the Stream ID field set appropriately. A MAX_STREAM_DATA frame indicates +the maximum absolute byte offset of a stream. A receiver could use the current offset of data consumed to determine the flow control offset to be advertised. A receiver MAY send MAX_STREAM_DATA frames in multiple packets in order to make sure that the sender receives an update before running out of flow control @@ -729,10 +717,9 @@ credit, even if one of the packets is lost. A receiver advertises credit for a connection by sending a MAX_DATA frame, which indicates the maximum of the sum of the absolute byte offsets of all streams. A -receiver maintains a cumulative sum of bytes received on all contributing -streams, which is used to check for flow control violations. A receiver might -use a sum of bytes consumed on all streams to determine the maximum data limit -to be advertised. +receiver maintains a cumulative sum of bytes received on all streams, which is +used to check for flow control violations. A receiver might use a sum of bytes +consumed on all streams to determine the maximum data limit to be advertised. A receiver can advertise a larger offset by sending MAX_STREAM_DATA or MAX_DATA frames at any time during the connection. A receiver cannot renege on an @@ -769,12 +756,14 @@ resource commitment and overhead when determining how large a limit is advertised. A receiver MAY use an autotuning mechanism to tune the frequency and amount of -advertised additional credit. +advertised additional credit based on a round-trip time estimate and the rate at +which the receiving application consumes data, similar to common TCP +implementations. If a sender runs out of flow control credit, it will be unable to send new data and is considered blocked. It is generally considered best to not let the -sender go into quiescence. To avoid blocking a sender, and to reasonably -account for the possibility of loss, a receiver should send a MAX_DATA or +sender become blocked. To avoid blocking a sender, and to reasonably account +for the possibility of loss, a receiver should send a MAX_DATA or MAX_STREAM_DATA frame at least two round trips before it expects the sender to get blocked. @@ -786,8 +775,8 @@ peer chooses to not send STREAM_DATA_BLOCKED or DATA_BLOCKED frames. ## Handling Stream Cancellation {#stream-cancellation} -Given enough time, both endpoints must agree on flow control state, to avoid -flow control violations or deadlock. +Endpoints need to eventually agree on the amount of flow control credit that has +been consumed, to avoid either exceeding flow control limits or deadlocking. On receipt of a RST_STREAM frame, an endpoint will tear down state for the matching stream and ignore further data arriving on that stream. If a @@ -4037,7 +4026,7 @@ The fields are: Stream ID: -: A variable-length integer encoding of the Stream ID of the stream being +: A variable-length integer encoding of the stream ID of the stream being terminated. Application Protocol Error Code: @@ -4168,7 +4157,7 @@ The fields in the MAX_STREAM_DATA frame are as follows: Stream ID: -: The Stream ID of the affected stream, encoded as a variable-length integer. +: The stream ID of the affected stream, encoded as a variable-length integer. Maximum Stream Data: @@ -4473,7 +4462,7 @@ The fields are: Stream ID: -: A variable-length integer carrying the Stream ID of the stream being ignored. +: A variable-length integer carrying the stream ID of the stream being ignored. Application Error Code: @@ -4756,7 +4745,7 @@ are present in the frame. * The OFF bit (0x04) in the frame type is set to indicate that there is an Offset field present. When set to 1, the Offset field is present; when set to - 0, the Offset field is absent and the Stream Data starts at an offset of 0 + 0, the Offset field is absent and the stream data starts at an offset of 0 (that is, the frame contains the first bytes of the stream, or the end of a stream that includes no data). @@ -4819,8 +4808,8 @@ the offset of the next byte that would be sent. When new data is to be sent on a stream, a sender MUST set the encapsulating STREAM frame's offset field to the stream offset of the first byte of this new data. The first byte in the stream has an offset of 0. The largest offset -delivered on a stream - the sum of the re-constructed offset and data length - -MUST be less than 2^62. +delivered on a stream - the sum of the offset and data length - MUST be less +than 2^62. ## CRYPTO Frame {#frame-crypto} @@ -4867,10 +4856,10 @@ There is a separate flow of cryptographic handshake data in each encryption level, each of which starts at an offset of 0. This implies that each encryption level is treated as a separate CRYPTO stream of data. -Unlike STREAM frames, which include a Stream ID indicating to which stream the -data belongs, the CRYPTO frame carries data for a single stream per encryption -level. The stream does not have an explicit end, so CRYPTO frames do not have a -FIN bit. +Unlike STREAM frames, which include a Stream ID field indicating to which stream +the data belongs, the CRYPTO frame carries data for a single stream per +encryption level. The stream does not have an explicit end, so CRYPTO frames do +not have a FIN bit. ## Extension Frames From 97294c8a6b441dbd832b6406fd149b4d14f897c4 Mon Sep 17 00:00:00 2001 From: Jana Iyengar Date: Sat, 3 Nov 2018 16:26:01 +0700 Subject: [PATCH 7/7] more trimming --- draft-ietf-quic-transport.md | 30 +++++------------------------- 1 file changed, 5 insertions(+), 25 deletions(-) diff --git a/draft-ietf-quic-transport.md b/draft-ietf-quic-transport.md index ef4439324d..ae9390ac1b 100644 --- a/draft-ietf-quic-transport.md +++ b/draft-ietf-quic-transport.md @@ -242,9 +242,7 @@ x (*) ... Streams in QUIC provide a lightweight, ordered byte-stream abstraction to an application. An alternative view of QUIC streams is as an elastic "message" -abstraction, similar to the way ephemeral streams are used in SST -{{?SST=DOI.10.1145/1282427.1282421}}, which may be a more appealing description -for some applications. +abstraction. Streams can be created by sending data. Other processes associated with stream management - ending, cancelling, and managing flow control - are all designed to @@ -2708,28 +2706,6 @@ streams as necessary in outgoing packets without losing transmission efficiency to underfilled packets. -## Frame Priority {#frame-priority} - -Often, there will be limits on what can be transmitted as a result of connection -flow control or the current congestion controller state. - -Giving preference to the transmission of its own management functions ensures -that a protocol functions efficiently. That is, prioritizing frames other than -STREAM frames ensures that loss recovery, congestion control, and flow control -operate effectively. - -CRYPTO frames SHOULD be prioritized over STREAM frames prior to the completion -of the cryptographic handshake. This includes the retransmission of the second -flight of client handshake messages, that is, the TLS Finished and any client -authentication messages. - -STREAM data in frames determined to be lost SHOULD be retransmitted before -sending new data, unless priorities specified by the application indicate -otherwise (see {{stream-prioritization}}). Retransmitting lost stream data can -fill in gaps, which allows the peer to consume already received data and free up -the flow control window. - - ## Packet Processing and Acknowledgment {#processing-and-ack} A packet MUST NOT be acknowledged until packet protection has been successfully @@ -2893,6 +2869,10 @@ containing that information is acknowledged. * PADDING frames contain no information, so lost PADDING frames do not require repair. +Endpoints SHOULD prioritize retransmission of data over sending new data, unless +priorities specified by the application indicate otherwise (see +{{stream-prioritization}}). + Upon detecting losses, a sender MUST take appropriate congestion control action. The details of loss detection and congestion control are described in {{QUIC-RECOVERY}}.