From d2c9ae53b6fe5ef46432a1fca1f1b8c5c8f64f8d Mon Sep 17 00:00:00 2001 From: ianswett Date: Tue, 16 Oct 2018 21:07:16 -0400 Subject: [PATCH 01/10] Amplification limit in recovery Follow up for Issue #1764 --- draft-ietf-quic-recovery.md | 52 ++++++++++--------------------------- 1 file changed, 14 insertions(+), 38 deletions(-) diff --git a/draft-ietf-quic-recovery.md b/draft-ietf-quic-recovery.md index d506fea1b9..1a595a8808 100644 --- a/draft-ietf-quic-recovery.md +++ b/draft-ietf-quic-recovery.md @@ -320,10 +320,17 @@ SHOULD be set for twice the newly computed smoothed RTT. When crypto packets are sent, the sender SHOULD set a timer for the crypto timeout period. Upon timeout, the sender MUST retransmit all unacknowledged -CRYPTO data by calling RetransmitAllUnackedCryptoData(). On each consecutive -expiration of the crypto timer without receiving an acknowledgement for a new -packet, the sender SHOULD double the crypto handshake timeout and set a timer -for this period. +CRYPTO data if possible. + +If the server has not confirmed the path, it must not send more than 3 +times the number of received bytes, as specified in the {{QUIC-TRANSPORT}}. +If not all unacknowledged CRYPTO data can be sent, then all CRYPTO data in +Initial encryption should be retransmitted. If no bytes may be sent, +then no alarm should be armed until bytes have been received by the peer. + +On each consecutive expiration of the crypto timer without receiving an +acknowledgement for a new packet, the sender SHOULD double the crypto +handshake timeout and set a timer for this period. When crypto packets are outstanding, the TLP and RTO timers are not active. @@ -763,37 +770,6 @@ duration of the timer is based on the timer's mode, which is set in the packet and timer events further below. The function SetLossDetectionTimer defined below shows how the single timer is set. -#### Crypto Handshake Timer - -When a connection has unacknowledged crypto packets, the crypto handshake timer -is set and when it expires, all unacknowledged CRYPTO data is retransmitted. - -When stateless rejects are in use, the connection is considered immediately -closed once a reject is sent, so no timer is set to retransmit the reject. - -Version negotiation packets are always stateless, and MUST be sent once per -handshake packet that uses an unsupported QUIC version, and MAY be sent in -response to 0-RTT packets. - -#### Tail Loss Probe and Retransmission Timer - -Tail loss probes {{?TLP}} and retransmission timeouts {{?RFC6298}} -are timer based mechanisms to recover from cases when there are -outstanding retransmittable packets, but an acknowledgement has -not been received in a timely manner. - -The TLP and RTO timers are armed when there are no unacknowledged crypto -packets. The TLP timer is set until the max number of TLP packets have been -sent, and then the RTO timer is set. - -#### Early Retransmit Timer - -Early retransmit {{?RFC5827}} is implemented with a 1/4 RTT timer. It is -part of QUIC's time based loss detection, but is always enabled, even when -only packet reordering loss detection is enabled. - -#### Pseudocode - Pseudocode for SetLossDetectionTimer follows: ~~~ @@ -838,8 +814,8 @@ Pseudocode for SetLossDetectionTimer follows: ### On Timeout -QUIC uses one loss recovery timer, which when set, can be in one of several -modes. When the timer expires, the mode determines the action to be performed. +When the loss detection timer expires, the timer's mode determines the action +to be performed. Pseudocode for OnLossDetectionTimeout follows: @@ -847,7 +823,7 @@ Pseudocode for OnLossDetectionTimeout follows: OnLossDetectionTimeout(): if (crypto packets are outstanding): // Crypto handshake timeout. - RetransmitAllUnackedCryptoData() + RetransmitUnackedCryptoData() crypto_count++ else if (loss_time != 0): // Early retransmit or Time Loss Detection From 0cc808b873a118f6708d6547d186f3b2fbdfda78 Mon Sep 17 00:00:00 2001 From: ianswett Date: Tue, 16 Oct 2018 21:09:15 -0400 Subject: [PATCH 02/10] Update draft-ietf-quic-recovery.md --- draft-ietf-quic-recovery.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-ietf-quic-recovery.md b/draft-ietf-quic-recovery.md index 1a595a8808..b3bb565f1d 100644 --- a/draft-ietf-quic-recovery.md +++ b/draft-ietf-quic-recovery.md @@ -322,7 +322,7 @@ When crypto packets are sent, the sender SHOULD set a timer for the crypto timeout period. Upon timeout, the sender MUST retransmit all unacknowledged CRYPTO data if possible. -If the server has not confirmed the path, it must not send more than 3 +Until the server has confirmed the path, it must not send more than 3 times the number of received bytes, as specified in the {{QUIC-TRANSPORT}}. If not all unacknowledged CRYPTO data can be sent, then all CRYPTO data in Initial encryption should be retransmitted. If no bytes may be sent, From 1730b2db4aafa13f0abf5df1ad08ad9abe9bf15c Mon Sep 17 00:00:00 2001 From: Martin Thomson Date: Wed, 17 Oct 2018 09:17:15 -0400 Subject: [PATCH 03/10] Update draft-ietf-quic-recovery.md --- draft-ietf-quic-recovery.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-ietf-quic-recovery.md b/draft-ietf-quic-recovery.md index b3bb565f1d..3666e97cf5 100644 --- a/draft-ietf-quic-recovery.md +++ b/draft-ietf-quic-recovery.md @@ -322,7 +322,7 @@ When crypto packets are sent, the sender SHOULD set a timer for the crypto timeout period. Upon timeout, the sender MUST retransmit all unacknowledged CRYPTO data if possible. -Until the server has confirmed the path, it must not send more than 3 +Until the server has validated the client's address on the path, it must not send more than 3 times the number of received bytes, as specified in the {{QUIC-TRANSPORT}}. If not all unacknowledged CRYPTO data can be sent, then all CRYPTO data in Initial encryption should be retransmitted. If no bytes may be sent, From e7c83d267088cda790341f728205601e6f8a781b Mon Sep 17 00:00:00 2001 From: ianswett Date: Wed, 17 Oct 2018 09:22:54 -0400 Subject: [PATCH 04/10] Crypto retransmission timer --- draft-ietf-quic-recovery.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/draft-ietf-quic-recovery.md b/draft-ietf-quic-recovery.md index 3666e97cf5..422e3ba3e7 100644 --- a/draft-ietf-quic-recovery.md +++ b/draft-ietf-quic-recovery.md @@ -300,10 +300,10 @@ timer for TCP as well, and this document incorporates this advancement. Timer-based loss detection recovers from losses that cannot be handled by ack-based loss detection. It uses a single timer which switches between -a crypto handshake retransmission timer, a Tail Loss Probe timer and -Retransmission Timeout mechanisms. +a crypto retransmission timer, a Tail Loss Probe timer and Retransmission +Timeout mechanisms. -### Crypto Handshake Timeout +### Crypto Retransmission Timeout Data in CRYPTO frames is critical to QUIC transport and crypto negotiation, so a more aggressive timeout is used to retransmit it. @@ -322,15 +322,15 @@ When crypto packets are sent, the sender SHOULD set a timer for the crypto timeout period. Upon timeout, the sender MUST retransmit all unacknowledged CRYPTO data if possible. -Until the server has validated the client's address on the path, it must not send more than 3 -times the number of received bytes, as specified in the {{QUIC-TRANSPORT}}. +Until the server has validated the client's address on the path, the number of +bytes it can send is limited, as specified in the {{QUIC-TRANSPORT}}. If not all unacknowledged CRYPTO data can be sent, then all CRYPTO data in Initial encryption should be retransmitted. If no bytes may be sent, then no alarm should be armed until bytes have been received by the peer. On each consecutive expiration of the crypto timer without receiving an acknowledgement for a new packet, the sender SHOULD double the crypto -handshake timeout and set a timer for this period. +retransmission timeout and set a timer for this period. When crypto packets are outstanding, the TLP and RTO timers are not active. @@ -467,7 +467,7 @@ delayed acknowledgement should be generated after processing incoming packets. ### Crypto Handshake Data In order to quickly complete the handshake and avoid spurious retransmissions -due to crypto handshake timeouts, crypto packets SHOULD use a very short ack +due to crypto retransmission timeouts, crypto packets SHOULD use a very short ack delay, such as 1ms. ACK frames MAY be sent immediately when the crypto stack indicates all data for that encryption level has been received. @@ -546,8 +546,8 @@ are described in this section. loss_detection_timer: : Multi-modal timer used for loss detection. -handshake_count: -: The number of times all unacknowledged handshake data has been +crypto_count: +: The number of times all unacknowledged CRYPTO data has been retransmitted without receiving an ack. tlp_count: @@ -623,7 +623,7 @@ follows: ~~~ loss_detection_timer.reset() - handshake_count = 0 + crypto_count = 0 tlp_count = 0 rto_count = 0 if (kUsingTimeLossDetection) @@ -713,7 +713,7 @@ Pseudocode for OnAckReceived and UpdateRtt follow: if (rto_count > 0 && smallest_newly_acked > largest_sent_before_rto): OnRetransmissionTimeoutVerified(smallest_newly_acked) - handshake_count = 0 + crypto_count = 0 tlp_count = 0 rto_count = 0 @@ -822,7 +822,7 @@ Pseudocode for OnLossDetectionTimeout follows: ~~~ OnLossDetectionTimeout(): if (crypto packets are outstanding): - // Crypto handshake timeout. + // Crypto retransmission timeout. RetransmitUnackedCryptoData() crypto_count++ else if (loss_time != 0): From 360869ecd82c71cab00adad53330efbc51b4aed7 Mon Sep 17 00:00:00 2001 From: ianswett Date: Wed, 17 Oct 2018 09:24:53 -0400 Subject: [PATCH 05/10] Update draft-ietf-quic-recovery.md --- draft-ietf-quic-recovery.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/draft-ietf-quic-recovery.md b/draft-ietf-quic-recovery.md index 422e3ba3e7..a4f766b924 100644 --- a/draft-ietf-quic-recovery.md +++ b/draft-ietf-quic-recovery.md @@ -467,9 +467,9 @@ delayed acknowledgement should be generated after processing incoming packets. ### Crypto Handshake Data In order to quickly complete the handshake and avoid spurious retransmissions -due to crypto retransmission timeouts, crypto packets SHOULD use a very short ack -delay, such as 1ms. ACK frames MAY be sent immediately when the crypto stack -indicates all data for that encryption level has been received. +due to crypto retransmission timeouts, crypto packets SHOULD use a very short +ack delay, such as 1ms. ACK frames MAY be sent immediately when the crypto +stack indicates all data for that encryption level has been received. ### ACK Ranges From 97f8bf265bb724cb06468cafd018bffed07fdf96 Mon Sep 17 00:00:00 2001 From: ianswett Date: Wed, 17 Oct 2018 15:09:53 -0400 Subject: [PATCH 06/10] Update draft-ietf-quic-recovery.md --- draft-ietf-quic-recovery.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/draft-ietf-quic-recovery.md b/draft-ietf-quic-recovery.md index a4f766b924..cf39d34259 100644 --- a/draft-ietf-quic-recovery.md +++ b/draft-ietf-quic-recovery.md @@ -323,9 +323,9 @@ timeout period. Upon timeout, the sender MUST retransmit all unacknowledged CRYPTO data if possible. Until the server has validated the client's address on the path, the number of -bytes it can send is limited, as specified in the {{QUIC-TRANSPORT}}. -If not all unacknowledged CRYPTO data can be sent, then all CRYPTO data in -Initial encryption should be retransmitted. If no bytes may be sent, +bytes it can send is limited, as specified in {{QUIC-TRANSPORT}}. +If not all unacknowledged CRYPTO data can be sent, then all CRYPTO data sent +in Initial packets should be retransmitted. If no bytes may be sent, then no alarm should be armed until bytes have been received by the peer. On each consecutive expiration of the crypto timer without receiving an From b523410378cac81df39f579543275b53f85100e2 Mon Sep 17 00:00:00 2001 From: ianswett Date: Thu, 18 Oct 2018 16:52:07 -0400 Subject: [PATCH 07/10] Client always sends --- draft-ietf-quic-recovery.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/draft-ietf-quic-recovery.md b/draft-ietf-quic-recovery.md index cf39d34259..894054567e 100644 --- a/draft-ietf-quic-recovery.md +++ b/draft-ietf-quic-recovery.md @@ -318,15 +318,23 @@ is available, or if the network changes, the initial RTT SHOULD be set to 100ms. When an acknowledgement is received, a new RTT is computed and the timer SHOULD be set for twice the newly computed smoothed RTT. -When crypto packets are sent, the sender SHOULD set a timer for the crypto +When crypto packets are sent, the sender MUST set a timer for the crypto timeout period. Upon timeout, the sender MUST retransmit all unacknowledged CRYPTO data if possible. Until the server has validated the client's address on the path, the number of bytes it can send is limited, as specified in {{QUIC-TRANSPORT}}. -If not all unacknowledged CRYPTO data can be sent, then all CRYPTO data sent -in Initial packets should be retransmitted. If no bytes may be sent, -then no alarm should be armed until bytes have been received by the peer. +If not all unacknowledged CRYPTO data can be sent, then all unacknowledged +CRYPTO data sent in Initial packets should be retransmitted. If no bytes +may be sent, then no alarm should be armed until bytes have been received from +the client. + +Because the server may be blocked until more packets are received, the +client MUST arm the crypto retransmission alarm even if there is no +unacknowledged CRYPTO data. If the timer expires and the client has no +CRYPTO data to retransmit and does not have Handshake keys, it SHOULD send +an Initial packet in a UDP datagram of at least 1200 octets. +If the client has Handshake keys, it SHOULD send a Handshake packet. On each consecutive expiration of the crypto timer without receiving an acknowledgement for a new packet, the sender SHOULD double the crypto From 289a942a541a38bbd419d6120732533498531eff Mon Sep 17 00:00:00 2001 From: Martin Thomson Date: Thu, 18 Oct 2018 21:38:41 -0400 Subject: [PATCH 08/10] Update draft-ietf-quic-recovery.md Co-Authored-By: ianswett --- draft-ietf-quic-recovery.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-ietf-quic-recovery.md b/draft-ietf-quic-recovery.md index 894054567e..100be305a2 100644 --- a/draft-ietf-quic-recovery.md +++ b/draft-ietf-quic-recovery.md @@ -330,7 +330,7 @@ may be sent, then no alarm should be armed until bytes have been received from the client. Because the server may be blocked until more packets are received, the -client MUST arm the crypto retransmission alarm even if there is no +client MUST start the crypto retransmission timer even if there is no unacknowledged CRYPTO data. If the timer expires and the client has no CRYPTO data to retransmit and does not have Handshake keys, it SHOULD send an Initial packet in a UDP datagram of at least 1200 octets. From d2f45ff289284845607edc032c4f6bf50f47621f Mon Sep 17 00:00:00 2001 From: Martin Thomson Date: Fri, 19 Oct 2018 05:24:34 -0400 Subject: [PATCH 09/10] Update draft-ietf-quic-recovery.md Co-Authored-By: ianswett --- draft-ietf-quic-recovery.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-ietf-quic-recovery.md b/draft-ietf-quic-recovery.md index 100be305a2..b45c37260f 100644 --- a/draft-ietf-quic-recovery.md +++ b/draft-ietf-quic-recovery.md @@ -326,7 +326,7 @@ Until the server has validated the client's address on the path, the number of bytes it can send is limited, as specified in {{QUIC-TRANSPORT}}. If not all unacknowledged CRYPTO data can be sent, then all unacknowledged CRYPTO data sent in Initial packets should be retransmitted. If no bytes -may be sent, then no alarm should be armed until bytes have been received from +can be sent, then no alarm should be armed until bytes have been received from the client. Because the server may be blocked until more packets are received, the From dd7fc95cadd189a1430da38f7054034036d689af Mon Sep 17 00:00:00 2001 From: Martin Thomson Date: Fri, 19 Oct 2018 05:25:18 -0400 Subject: [PATCH 10/10] Update draft-ietf-quic-recovery.md Co-Authored-By: ianswett --- draft-ietf-quic-recovery.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-ietf-quic-recovery.md b/draft-ietf-quic-recovery.md index b45c37260f..0d4b79cd07 100644 --- a/draft-ietf-quic-recovery.md +++ b/draft-ietf-quic-recovery.md @@ -329,7 +329,7 @@ CRYPTO data sent in Initial packets should be retransmitted. If no bytes can be sent, then no alarm should be armed until bytes have been received from the client. -Because the server may be blocked until more packets are received, the +Because the server could be blocked until more packets are received, the client MUST start the crypto retransmission timer even if there is no unacknowledged CRYPTO data. If the timer expires and the client has no CRYPTO data to retransmit and does not have Handshake keys, it SHOULD send