Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Editorial cleanup of Recovery #1957

Merged
merged 17 commits into from
Nov 4, 2018
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 86 additions & 72 deletions draft-ietf-quic-recovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,18 +163,19 @@ across packet number spaces.

### Monotonically Increasing Packet Numbers

TCP conflates transmission sequence number at the sender with delivery sequence
number at the receiver, which results in retransmissions of the same data
carrying the same sequence number, and consequently to problems caused by
"retransmission ambiguity". QUIC separates the two: QUIC uses a packet number
for transmissions, and any application data is sent in one or more streams,
with delivery order determined by stream offsets encoded within STREAM frames.

QUIC's packet number is strictly increasing, and directly encodes transmission
order. A higher QUIC packet number signifies that the packet was sent later,
and a lower QUIC packet number signifies that the packet was sent earlier. When
a packet containing frames is deemed lost, QUIC rebundles necessary frames in a
new packet with a new packet number, removing ambiguity about which packet is
TCP conflates transmission order at the sender with delivery order at the
receiver, which results in retransmissions of the same data carrying the same
sequence number, and consequently leads to "retransmission ambiguity". QUIC
separates the two: QUIC uses a packet number to indicate transmission order,
and any application data is sent in one or more streams, with delivery order
determined by stream offsets encoded within STREAM frames.

QUIC's packet number is strictly increasing within a packet number space,
and directly encodes transmission order. A higher packet number signifies
that the packet was sent later, and a lower packet number signifies that
the packet was sent earlier. When a packet containing retransmittable
frames is detected lost, QUIC rebundles necessary frames in a new packet
with a new packet number, removing ambiguity about which packet is
acknowledged when an ACK is received. Consequently, more accurate RTT
measurements can be made, spurious retransmissions are trivially detected, and
mechanisms such as Fast Retransmit can be applied universally, based only on
Expand Down Expand Up @@ -236,7 +237,9 @@ underestimation of min RTT, which in turn prevents underestimating smoothed RTT.
Ack-based loss detection implements the spirit of TCP's Fast Retransmit
{{?RFC5681}}, Early Retransmit {{?RFC5827}}, FACK, and SACK loss recovery
{{?RFC6675}}. This section provides an overview of how these algorithms are
implemented in QUIC.
implemented in QUIC. Though both time-based loss detection and early retransmit
use a timer, they are part of ack-based detection because they do not use a
timer to send probes, but rather to declare packets lost.

### Fast Retransmit

Expand All @@ -247,27 +250,45 @@ the acknowledgement indicates that a later packet was received, while the
reordering threshold provides some tolerance for reordering of packets in the
network.

Spuriously declaring packets lost leads to unnecessary retransmissions and
may result in degraded performance due to the actions of the congestion
controller upon detecting loss. Implementations that detect spurious
retransmissions and increase the reordering threshold in packets or time
MAY choose to start with smaller initial reordering thresholds to minimize
recovery latency.

#### Packet Threshold

The RECOMMENDED initial value for kReorderingThreshold is 3, based on
TCP loss recovery {{?RFC5681}} {{?RFC6675}}. Some networks may exhibit higher
degrees of reordering, causing a sender to detect spurious losses. Spuriously
declaring packets lost leads to unnecessary retransmissions and may result in
degraded performance due to the actions of the congestion controller upon
detecting loss. Implementers MAY use algorithms developed for TCP, such as
degrees of reordering, causing a sender to detect spurious losses.
Implementers MAY use algorithms developed for TCP, such as
TCP-NCR {{?RFC4653}}, to improve QUIC's reordering resilience.

QUIC implementations can use time-based loss detection to handle reordering
based on time elapsed since the packet was sent. This may be used either as
a replacement for a packet reordering threshold or in addition to it.
The RECOMMENDED time threshold, expressed as a fraction of the
round-trip time (kTimeReorderingFraction), is 1/8.
#### Time-based

Time-based loss detection uses a time threshold to determine how much
reordering to tolerate. In this document, the threshold is expressed as a
fraction of an RTT, but implemenantations MAY experiment with absolute
thresholds. This may be used either as a replacement for a packet reordering
threshold or in addition to it.

When a larger packet is acknowledged, if it was sent more than the threshold
after any in flight packets, those packets are immediately declared lost.
Otherwise, a timer is set for the the reordering threshold minus the time
difference between the earliest in flight packet and the largest newly
acknowledged packet. Note that in some cases the timer could become longer when
packets are acknowleged out of order. The RECOMMENDED time threshold, expressed
as a fraction of the round-trip time (kTimeReorderingFraction), is 1/8.

### Early Retransmit

Unacknowledged packets close to the tail may have fewer than
kReorderingThreshold retransmittable packets sent after them. Loss of such
packets cannot be detected via Fast Retransmit. To enable ack-based loss
detection of such packets, receipt of an acknowledgment for the last outstanding
retransmittable packet triggers the Early Retransmit process, as follows.
packets cannot be detected via Packet Threshold Fast Retransmit. To enable
ack-based loss detection of such packets, receipt of an acknowledgment for
the last outstanding retransmittable packet triggers the Early Retransmit
process, as follows.

If there are unacknowledged in-flight packets still pending, they should
be marked as lost. To compensate for the reduced reordering resilience, the
Expand Down Expand Up @@ -298,9 +319,9 @@ prone to spurious retransmissions due to its reduced reordering resilence
without the timer. This observation led Linux TCP implementers to implement a
timer for TCP as well, and this document incorporates this advancement.

## Timer-based Detection
## Timeout Loss Detection

Timer-based loss detection recovers from losses that cannot be handled by
Timeout loss detection recovers from losses that cannot be handled by
ack-based loss detection. It uses a single timer which switches between
a crypto retransmission timer, a Tail Loss Probe timer and Retransmission
Timeout mechanisms.
Expand Down Expand Up @@ -426,22 +447,22 @@ QUIC's RTO algorithm differs from TCP in that the firing of an RTO timer is not
considered a strong enough signal of packet loss, so does not result in an
immediate change to congestion window or recovery state. An RTO timer expires
only when there's a prolonged period of network silence, which could be caused
by a change in the underlying network RTT.
by a change in the network RTT.

QUIC also diverges from TCP by including MaxAckDelay in the RTO period. Since
QUIC corrects for this delay in its SRTT and RTTVAR computations, it is
necessary to add this delay explicitly in the TLP and RTO computation.

When an acknowledgment is received for a packet sent on an RTO event, any
unacknowledged packets with lower packet numbers than those acknowledged MUST be
marked as lost. If an acknowledgement for a packet sent on an RTO is received
at the same time packets sent prior to the first RTO are acknowledged, the RTO
is considered spurious and standard loss detection rules apply.
When an ACK is received that acknowledges only one or more packets sent on
an RTO event, all unacknowledged packets with lower packet numbers MUST be
marked as lost. If packets sent prior to the first RTO are acknowledged in
the same ACK, the RTO is considered spurious and standard loss detection rules
apply.

A packet sent when an RTO timer expires MAY carry new data if available or
unacknowledged data to potentially reduce recovery time. Since this packet is
sent as a probe into the network prior to establishing any packet loss, prior
unacknowledged packets SHOULD NOT be marked as lost.
unacknowledged packets SHOULD NOT be marked as lost when the timer expires.

A packet sent on an RTO timer MUST NOT be blocked by the sender's congestion
controller. A sender MUST however count these packets as being in flight, since
Expand All @@ -450,11 +471,10 @@ this packet adds network load without establishing packet loss.
## Generating Acknowledgements

QUIC SHOULD delay sending acknowledgements in response to packets, but MUST NOT
excessively delay acknowledgements of packets containing frames other than ACK.
Specifically, implementations MUST attempt to enforce a maximum
ack delay to avoid causing the peer spurious timeouts. The maximum ack delay
is communicated in the `max_ack_delay` transport parameter and the default
value is 25ms.
excessively delay acknowledgements of retransmittable packets. Specifically,
implementations MUST attempt to enforce a maximum ack delay to avoid causing
the peer spurious timeouts. The maximum ack delay is communicated in the
`max_ack_delay` transport parameter and the default value is 25ms.

ianswett marked this conversation as resolved.
Show resolved Hide resolved
An acknowledgement SHOULD be sent immediately upon receipt of a second
packet but the delay SHOULD NOT exceed the maximum ack delay. QUIC recovery
Expand Down Expand Up @@ -501,13 +521,13 @@ frame may be saved. When a packet containing an ACK frame is acknowledged, the
receiver can stop acknowledging packets less than or equal to the largest
acknowledged in the sent ACK frame.

In cases without ACK frame loss, this algorithm allows for a minimum of 1 RTT of
reordering. In cases with ACK frame loss, this approach does not guarantee that
every acknowledgement is seen by the sender before it is no longer included in
the ACK frame. Packets could be received out of order and all subsequent ACK
frames containing them could be lost. In this case, the loss recovery algorithm
may cause spurious retransmits, but the sender will continue making forward
progress.
In cases without ACK frame loss, this algorithm allows for a minimum of 1 RTT
of reordering. In cases with ACK frame loss and reordering, this approach does
not guarantee that every acknowledgement is seen by the sender before it is no
longer included in the ACK frame. Packets could be received out of order and
all subsequent ACK frames containing them could be lost. In this case, the
loss recovery algorithm may cause spurious retransmits, but the sender will
continue making forward progress.
ianswett marked this conversation as resolved.
Show resolved Hide resolved

## Tracking Sent Packets {#tracking-sent-packets}

Expand Down Expand Up @@ -777,16 +797,13 @@ Pseudocode for OnAckReceived and UpdateRtt follow:

### On Packet Acknowledgment

When a packet is acked for the first time, the following OnPacketAcked function
is called. Note that a single ACK frame may newly acknowledge several packets.
OnPacketAcked must be called once for each of these newly acked packets.

OnPacketAcked takes one parameter, acked_packet, which is the struct of the
newly acked packet.
When a packet is acknowledged for the first time, the following OnPacketAcked
function is called. Note that a single ACK frame may newly acknowledge several
packets. OnPacketAcked must be called once for each of these newly acknowledged
packets.

If this is the first acknowledgement following RTO, check if the smallest newly
acknowledged packet is one sent by the RTO, and if so, inform congestion control
of a verified RTO, similar to F-RTO {{?RFC5682}}.
OnPacketAcked takes one parameter, acked_packet, which is the struct detailed in
{{sent-packets-fields}}.

Pseudocode for OnPacketAcked follows:

Expand Down Expand Up @@ -878,15 +895,13 @@ Pseudocode for OnLossDetectionTimeout follows:

### Detecting Lost Packets

Packets in QUIC are only considered lost once a larger packet number in
the same packet number space is acknowledged. DetectLostPackets is called
every time an ack is received and operates on the sent_packets for that
packet number space. If the loss detection timer expires and the loss_time
is set, the previous largest acked packet is supplied.

#### Pseudocode
DetectLostPackets is called every time an ACK is received and operates on
the sent_packets for that packet number space. If the loss detection timer
expires and the loss_time is set, the previous largest acknowledged packet
is supplied.

DetectLostPackets takes one parameter, acked, which is the largest acked packet.
DetectLostPackets takes one parameter, largest_acked, which is the largest
acked packet.

Pseudocode for DetectLostPackets follows:

Expand Down Expand Up @@ -977,11 +992,10 @@ congestion window.
## Recovery Period

Recovery is a period of time beginning with detection of a lost packet or an
increase in the ECN-CE counter. Because QUIC retransmits stream data and control
frames, not packets, it defines the end of recovery as a packet sent after the
start of recovery being acknowledged. This is slightly different from TCP's
definition of recovery, which ends when the lost packet that started recovery is
acknowledged.
increase in the ECN-CE counter. Because QUIC does not retransmit packets,
it defines the end of recovery as a packet sent after the start of recovery
being acknowledged. This is slightly different from TCP's definition of
recovery, which ends when the lost packet that started recovery is acknowledged.

The recovery period limits congestion window reduction to once per round trip.
During recovery, the congestion window remains unchanged irrespective of new
Expand Down Expand Up @@ -1098,8 +1112,8 @@ variables as follows:

### On Packet Sent

Whenever a packet is sent, and it contains non-ACK frames,
the packet increases bytes_in_flight.
Whenever a packet is sent, and it contains non-ACK frames, the packet
increases bytes_in_flight.

~~~
OnPacketSentCC(bytes_sent):
Expand All @@ -1108,7 +1122,7 @@ the packet increases bytes_in_flight.

### On Packet Acknowledgement

Invoked from loss detection's OnPacketAcked and is supplied with
Invoked from loss detection's OnPacketAcked and is supplied with the
acked_packet from sent_packets.

~~~
Expand All @@ -1133,7 +1147,7 @@ acked_packet from sent_packets.
### On New Congestion Event

Invoked from ProcessECN and OnPacketsLost when a new congestion event is
detected. May starts a new recovery period and reduces the congestion
detected. May start a new recovery period and reduces the congestion
window.

~~~
Expand Down