ECN echo encoding, detailed packet level or ECN marked bytes

[IngJohEricsson]

Once the issue #804 whether ECN should be in the ACK frame or in a separate frame it is time to discuss the format. There are two possible ways to do it

1. The status of the ECN bits for each received packet is indicated. Fig 2 and 3 in https://tools.ietf.org/id/draft-johansson-quic-ecn-01.txt suggests two formats, my preference is the version presented in figure 3 (alt 2), this way the ECN bits are placed at the end of the ACK frame.

2. Accumulated number of bytes marked '01','10' and '11' (possibly also '00'). An example is presented in figure 2 in https://tools.ietf.org/id/draft-johansson-quic-ecn-03.txt. If the ECN echo is in the ACK frame then it is not necessary with a counter for '00' marked bytes as this value can be inferred from the already existing ACKs and the counters for the '01','10' and '11' marked bytes.

Proponents for the detailed packet level indication point at a possibility that future congestion control algorithms can benefit from this level of detail. On the other hand, scalable congestion control algorithms like DCTCP can work well with the accumulated bytes count.

[ianswett]

I would favor option 2, since I believe it's simpler to implement and easy to use in congestion control.

Also, assuming ECN is in the ACK frame, you will know the ECN marking was on a relatively small set of newly acknowledged packets.

[britram]

+1 to option 2; it gives sufficient information, and seems easier to implement.

[janaiyengar]

I don't think I understand option 2. From the doc, it seems to be a count of the different markings for the packets that are being acked in the ACK frame, is that correct?

If so, that seems fine. We don't really care about exactly which packets were marked, right? Of course, one thing to bear in mind is that a receiver can arbitrarily keep sending acks for old packets, and therefore could echo ECE for an arbitrarily long time. I'm not sure if this leads to any issues yet, but it's something that one may need to be careful about when interpreting the ECE marks on a received ack.

[janaiyengar]

Actually, here's a case: #ECE remains the same across two acks, but the ack stops acking an old packet and acks a new packet. This could mean two things: the old packet and the new packet both had no ECE, or they both had ECE. In the first case there's nothing to be done, but in the second case, the receiver of the ack may want to reduce the window, since the new packet could be part of a new window of data.

I think this problem applies to both alternatives.

[ianswett]

I haven't looked closely at the doc, but I assumed that the total count of packets/bytes marked is monotonically increasing and eventually wraps. So there's no issue if a marked packet stops being acknowledged, because the total count of marks received never decreases.

[IngJohEricsson]

Yes, for option 2 the idea is that the receiver accumulates the bytes (QUIC header+payload) that are marked ECT(0), ECT(1) and CE. This means three 48-bit counters (or 64 bit if that is easier), in other words monotonically increasing counters, with possible wrap around. The LSB of these counters are put in the ECN ACK. The details are subject to discussion...
What is neccessary is a mechanism that discards duplicate packets to avoid that duplicates increase the counters more than once.
The fact that the counters can wrap around is not a problem as the congestion control should only care about the delta increase of the CE marked bytes.

[ianswett]

Duplicate packet numbers should be discarded at a very early stage in QUIC packet processing, long before any of the congestion control code is executed, so I believe duplicate packets should not handled for you.

[janaiyengar]

Ah, yes, if it's monotonically increasing, then that seems fine. I agree that duplicates should not be an issue.

[mirjak]

Alternatively, one could also add ECN information per ACK block, basically saying that all packets in this block were marked with this ECN codepoint. This however only makes sense under the assumptions that most packets are marked with the same codepoint (usually ECT(0) today) and only a few packets are e.g. CE marked. In this case you probably have the same number of ACK blocks as with loss today. But I'm not sure if this assumption still holds in the future with things like L4S or if used in data centers with some congestion control similar to DCTCP.

[IngJohEricsson]

Yes, this option was also listed in fig 2 in https://tools.ietf.org/id/draft-johansson-quic-ecn-01.txt . I have done some minimal inspection of L4S ECN marking patterns with DCTCP as congestion control. If the bootleneck is configured to ECN mark packets that are queued up more than a fixed value (say 2ms) you tend to get a periodic marking pattern, which could indicate that the alternative Mirja suggests can actually work, at least with one flow. Things get more complicated with multiple flows and also with the dualQ implemenetations that are suggested by Koen De Schepper and Bob Briscoe, in which case one see more random ECN marking, also my laymans impelmentation of an ECN marker for TCP HULL exhibit a random marking pattern.

[janaiyengar]

Discussion to continue in ECN design team (https://github.com/quicwg/base-drafts/wiki/ECN-in-QUIC).

[martinthomson]

Closed by #1372.