Variable-length fields

[larseggert]

Packet numbers, stream offsets, various ACK information and many other fields can currently be encoded as variable length. Do we have some data on the benefits of this complexity? Would it be sufficient to use fewer increments, or could we even move to fixed length encoding for some or all of these fields?

[MikeBishop]

Anecdotally, Christian's implementation experience was that it wasn't that hard to implement and did save a noticeable amount of packet overhead. I presume the Google folks have similar experience and possibly numbers.

If we find that we can use a particular length most of the time, it might be reasonable to simplify to only that length and full-length, though. I agree that having so many options can be confusing.

[ianswett]

We don't have any quantitative performance data I'm aware of. In the case of acks, where before we did variably sized packet numbers, acks could consume an entire packet, which caused lots of problems, not just a latency hit. But that was early on, and I don't have good numbers for the impact of that. I will say that there are cost benefits to saving 1-2% of internet bandwidth which may be more important than a small change in latency. The only case that ends up being complex to implement in our experience is the variable packet number length. That's complex because if you retransmit a packet with a longer packet number length, it may suddenly be too large for the path, and be consistently dropped, so you have to save the original packet length for retransmissions. In that case, I can imagine a connection negotiating a single length and using it the entire time, but the current approach allows a fixed size for the sender without negotiation, so it seems simpler overall. Some simplification may be possible, but we've already simplified the easy cases, such as only allowing 0 or 8 byte connection ids(before there were 4 lengths). I will note the ability to send a 6 byte packet number instead of an 8 byte one is silly, and we should just do 1, 2, 4 and 8.

…

On Tue, Nov 29, 2016 at 4:36 PM, MikeBishop ***@***.***> wrote: Anecdotally, Christian's implementation experience was that it wasn't that hard to implement and did save a noticeable amount of packet overhead. I presume the Google folks have similar experience and possibly numbers. If we find that we can use a particular length most of the time, it might be reasonable to simplify to *only* that length and full-length, though. I agree that having so many options can be confusing. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#40 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATJJcWiYdtGsE-hGL8lUTPadIWiTbfDAks5rDJr0gaJpZM4K9hGj> .

[britram]

(from a side conversation with @ekr) one solution to this on the stream layer would be to use a variable-length integer encoding that embeds the length within the encoded value itself, i.e.:

00bbbbbb = 6 bit unsigned
01bbbbbb bbbbbbbb = 14 bit unsigned
10bbbbbb bbbbbbbb bbbbbbbb bbbbbbbb = 30 bit unsigned
11bbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb = 62 bit unsigned

(The utility of being able to quickly calculate offsets without parsing an entire header for values in the packet header means, IMO, that it still makes sense to try and use fixed-length fields or, to the extent possible, fixed-structure headers as in #148)

[mnot]

As discussed in Tokyo, editors to go through their respective headers and come up with proposals to potentially limit variability.

[martinthomson]

With #361, let's call this done for now. Any residual issues can be brought up with new issues (I suspect that we're not entirely done with variable length encodings for packet numbers, for example).