Latency Spin Bit #609

huitema · 2017-06-08T07:35:34Z

Adding text to the transport spec to describe the "latency spin bit".

Syncto 03 drafts

This is the result of the discussion of latency monitoring during the Paris interim meeting.

britram · 2017-06-08T07:40:27Z

I like this approach, but would like to dig into how it breaks in corner cases.

It seems to me to be quite related to "coloring" based approaches, see e.g. https://datatracker.ietf.org/doc/draft-ietf-ippm-alt-mark/

hardie · 2017-06-08T07:45:11Z

I also like this approach, and I think the one key corner case (out-of-order packet arrival) can be handled by description of how to smooth the base RTT signal. That would allow out of color packets to be seen as out-of-order indications rather than screwing up the RTT calculation.

martinthomson

Is this a per-path signal? I think that it is and that we need more text about that.

martinthomson · 2017-06-08T07:47:57Z

draft-ietf-quic-transport.md

@@ -198,6 +198,8 @@ strengths of QUIC include:

 * Version negotiation

+* Support for monitoring


I would rather that we not include this section.

in any case "support for monitoring" is much broader than what this is.

If you completely remove the question, then the bit description just appears out of nowhere. But then, yes, we could be specific about latency monitoring, rather than monitoring in general.

mikkelfj · 2017-06-08T07:52:37Z

I would suggest to make it optional. There are some privacy issues associated with this allowing to assocate packets with a connection. Though UDP also associates packets, alternate transport layers can obfuscate routing.

mirjak · 2017-06-08T08:02:59Z

Instead of using a 'whole' bit, you can also have another packet type for protected packets. And if you define a well know patter that is used, you can even measure loss with it.

mirjak · 2017-06-08T08:28:07Z

Or just have the flag on short packets...

Adding a reference to draft-ietf-ippm-alt-mark, as Brian suggested in his review.

huitema · 2017-06-08T08:46:06Z

I get Martin's point about per path signal, which it indeed is. But then, we don't really have an end-to-end vs path distinction yet. In the current transport draft, there seems to be just one path at a time. I think the distinction will only become apparent when we start specifying QUIC multipath.

ianswett

There's no discussion of what to do when marked packets are lost. I think there needs to be to ensure the signal doesn't die during the connection.

ianswett · 2017-06-08T08:41:59Z

draft-ietf-quic-transport.md

@@ -294,6 +296,15 @@ QUIC version negotiation allows for multiple versions of the protocol to be
 deployed and used concurrently. Version negotiation is described in
 {{version-negotiation}}.

+## Support for Latency Monitoring
+
+Network administrators often find difficult to monitor the behavior


nit: how about "Network administrators have difficulty monitoring ..."?

ianswett · 2017-06-08T08:42:12Z

draft-ietf-quic-transport.md

+
+Network administrators often find difficult to monitor the behavior
+of encrypted protocols. Bandwidth can be monitored by observing the
+flow of encrypted packets, but the usal tools for monitoring


ianswett · 2017-06-08T08:45:17Z

draft-ietf-quic-transport.md

+flow of encrypted packets, but the usal tools for monitoring
+latency require observing packet numbers and acknowledgements, which
+in QUIC are encrypted.
+QUIC packets include a clear text latency spin bit that enables


nit: "QUIC packets enable monitoring of latency by including a clear text latency spin bit that is reflected between peers."

ianswett · 2017-06-08T08:47:42Z

draft-ietf-quic-transport.md

+the network path. This bit is set as follow:
+
+* The connection responder sets the spin bit value to the value of the
+  spin bit in the last packet received from the connection initiator.


What if two packets were received and the first had the spin bit set and the second did not? I would think the spin bit would still be set by the responder in that case?

Also, can you add clarifying text about what to do when exiting quiescence?

Ian, there are two references to quiescence in the current draft. One about the Ping frame, "The default is to send a PING frame after 15 seconds of quiescence." The other about flow control, "it is generally considered best to not let the sender go into quiescence if avoidable." But the concept of quiescence is not well defined. Is it just "has not sent packet for a long time"?
What would you suggest to do?

What if two packets were received and the first had the spin bit set and the second did not? I would think the spin bit would still be set by the responder in that case?

If an old packet race ahead this could add noise to signal. I think it would be good to explain how this would work, or alternatively filter out the noise by only flipping when a new highest packet number is received. Noise could probably be used effectively if understood by analyser.

Scratch my comment, I think if we define this correctly, we don't need to talk about quiescence, it'll just naturally work out.

One more comment: I think we need text on packet reordering here. Possibly this should be the largest packet received instead of the last?

huitema · 2017-06-08T12:53:00Z

Mirja, I hesitated between the "packet type" design and the "spin bit". I like having Red/Blue packet types, but I am concerned with the interaction with issue 311, "GREASE the packet type octet". If we do grease the packet type, we will need to leave the spin bit in clear text. This is much easier to do with a reserved bit than with the packet type.

mirjak · 2017-06-08T13:13:41Z

Yes, there is dependency on greasing. We could also just grease the long header because the short header has a bunch of flags that can anyway change all the time. I even thought about having only flags with the short header (and no type anymore) because that's nearly what we have already and that makes sense to me (and also looks slightly clearer to me) because all control packets are long headers. For me the point of the short header is basically a packet where you want to save some of the header bits and use it for payload instead and the flags simply define which part of the header bits you omit.

mikkelfj · 2017-06-08T14:58:33Z

There is also the option to grease the type but reserve a single ungreased value for monitoring packets.

The benefit of a separate packet is both more payload for monitoring, and disassociation with other packets to better protect against timing analysis.

ianswett

I like the 'spin bit' approach personally.

ianswett · 2017-06-08T22:24:20Z

draft-ietf-quic-transport.md

+the network path. This bit is set as follow:
+
+* The connection responder sets the spin bit value to the value of the
+  spin bit in the last packet received from the connection initiator.


Also, can you add clarifying text about what to do when exiting quiescence?

martinthomson · 2017-06-09T07:09:00Z

There is also the option to grease the type but reserve a single ungreased value for monitoring packets.

This is unaffected by my favourite form of greasing (xor with the last octet of the unencrypted frame). We have a choice as to whether or not this is greased, of course.

martinthomson · 2017-06-09T15:04:36Z

Thinking about this some more, this bit will flip lots in ways that aren't immediately obvious. I think that we can safely not grease this. (We could also grease it, but that makes it a teensy bit less accessible to the devices who we intend to consume it, so not greasing is probably best.) Either way, that's not a question we need to resolve with the change as it stands.

I still strongly think that we should avoid adding this to the list of transport features, and I think that you should address @ianswett's comments about dropping an entire flight (the answer being just "oh well" here).

martinthomson · 2017-06-09T15:05:50Z

Oh, and I really think that this needs to be crisp about being per-path. I think that we need to explicitly sasy that we reset the signal when the IP or port at either end changes.

Fixes suggested in Ian's review.

ianswett · 2017-06-13T14:13:21Z

draft-ietf-quic-transport.md

+* The connection responder sets the spin bit value to the value of the
+  spin bit in the last packet received from the connection initiator.
+
+* The connection initiator sets the spin bit value to the opposite


I'm confused about why it's the opposite? My mental model is that the bit would be set to 1 on a packet, and then it would be bounced back and forth, so there would only be a single packet marked with the 1 spin bit at any given point in time.

Ok, ekr explained to me out of band that I misunderstood how the spin bit worked. I think the design here is more robust to loss than what I was thinking of, so I'm happy with it.

However, an overview sentence explaining what it's doing (ie: mark the entire flight with the spin bit, etc) would have prevented my confusion.

janaiyengar · 2017-06-13T14:35:05Z

Can we please move discussions to issues, and not have them on PRs?

britram · 2017-06-14T14:25:30Z

@janaiyengar I filed #631 for that purpose.

britram · 2017-06-14T14:39:54Z

And the loss-detection inference ability of this proposal can be discussed in #632.

janaiyengar · 2017-06-14T19:56:08Z

I've moved discussion to the issue (thanks for filing it, @britram).

ianswett · 2017-06-14T20:09:21Z

@janaiyengar To clarify, I think my comments were editorial, and so those still belong here, not in the issue?

janaiyengar · 2017-06-14T20:14:42Z

Ian -- I wasn't responding to your comments specifically, but I wanted to move general discussion over to the issue.

…

On Wed, Jun 14, 2017 at 9:09 PM, ianswett ***@***.***> wrote: @janaiyengar <https://github.com/janaiyengar> To clarify, I think my comments were editorial, and so those still belong here, not in the issue? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#609 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKjg1AlptAKveShqQ2-_WMy689YPEfzFks5sED30gaJpZM4Nzrwr> .

ianswett · 2017-06-14T20:17:09Z

Great, thanks for clarifying.

mikkelfj · 2017-06-18T06:09:22Z

draft-ietf-quic-transport.md

+the network path. This bit is set as follow:
+
+* The connection responder sets the spin bit value to the value of the
+  spin bit in the last packet received from the connection initiator.


The terminology elsewhere appears to be Client and Server, not Connecton Initiator and Connection Responder, although I prefer the latter since the protocol is not necessarily used in a client-server setup.

"connection initiator" and "connection responder" are just pretentious ways of saying client and server. We happily use client-server protocols in non-client-server arrangements all the time.

Martin, initiator and responder are certainly equivalent to client and server when the application is HTTP. Not necessarily so is other applications, e.g. DNS or Web RTC. But as you seem to feel strongly about it, lets use Client and Server in the PR to make it consistent with the rest of the document.

Replacing initiator/responder by client and server for consistency with the rest of the document. Adding some minimal explanation of the expected behavior.

The phrase is a bit clumsy, but this solves the issues of flipping the bit when packets are received out of order.

ianswett

Thanks Christian, I think this LGTM.

ianswett · 2017-06-27T00:52:57Z

draft-ietf-quic-transport.md


 * The client sets the spin bit value to the opposite
-  of the last value received from the server, or to 0
+  of the value set in the packet received from the server with the
+  largest sequence number, or to 0


nit: I think these two lines may fit on one line?

martinthomson · 2018-01-15T23:37:33Z

Aside from being bitrotten, #1046 covers this now.

huitema added 2 commits June 8, 2017 09:08

Merge pull request #1 from quicwg/master

ed3f170

Syncto 03 drafts

Adding description of the latency spin bit

112e1f7

This is the result of the discussion of latency monitoring during the Paris interim meeting.

Update draft-ietf-quic-transport.md

f57f7f4

martinthomson reviewed Jun 8, 2017

View reviewed changes

Rename the section to "support latency monitoring"

18434c7

Adding reference to IPPM draft

ab881a9

Adding a reference to draft-ietf-ippm-alt-mark, as Brian suggested in his review.

ianswett reviewed Jun 8, 2017

View reviewed changes

martinthomson added -transport design An issue that affects the design of the protocol; resolution requires consensus. labels Jun 12, 2017

Editorial fixes

0afcbb9

Fixes suggested in Ian's review.

ianswett reviewed Jun 13, 2017

View reviewed changes

britram mentioned this pull request Jun 14, 2017

On-path calculation of RTT #631

Closed

This was referenced Jun 14, 2017

Packet number echo with fixed-length, 32-bit packet and echo numbers #393

Closed

Packet number echo with variable-length numbering #391

Closed

On-path calculation of loss and congestion #632

Closed

mikkelfj reviewed Jun 18, 2017

View reviewed changes

huitema added 2 commits June 18, 2017 18:31

Changes suggested by Martin and Ian

981d9e2

Replacing initiator/responder by client and server for consistency with the rest of the document. Adding some minimal explanation of the expected behavior.

Largest sequence number

47b685c

The phrase is a bit clumsy, but this solves the issues of flipping the bit when packets are received out of order.

ianswett reviewed Jun 27, 2017

View reviewed changes

janaiyengar added the parked An issue that we can't immediately address; for future discussion. label Aug 15, 2017

mnot removed the design An issue that affects the design of the protocol; resolution requires consensus. label Sep 25, 2017

britram mentioned this pull request Jan 9, 2018

Latency Spin Bit, 2018 edition #1046

Closed

martinthomson closed this Jan 15, 2018

		@@ -198,6 +198,8 @@ strengths of QUIC include:

		* Version negotiation

		* Support for monitoring

Latency Spin Bit #609

Latency Spin Bit #609

Conversation

huitema commented Jun 8, 2017

britram commented Jun 8, 2017

hardie commented Jun 8, 2017

martinthomson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mikkelfj commented Jun 8, 2017

mirjak commented Jun 8, 2017

mirjak commented Jun 8, 2017

huitema commented Jun 8, 2017

ianswett left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

huitema commented Jun 8, 2017

mirjak commented Jun 8, 2017

mikkelfj commented Jun 8, 2017 • edited Loading

ianswett left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martinthomson commented Jun 9, 2017

martinthomson commented Jun 9, 2017

martinthomson commented Jun 9, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

janaiyengar commented Jun 13, 2017

britram commented Jun 14, 2017

britram commented Jun 14, 2017

janaiyengar commented Jun 14, 2017

ianswett commented Jun 14, 2017

janaiyengar commented Jun 14, 2017 via email

ianswett commented Jun 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ianswett left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martinthomson commented Jan 15, 2018

mikkelfj commented Jun 8, 2017 •

edited

Loading