New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reliable identification of the initial packet for a connection #185

Closed
igorlord opened this Issue Jan 20, 2017 · 13 comments

Comments

Projects
None yet
8 participants
@igorlord
Contributor

igorlord commented Jan 20, 2017

Some path elements may need to statelessly identify an initial packet of a connection. These path elements could include load balancers and organization's border security systems. Ability to identify the initial packet may also be required for systems implementing Stateless Rejects #60.

Since QUIC allows for packet number truncation, the most straightforward way to identify the initial packet is by checking for packet number 1 and VERSION flag set.

Hence:

  • The language of section 6.1 Version Negotiation would need to change SHOULD to MUST in "All subsequent packets sent by the client SHOULD have the VERSION flag unset".
  • Section 6.1 Version Negotiation should also gain language like: "If Version negotiation is not complete in 2^6 packets, any endpoint MAY reset this connection". (This is a weaker statement than #143.)
  • Proposal in issue #35 is not compatible with this issue. If packet numbers are allowed to be deliberately skipped, we should require that packets with VERSION set MUST NOT be skipped.
@MikeBishop

This comment has been minimized.

Show comment
Hide comment
@MikeBishop

MikeBishop Jan 21, 2017

Contributor

You're entirely correct that #35 is not compatible, but it's more fundamental: This issue suggests that we make it possible to identify and act on a particular packet when you're not a party to the connection; #35 suggests making it more difficult to act on and therefore ossify an exposed protocol element.

I suspect you'll need to back up and motivate why we want to allow path elements to identify a particular packet in a connection. "Some... may need to" isn't a justification. My gut reaction is that load balancers at least should be statelessly operating on the connection ID itself, whether it's the first packet or the billionth.

Contributor

MikeBishop commented Jan 21, 2017

You're entirely correct that #35 is not compatible, but it's more fundamental: This issue suggests that we make it possible to identify and act on a particular packet when you're not a party to the connection; #35 suggests making it more difficult to act on and therefore ossify an exposed protocol element.

I suspect you'll need to back up and motivate why we want to allow path elements to identify a particular packet in a connection. "Some... may need to" isn't a justification. My gut reaction is that load balancers at least should be statelessly operating on the connection ID itself, whether it's the first packet or the billionth.

@igorlord

This comment has been minimized.

Show comment
Hide comment
@igorlord

igorlord Jan 21, 2017

Contributor

I suspect you'll need to back up and motivate why we want to allow path elements to identify a particular packet in a connection.

Certainly. Let's take a load balancer system. Think LVS or Google Maglev. These systems consist of multiple machines serving as load balancers, each with imperfect availability, and packets belonging to a single connection may arrive at different load balancer servers.

These load balancers are not proxies. They are more like routers. They do not terminate connections but rather forward packets to the appropriate backend servers.

Load balancer's job is to:
a) For a new connection, select an appropriate server to handle it, and
b) Keep sending all packets that are a part of the connection to the same server, and
c) Keep doing (b) even if the set of machines eligible to serve new connections of this type changes dynamically (say, every second)

To do its job, at a minimum, the load balancer needs to identify new connections (a). Identifying first packet (subject of this issue) is trivial with TCP (SYN). It should be trivial with QUIC, too.

Contributor

igorlord commented Jan 21, 2017

I suspect you'll need to back up and motivate why we want to allow path elements to identify a particular packet in a connection.

Certainly. Let's take a load balancer system. Think LVS or Google Maglev. These systems consist of multiple machines serving as load balancers, each with imperfect availability, and packets belonging to a single connection may arrive at different load balancer servers.

These load balancers are not proxies. They are more like routers. They do not terminate connections but rather forward packets to the appropriate backend servers.

Load balancer's job is to:
a) For a new connection, select an appropriate server to handle it, and
b) Keep sending all packets that are a part of the connection to the same server, and
c) Keep doing (b) even if the set of machines eligible to serve new connections of this type changes dynamically (say, every second)

To do its job, at a minimum, the load balancer needs to identify new connections (a). Identifying first packet (subject of this issue) is trivial with TCP (SYN). It should be trivial with QUIC, too.

@igorlord

This comment has been minimized.

Show comment
Hide comment
@igorlord

igorlord Jan 22, 2017

Contributor

One more reason for changing SHOULD to MUST of section 6.1 Version Negotiation in "All subsequent packets sent by the client SHOULD have the VERSION flag unset".

In draft-ietf-quic-tls-01, 6.1.1. Initial Key Transitions, we require:

Packets protected with 1-RTT keys have a KEY_PHASE bit set to 1. These packets also have a VERSION bit set to 0.

If I understand it correctly, TLS 1-RTT keys should be available at the same time as the completion of the Version negotiation. So changing of SHOULD to MUST is mandated by draft-ietf-quic-tls-01 (6.1.1). Right?

Contributor

igorlord commented Jan 22, 2017

One more reason for changing SHOULD to MUST of section 6.1 Version Negotiation in "All subsequent packets sent by the client SHOULD have the VERSION flag unset".

In draft-ietf-quic-tls-01, 6.1.1. Initial Key Transitions, we require:

Packets protected with 1-RTT keys have a KEY_PHASE bit set to 1. These packets also have a VERSION bit set to 0.

If I understand it correctly, TLS 1-RTT keys should be available at the same time as the completion of the Version negotiation. So changing of SHOULD to MUST is mandated by draft-ietf-quic-tls-01 (6.1.1). Right?

@ianswett

This comment has been minimized.

Show comment
Hide comment
@ianswett

ianswett Jan 22, 2017

Contributor

If we want to make identification easier, I'd suggest when the version bit is set, we also include 4 fixed bytes that identify the protocol as QUIC. I'd suggest the string QUIC, so it is easy to identify in wireshark as well.

I have mixed feelings on making it easier to identify UDP traffic QUIC, but I think it's going to be a thing people will do, so I'd rather have them latch onto an explicit signal than hardcode the version number or something worse.(public flags anyone...)

Two notes:

  1. Initial packets from the client cannot have a truncated connection ID, because the peer has to inform you it's acceptable in the handshake.
  2. There are proposals to randomize the initial packet number, so relying on it being packet number 1 would conflict with that.
Contributor

ianswett commented Jan 22, 2017

If we want to make identification easier, I'd suggest when the version bit is set, we also include 4 fixed bytes that identify the protocol as QUIC. I'd suggest the string QUIC, so it is easy to identify in wireshark as well.

I have mixed feelings on making it easier to identify UDP traffic QUIC, but I think it's going to be a thing people will do, so I'd rather have them latch onto an explicit signal than hardcode the version number or something worse.(public flags anyone...)

Two notes:

  1. Initial packets from the client cannot have a truncated connection ID, because the peer has to inform you it's acceptable in the handshake.
  2. There are proposals to randomize the initial packet number, so relying on it being packet number 1 would conflict with that.
@igorlord

This comment has been minimized.

Show comment
Hide comment
@igorlord

igorlord Jan 23, 2017

Contributor

The issue here is not identifying traffic as QUIC (it could be a different issue).

The issue is being able to identify new client connections statelessly (just by observing the packets).

This is needed for load balancers. As a load balancer I want to know whether I need to:

  • pick a backend server for this connection (i.e. processing "TCP SYN" or "QUICK packet 1"), or
  • try to identify which backend server is already handling this connection and forward the packet that way ("TCP ACK or RST" or "QUICK packet > 1").

This is especially important, since I would like to keep load balancers stateless and will want to use "Stateless Reject" to encode the load balancing decision for a particular connection. So I need to be able to statelessly identify initial packets and send "Stateless Reject" to them.

draft-ietf-quic-tls-01 already requires (6.1.1. Initial Key Transitions) that packets without 0-RTT protection have KEY_PHASE=0, VERSION=1; with 0-RTT have KEY_PHASE=1, VERSION=1; with 1-RTT have KEY_PHASE=1, VERSION=0.

So I would like to bring draft-ietf-quic-transport-01 is agreement with draft-ietf-quic-tls-01 and make setting VERSION=0 a MUST instead of SHOULD.

If there is a strong desire (for a good reason) to randomize the initial packet number, I think I can live with it, and send "Stateless Rejects" to all packets with KEY_PHASE=0, VERSION=1.

Contributor

igorlord commented Jan 23, 2017

The issue here is not identifying traffic as QUIC (it could be a different issue).

The issue is being able to identify new client connections statelessly (just by observing the packets).

This is needed for load balancers. As a load balancer I want to know whether I need to:

  • pick a backend server for this connection (i.e. processing "TCP SYN" or "QUICK packet 1"), or
  • try to identify which backend server is already handling this connection and forward the packet that way ("TCP ACK or RST" or "QUICK packet > 1").

This is especially important, since I would like to keep load balancers stateless and will want to use "Stateless Reject" to encode the load balancing decision for a particular connection. So I need to be able to statelessly identify initial packets and send "Stateless Reject" to them.

draft-ietf-quic-tls-01 already requires (6.1.1. Initial Key Transitions) that packets without 0-RTT protection have KEY_PHASE=0, VERSION=1; with 0-RTT have KEY_PHASE=1, VERSION=1; with 1-RTT have KEY_PHASE=1, VERSION=0.

So I would like to bring draft-ietf-quic-transport-01 is agreement with draft-ietf-quic-tls-01 and make setting VERSION=0 a MUST instead of SHOULD.

If there is a strong desire (for a good reason) to randomize the initial packet number, I think I can live with it, and send "Stateless Rejects" to all packets with KEY_PHASE=0, VERSION=1.

@mirjak

This comment has been minimized.

Show comment
Hide comment
@mirjak

mirjak Jan 23, 2017

Contributor

I still don't see why you need to identify the first packet. Isn't for a load balancer simply the first packet of a connection the first ones it sees when it doesn't have a mapping yet? And if the load balancers change dynamically, don't you have to sync state anyway all the time?

Contributor

mirjak commented Jan 23, 2017

I still don't see why you need to identify the first packet. Isn't for a load balancer simply the first packet of a connection the first ones it sees when it doesn't have a mapping yet? And if the load balancers change dynamically, don't you have to sync state anyway all the time?

@igorlord

This comment has been minimized.

Show comment
Hide comment
@igorlord

igorlord Jan 23, 2017

Contributor

That would be true for a completely stateful load balancer. And that's exactly the kind of load balancing system I'd like to avoid having to build -- a system that requires all nodes to be completely in sync and aware of all connections real-time.

This system is very expensive (CPU and bandwidth), especially when you think global scale (not just in-datacenter). And it may not work very reliably anyway due to packets for a single connection arriving very quickly, since if packet 1 and 2 arrive at different nodes, there may had been not enough time for all nodes to get in sync. If you try to fix the previous problem by adding a request-response mechanism for connections you do not know about, you need to worry about your vulnerability to an attack with random packets, all of which may need to go through this expensive request-response. And simply setting up rate limiting on request-response to deal with attacks would not work, since there could be legitimate events that could cause a ton of traffic shifting load balancers all together even in a well-designed system (think BGP change for a global load balancer or a server crash for a local one).

A little more on this in issue #205.

(Our current system for TCP is "mostly stateless".)

Contributor

igorlord commented Jan 23, 2017

That would be true for a completely stateful load balancer. And that's exactly the kind of load balancing system I'd like to avoid having to build -- a system that requires all nodes to be completely in sync and aware of all connections real-time.

This system is very expensive (CPU and bandwidth), especially when you think global scale (not just in-datacenter). And it may not work very reliably anyway due to packets for a single connection arriving very quickly, since if packet 1 and 2 arrive at different nodes, there may had been not enough time for all nodes to get in sync. If you try to fix the previous problem by adding a request-response mechanism for connections you do not know about, you need to worry about your vulnerability to an attack with random packets, all of which may need to go through this expensive request-response. And simply setting up rate limiting on request-response to deal with attacks would not work, since there could be legitimate events that could cause a ton of traffic shifting load balancers all together even in a well-designed system (think BGP change for a global load balancer or a server crash for a local one).

A little more on this in issue #205.

(Our current system for TCP is "mostly stateless".)

@martinthomson

This comment has been minimized.

Show comment
Hide comment
@martinthomson

martinthomson Jan 24, 2017

Member

@ianswett, do you want that four bytes AND #167?

Member

martinthomson commented Jan 24, 2017

@ianswett, do you want that four bytes AND #167?

@ianswett

This comment has been minimized.

Show comment
Hide comment
@ianswett

ianswett Jan 24, 2017

Contributor

I think it was mentioned today that this is no longer an issue, is that correct @igorlord ?

Contributor

ianswett commented Jan 24, 2017

I think it was mentioned today that this is no longer an issue, is that correct @igorlord ?

@igorlord

This comment has been minimized.

Show comment
Hide comment
@igorlord

igorlord Jan 25, 2017

Contributor

@ianswett Yes. It seems resolved for now, although there are proposals (like #203) that may need reopening this issue.

Contributor

igorlord commented Jan 25, 2017

@ianswett Yes. It seems resolved for now, although there are proposals (like #203) that may need reopening this issue.

@janaiyengar

This comment has been minimized.

Show comment
Hide comment
@janaiyengar

janaiyengar Feb 11, 2017

Contributor

How was this issue resolved?

I don't understand the premise of the problem here. If you want to build something completely stateless, the only way to do it is ECMP with a consistent hash. You can do that on any packet in the connection and it won't matter.

But if you're doing anything smarter, you're going to be stateful. You cannot know which server to direct traffic for existing connections to without maintaining state. The state that you need it a table mapping 4-tuple to server for TCP, and you could do the same with connection ID to server for QUIC.

Since you will have this map (this may be the "mostly stateless" part you mention earlier), any received packet that has a connection ID that's missing from the map can create a new entry. You can be smarter about it, but this is a fairly simple and largely effective algorithm that ought to work.

Contributor

janaiyengar commented Feb 11, 2017

How was this issue resolved?

I don't understand the premise of the problem here. If you want to build something completely stateless, the only way to do it is ECMP with a consistent hash. You can do that on any packet in the connection and it won't matter.

But if you're doing anything smarter, you're going to be stateful. You cannot know which server to direct traffic for existing connections to without maintaining state. The state that you need it a table mapping 4-tuple to server for TCP, and you could do the same with connection ID to server for QUIC.

Since you will have this map (this may be the "mostly stateless" part you mention earlier), any received packet that has a connection ID that's missing from the map can create a new entry. You can be smarter about it, but this is a fairly simple and largely effective algorithm that ought to work.

@igorlord

This comment has been minimized.

Show comment
Hide comment
@igorlord

igorlord Feb 11, 2017

Contributor
Contributor

igorlord commented Feb 11, 2017

@MikeBishop

This comment has been minimized.

Show comment
Hide comment
@MikeBishop

MikeBishop Mar 9, 2017

Contributor

#361 purports to address this; please re-open or file a new issue if that's incorrect.

Contributor

MikeBishop commented Mar 9, 2017

#361 purports to address this; please re-open or file a new issue if that's incorrect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment