Egress policies for raw-TCP traffic

[haiyanmeng]

Decisions to be made:

• Should we allow Substrate admin to configure how raw-TCP egress traffic should be handled?
• Are there any needs to do credential injections for raw-TCP egress traffic?

[haiyanmeng]

Eitan Yarmush (@EItanya) , IIUC, currently both dataplanes (Envoy and AgentGateway) allow raw-TCP traffic to pass through directly when MITM is enabled.

Based on your #1024 (comment) , it seems that you think we should provide some knobs to control this behavior. Right?

cc Lior Lieberman (@LiorLieberman) Bowei Du (@bowei) Keith Mattix II (@keithmattix)

[keithmattix]

We should definitely do this, but o believe it was explicitly deferred

[haiyanmeng]

Eitan Yarmush (@EItanya) , if this work is deferred, may I switch #1024 from draft back to Ready for Review, since it aims to test raw-TCP traffic is allowed to passthrough when MITM is enabled?

[EItanya]

I think we need to pick how it we want it to work. For example, are all protocols served on a single port? The PEP is meant to be pluggable so we should come up with a standard here. For example, do we protocol sniff and serve all traffic types on one port, or do we allow for multiple ports with protocol selection?

[keithmattix]

Agreed; problem is that we can't possibly sniff and support server first TCP protocols (e.g SSH or MySQL) on the same port. So we have to define some way to tell the PEP that certain outbound ports are for server first protocols or are explicitly TCP

[haiyanmeng]

problem is that we can't possibly sniff and support server first TCP protocols (e.g SSH or MySQL) on the same port.

+1

So we have to define some way to tell the PEP that certain outbound ports are for server first protocols or are explicitly TCP

IIUC, the behavior of the egress gateway config of both dataplanes when MITM is enabled is:

1. if it is HTTPS traffic, do MITM and enforce egress policies;
2. if it is HTTP traffic, enforce egress policies;
3. otherwise (this is raw-TCP traffic), allow the traffic to passthrough.

Keith Mattix II (@keithmattix) , are you suggesting for the (3) case, the PEP should have a port allowlist, instead of letting all raw-TCP traffic passthrough?

[keithmattix]

are you suggesting for the (3) case, the PEP should have a port allowlist, instead of letting all raw-TCP traffic passthrough?

My point is that the only way to know whether traffic falls into the 1 or 2 category is pausing processing and examining the first bytes that are sent from the client. If you try that with a server-first protocol, then you'll pause forever because the server sends the first bytes. You can set a timeout on the sniffing, but then your HTTP policy is trivially bypassable if you just hold the connection open longer than the listener timeout and then send your HTTP traffic.

This is why I think we need some degree of opt in for either TCP or HTTP traffic; my vote is to make TCP opt in because HTTP is more common