Consider the following setup:
- VPC 1 exposes 10.0.1.0/24, no NAT
- VPC 2 exposes public prefix 10.0.10.0/24 (masqueraded) to VPC 1
- VPC 3 exposes public prefix 10.0.10.0/24 (masqueraded) to VPC 1
Endpoint 10.0.10.10 on VPC 2 initiates a flow towards 10.0.1.1:80 on VPC 1, and gets allocated source port 9999. When endpoint 10.0.10.10 on VPC 3 attempts to initiate a flow towards 10.0.1.1:80 VPC 1, it should never get the same source port 9999 allocated - not for the same destination VPC, source IP, destination IP, and port.
However, there's nothing preventing this to happen in the current code. The NAT allocator was built at a time where I believed we wouldn't support overlapping prefixes exposed with masquerading, and there's no mechanism in place to prevent that case.
The consequence is that the packet from VPC 3 will create flow entries in both directions, the key for the return traffic (source VPC: VPC 1, from 10.0.1.1:80 to 10.0.10.10:9999) will be identical as for return traffic towards VPC 2, and will overwrite it; the destination VPC for flow from VPC 2 will become VPC 3. Follow-up replies for the flow from VPC 2 will get routed to VPC 3 instead, resulting in a possible cross-tenancy leak. Meanwhile, traffic from VPC 3 will get dropped by the endpoint at 10.0.1.1, given that the TCP socket on port 80 for 10.0.10.10:9999 is already in use.
We need to find a way to fix this.
Consider the following setup:
Endpoint 10.0.10.10 on VPC 2 initiates a flow towards 10.0.1.1:80 on VPC 1, and gets allocated source port 9999. When endpoint 10.0.10.10 on VPC 3 attempts to initiate a flow towards 10.0.1.1:80 VPC 1, it should never get the same source port 9999 allocated - not for the same destination VPC, source IP, destination IP, and port.
However, there's nothing preventing this to happen in the current code. The NAT allocator was built at a time where I believed we wouldn't support overlapping prefixes exposed with masquerading, and there's no mechanism in place to prevent that case.
The consequence is that the packet from VPC 3 will create flow entries in both directions, the key for the return traffic (source VPC: VPC 1, from 10.0.1.1:80 to 10.0.10.10:9999) will be identical as for return traffic towards VPC 2, and will overwrite it; the destination VPC for flow from VPC 2 will become VPC 3. Follow-up replies for the flow from VPC 2 will get routed to VPC 3 instead, resulting in a possible cross-tenancy leak. Meanwhile, traffic from VPC 3 will get dropped by the endpoint at 10.0.1.1, given that the TCP socket on port 80 for 10.0.10.10:9999 is already in use.
We need to find a way to fix this.