iptables: Fix wrong use of podCIDR in cluster node NAT exclusion #26397
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
By default, in the iptables-based masquerading mode, Cilium will only masquerade traffic coming from the local pod CIDR (`allocRange` in `installMasqueradeRules`). However, many IPAM modes such as ENI or multi-pool IPAM do not have a single pod CIDR. Instead, those modes rely on the `egress-masquerade-interfaces` setting, which masquerades all traffic if it leaves one of the `egress-masquerade-interfaces` devices. Therefore, the "exclude traffic to cluster nodes from masquerade" `CILIUM_POST_nat` rule should also respect the `egress-masquerade-interfaces` setting and not masquerade traffic regardless of the value of `allocRange` (which will not be valid in settings such as ENI mode). This likely has not manifested in ENI mode as an issue, because in ENI mode we derive the native routing CIDR (`snatDstExclusionCIDR` in `installMasqueradeRules`) from the EC2 VPC CIDR, which usually contains the node IPs too. However, we should not rely on that, since we are adding additional non-podCIDR based IPAM modes such as multi-pool where this will not be true. Related: cilium#22273 Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
gandro
added
the
release-note/bug
|
/test |
christarazi
added
sig/datapath
|
ci-ginko failed with LVH provisioning errors https://github.com/cilium/cilium/actions/runs/5334904372/jobs/9667395901 https://github.com/cilium/cilium/actions/runs/5334904372/jobs/9667397270 Restarting |
|
Louis says he does not have time to review |
jibi
approved these changes
Jun 27, 2023
|
Marking ready to merge. |
gandro
added
the
ready-to-merge
gandro
added a commit
to gandro/cilium
that referenced
this pull request
Jun 28, 2023
With cilium#26397 merged, iptables-based masquerading can now be used together with Multi-Pool IPAM, as long as `egressMasqueradeInterfaces` is set too. This commit adjusts the documentation to reflect that and improves the wording of that section a bit. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
borkmann
pushed a commit
that referenced
this pull request
Jul 3, 2023
With #26397 merged, iptables-based masquerading can now be used together with Multi-Pool IPAM, as long as `egressMasqueradeInterfaces` is set too. This commit adjusts the documentation to reflect that and improves the wording of that section a bit. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
joamaki
pushed a commit
to joamaki/cilium
that referenced
this pull request
Jul 6, 2023
[ upstream commit 970f881 ] With cilium#26397 merged, iptables-based masquerading can now be used together with Multi-Pool IPAM, as long as `egressMasqueradeInterfaces` is set too. This commit adjusts the documentation to reflect that and improves the wording of that section a bit. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> Signed-off-by: Jussi Maki <jussi@isovalent.com>
jibi
pushed a commit
that referenced
this pull request
Jul 10, 2023
[ upstream commit 970f881 ] With #26397 merged, iptables-based masquerading can now be used together with Multi-Pool IPAM, as long as `egressMasqueradeInterfaces` is set too. This commit adjusts the documentation to reflect that and improves the wording of that section a bit. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> Signed-off-by: Jussi Maki <jussi@isovalent.com>
ldelossa
pushed a commit
to ldelossa/cilium
that referenced
this pull request
Sep 27, 2023
[ upstream commit 970f881 ] With cilium#26397 merged, iptables-based masquerading can now be used together with Multi-Pool IPAM, as long as `egressMasqueradeInterfaces` is set too. This commit adjusts the documentation to reflect that and improves the wording of that section a bit. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> Signed-off-by: Jussi Maki <jussi@isovalent.com> Signed-off-by: Gilberto Bertin <jibi@cilium.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
By default, in the iptables-based masquerading mode, Cilium will only masquerade traffic coming from the local pod CIDR (
allocRangeininstallMasqueradeRules). However, many IPAM modes such as ENI or multi-pool IPAM do not have a single pod CIDR. Instead, those modes rely on theegress-masquerade-interfacessetting, which only masquerades traffic if it leaves one of theegress-masquerade-interfacesdevices.Therefore, the "exclude traffic to cluster nodes from masquerade"
CILIUM_POST_natrule should also respect theegress-masquerade-interfacessetting and not masquerade traffic regardless of the value ofallocRange(which will not be valid in settings such as ENI mode).This likely has not manifested in ENI mode as an issue, because in ENI mode we derive the native routing CIDR (
snatDstExclusionCIDRininstallMasqueradeRules) from the EC2 VPC CIDR, which usually contains the node IPs too. However, we should not rely on that, since we are adding additional non-podCIDR based IPAM modes such as multi-pool where this will not be true.Related: #22273
Open question: Should this be backported to v1.13? I'm not aware of any actual bugs this has caused so far in ENI mode (see above), but we should fix it nonetheless. I'd particularly would like to have this in v1.14, since it will fix iptables-based masquerading for the newly added Multi-Pool IPAM mode in v1.14.