New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AKS: avoid overlapping pod and service CIDRs #31504
Conversation
The default service CIDR of AKS clusters is 10.0.0.0/16 [1]. Unfortunately, we don't set a pod cidr for clusterpool IPAM, and hence use cilium's default of 10.0.0.0/8, which overlaps. This can lead to "fun" situations in which e.g. the kube-dns service ClusterIP is the same as the hubble-relay pod IP, or similar shenanigans. This usually breaks the cluster utterly. The fix is relatively straight-forward: set a pod CIDR for cilium which does not overlap with defaults of AKS. We chose 192.168.0.0/16 as this is what is recommended in [2]. [1]: https://learn.microsoft.com/en-us/azure/aks/configure-kubenet#create-an-aks-cluster-with-system-assigned-managed-identities [2]: https://learn.microsoft.com/en-us/azure/aks/azure-cni-powered-by-cilium#option-1-assign-ip-addresses-from-an-overlay-network Fixes: fbf3d38 (ci: add AKS workflow) Co-authored-by: Fabian Fischer <fabian.fischer@isovalent.com> Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
/ci-aks |
/test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
@viktor-kurchenko I don't know enough about our CI to know whether this should be backported or not - are these workflows run from Also, do we have this bug on other cloud providers? Seems like an easy oversight to make. Finally, this probably also wants to be documented somewhere? |
Marked for backport to all stable branches, as run from the respective workflow files. |
@bimmlerd sorry for the delay. To be honest I don't where we can document this ( |
Follow cilium/cilium#31504, quoting its commit description: > The default service CIDR of AKS clusters is 10.0.0.0/16 [1]. > Unfortunately, we don't set a pod cidr for clusterpool IPAM, and hence > use cilium's default of 10.0.0.0/8, which overlaps. This can > lead to "fun" situations in which e.g. the kube-dns service ClusterIP is > the same as the hubble-relay pod IP, or similar shenanigans. This > usually breaks the cluster utterly. > > The fix is relatively straight-forward: set a pod CIDR for cilium which > does not overlap with defaults of AKS. We chose 192.168.0.0/16 as this > is what is recommended in [2]. > > [1]: https://learn.microsoft.com/en-us/azure/aks/configure-kubenet#create-an-aks-cluster-with-system-assigned-managed-identities > [2]: https://learn.microsoft.com/en-us/azure/aks/azure-cni-powered-by-cilium#option-1-assign-ip-addresses-from-an-overlay-network Co-authored-by: Fabian Fischer <fabian.fischer@isovalent.com> Co-authored-by: David Bimmler <david.bimmler@isovalent.com> Signed-off-by: Tobias Klauser <tobias@cilium.io>
Follow cilium/cilium#31504, quoting its commit description: > The default service CIDR of AKS clusters is 10.0.0.0/16 [1]. > Unfortunately, we don't set a pod cidr for clusterpool IPAM, and hence > use cilium's default of 10.0.0.0/8, which overlaps. This can > lead to "fun" situations in which e.g. the kube-dns service ClusterIP is > the same as the hubble-relay pod IP, or similar shenanigans. This > usually breaks the cluster utterly. > > The fix is relatively straight-forward: set a pod CIDR for cilium which > does not overlap with defaults of AKS. We chose 192.168.0.0/16 as this > is what is recommended in [2]. > > [1]: https://learn.microsoft.com/en-us/azure/aks/configure-kubenet#create-an-aks-cluster-with-system-assigned-managed-identities > [2]: https://learn.microsoft.com/en-us/azure/aks/azure-cni-powered-by-cilium#option-1-assign-ip-addresses-from-an-overlay-network Co-authored-by: Fabian Fischer <fabian.fischer@isovalent.com> Co-authored-by: David Bimmler <david.bimmler@isovalent.com> Signed-off-by: Tobias Klauser <tobias@cilium.io>
Follow cilium/cilium#31504, quoting its commit description: > The default service CIDR of AKS clusters is 10.0.0.0/16 [1]. > Unfortunately, we don't set a pod cidr for clusterpool IPAM, and hence > use cilium's default of 10.0.0.0/8, which overlaps. This can > lead to "fun" situations in which e.g. the kube-dns service ClusterIP is > the same as the hubble-relay pod IP, or similar shenanigans. This > usually breaks the cluster utterly. > > The fix is relatively straight-forward: set a pod CIDR for cilium which > does not overlap with defaults of AKS. We chose 192.168.0.0/16 as this > is what is recommended in [2]. > > [1]: https://learn.microsoft.com/en-us/azure/aks/configure-kubenet#create-an-aks-cluster-with-system-assigned-managed-identities > [2]: https://learn.microsoft.com/en-us/azure/aks/azure-cni-powered-by-cilium#option-1-assign-ip-addresses-from-an-overlay-network Co-authored-by: Fabian Fischer <fabian.fischer@isovalent.com> Co-authored-by: David Bimmler <david.bimmler@isovalent.com> Signed-off-by: Tobias Klauser <tobias@cilium.io>
The default service CIDR of AKS clusters is 10.0.0.0/16 1. Unfortunately, we don't set a pod cidr for clusterpool IPAM, and hence use cilium's default of 10.0.0.0/8, which overlaps. This can lead to "fun" situations in which e.g. the kube-dns service ClusterIP is the same as the hubble-relay pod IP, or similar shenanigans. This usually breaks the cluster utterly.
The fix is relatively straight-forward: set a pod CIDR for cilium which does not overlap with defaults of AKS. We chose 192.168.0.0/16 as this is what is recommended in 2.
Fixes: fbf3d38 (ci: add AKS workflow)
Fixes: #30905