pkg/k8s: detect k8s mode if env variable K8S_NODE_NAME is set #11021
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Since the k8s service is only created after the container is started, kubelet is not fast enough to set `KUBERNETES_SERVICE_HOST` nor `KUBERNETES_SERVICE_PORT` in a container which can result in Cilium having non-expected behaviors such as: panicking upon initialization; use an autogenerated IPv4 allocated IP as Cilium won't detect which podCIDR the k8s node has set; Re-allocate cilium_host router IP address which can cause network disruption; Inability to restore endpoints since their IP do not belong to the autogenerated CIDR. As all Cilium DaemonSets have the K8S_NODE_NAME environment variable set we can detect if Cilium is running in k8s mode by also checking if this flag is set and not depend on `KUBERNETES_SERVICE_HOST` nor `KUBERNETES_SERVICE_PORT` for this detection. More info: kubernetes/kubernetes#40973 Signed-off-by: André Martins <andre@cilium.io>
aanm
added
kind/bug
|
test-me-please |
raybejjani
approved these changes
Apr 16, 2020
|
This broke the dev VM for developers not running k8s there like @jrajahalme and I, fix will be out soon. |
|
@joestringer @jrajahalme |
raybejjani
added a commit
to raybejjani/cilium
that referenced
this pull request
Apr 17, 2020
…t k8s
We've seen panics where it seems k8s isn't setup correctly but CRD
related operations occur, and segfault. This occurs when the kubernetes
service is not ready by the time cilium starts up and so cilium misses the
KUBERNETES_SERVICE_{HOST,PORT} settings resulting in it being
misconfigured.
See kubernetes/kubernetes#40973
See cilium#11021
Signed-off-by: Ray Bejjani <ray@isovalent.com>
|
We unconditionally write the hostname into the sysconfig file, for example |
pchaigno
pushed a commit
that referenced
this pull request
Apr 21, 2020
…t k8s
We've seen panics where it seems k8s isn't setup correctly but CRD
related operations occur, and segfault. This occurs when the kubernetes
service is not ready by the time cilium starts up and so cilium misses the
KUBERNETES_SERVICE_{HOST,PORT} settings resulting in it being
misconfigured.
See kubernetes/kubernetes#40973
See #11021
Signed-off-by: Ray Bejjani <ray@isovalent.com>
jrajahalme
added a commit
that referenced
this pull request
Apr 21, 2020
Fixes: #11021 Signed-off-by: Jarno Rajahalme <jarno@covalent.io>
christarazi
pushed a commit
that referenced
this pull request
Apr 22, 2020
Fixes: #11021 Signed-off-by: Jarno Rajahalme <jarno@covalent.io>
maintainer-s-little-helper
bot
moved this from Needs backport from master
to Backport pending to v1.7
in 1.7.3
jrfastab
pushed a commit
that referenced
this pull request
Apr 23, 2020
…t k8s [ upstream commit 1598f74 ] We've seen panics where it seems k8s isn't setup correctly but CRD related operations occur, and segfault. This occurs when the kubernetes service is not ready by the time cilium starts up and so cilium misses the KUBERNETES_SERVICE_{HOST,PORT} settings resulting in it being misconfigured. See kubernetes/kubernetes#40973 See #11021 Signed-off-by: Ray Bejjani <ray@isovalent.com> Signed-off-by: John Fastabend <john.fastabend@gmail.com>
jrfastab
pushed a commit
that referenced
this pull request
Apr 28, 2020
…t k8s [ upstream commit 1598f74 ] We've seen panics where it seems k8s isn't setup correctly but CRD related operations occur, and segfault. This occurs when the kubernetes service is not ready by the time cilium starts up and so cilium misses the KUBERNETES_SERVICE_{HOST,PORT} settings resulting in it being misconfigured. See kubernetes/kubernetes#40973 See #11021 Signed-off-by: Ray Bejjani <ray@isovalent.com> Signed-off-by: John Fastabend <john.fastabend@gmail.com>
joestringer
pushed a commit
that referenced
this pull request
Apr 29, 2020
…t k8s [ upstream commit 1598f74 ] We've seen panics where it seems k8s isn't setup correctly but CRD related operations occur, and segfault. This occurs when the kubernetes service is not ready by the time cilium starts up and so cilium misses the KUBERNETES_SERVICE_{HOST,PORT} settings resulting in it being misconfigured. See kubernetes/kubernetes#40973 See #11021 Signed-off-by: Ray Bejjani <ray@isovalent.com> Signed-off-by: John Fastabend <john.fastabend@gmail.com>
maintainer-s-little-helper
bot
moved this from Backport pending to v1.7
to Backport done to v1.7
in 1.7.3
christarazi
pushed a commit
that referenced
this pull request
Apr 30, 2020
…t k8s [ upstream commit 1598f74 ] We've seen panics where it seems k8s isn't setup correctly but CRD related operations occur, and segfault. This occurs when the kubernetes service is not ready by the time cilium starts up and so cilium misses the KUBERNETES_SERVICE_{HOST,PORT} settings resulting in it being misconfigured. See kubernetes/kubernetes#40973 See #11021 Signed-off-by: Ray Bejjani <ray@isovalent.com> Signed-off-by: Chris Tarazi <chris@isovalent.com>
maintainer-s-little-helper
bot
moved this from Needs backport from master
to Backport pending to v1.6
in 1.6.9
maintainer-s-little-helper
bot
moved this from Needs backport from master
to Backport pending to v1.6
in 1.6.9
joestringer
pushed a commit
that referenced
this pull request
May 4, 2020
…t k8s [ upstream commit 1598f74 ] We've seen panics where it seems k8s isn't setup correctly but CRD related operations occur, and segfault. This occurs when the kubernetes service is not ready by the time cilium starts up and so cilium misses the KUBERNETES_SERVICE_{HOST,PORT} settings resulting in it being misconfigured. See kubernetes/kubernetes#40973 See #11021 Signed-off-by: Ray Bejjani <ray@isovalent.com> Signed-off-by: Chris Tarazi <chris@isovalent.com>
maintainer-s-little-helper
bot
moved this from Backport pending to v1.6
to Backport done to v1.6
in 1.6.9
maintainer-s-little-helper
bot
moved this from Backport pending to v1.6
to Backport done to v1.6
in 1.6.9
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Since the k8s service is only created after the container is started,
kubelet is not fast enough to set
KUBERNETES_SERVICE_HOSTnorKUBERNETES_SERVICE_PORTin a container which can result in Ciliumhaving non-expected behaviors such as: panicking upon initialization; use
an autogenerated IPv4 allocated IP as Cilium won't detect which podCIDR
the k8s node has set; Re-allocate cilium_host router IP address which
can cause network disruption; Inability to restore endpoints since their
IP do not belong to the autogenerated CIDR.
As all Cilium DaemonSets have the K8S_NODE_NAME environment variable set
we can detect if Cilium is running in k8s mode by also checking if this
flag is set and not depend on
KUBERNETES_SERVICE_HOSTnorKUBERNETES_SERVICE_PORTfor this detection.More info: kubernetes/kubernetes#40973
Signed-off-by: André Martins andre@cilium.io