Kubernetes 1.12 and flannel does not work out of the box #1044
Comments
I can confirm as well—on 1.11.3 the configuration applies correctly. On 1.12.0 it does not. |
usign the toleration without a key worked for me. Would this be the solution? |
That sounds fine to me - flannel should probably tolerate all NoSchedule taints, since it's a critical piece of infrastructure. Anyone want to submit a PR? |
@caseydavenport I have submitted PR against master https://github.com/coreos/flannel/pull/1045/files But it will be good to have the same fix for the tag Considering that this is just a configuration change, maybe make a release |
thanks @outcoldman. it helps :) |
There seems to be an issue and deadlock with Flannel on v1.12 clusters where Flannel pods don't start on unready nodes and nodes don't become ready w/o Flannel / container networking. Issue upstream, albeit with kubeadm: coreos/flannel#1044 Follow up on commit or revert.
thanks @outcoldman ! it works like a charm. ;) |
Flannel should probably set
as the default tolerations set. This will ensure that the flannel ds tolerates all taints. |
For anyone willing to test the flannel fix for 1.12 , |
#trying on a pi2 b+ master kubeadm -n kube-system apply -f https://raw.githubusercontent.com/coreos/flannel/bc79dd1505b0c8681ece4de4c0d86c5cd2643275/Documentation/kube-flannel.ymlError: unknown command "apply" for "kubeadm" |
Try with kubectl rather than kubeadm. I think that was a typo.
Unfortunately I don't have a 1.12 cluster at the moment. I'll test this
when I do though.
…On Fri, 5 Oct 2018, 16:25 NerdyShawn, ***@***.***> wrote:
For anyone willing to test the flannel fix for 1.12 ,
kubeadm -n kube-system apply -f
https://raw.githubusercontent.com/coreos/flannel/bc79dd1505b0c8681ece4de4c0d86c5cd2643275/Documentation/kube-flannel.yml
#trying on a pi2 b+ master
`HypriotOS/armv7: ***@***.***Node01 in ~
kubeadm -n kube-system apply -f
https://raw.githubusercontent.com/coreos/flannel/bc79dd1505b0c8681ece4de4c0d86c5cd2643275/Documentation/kube-flannel.yml
Error: unknown command "apply" for "kubeadm"
Run 'kubeadm --help' for usage.
error: unknown command "apply" for "kubeadm"
`
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1044 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AKoi5ounjV64Fz2gZD0SGvXbTrK-4JHxks5uh3nagaJpZM4W9xBu>
.
|
Hello, |
Hi @NerdyShawn, I don't think you've got your kubectl configured correctly to connect to your cluster. As it seems like @rberg2 has managed to get this working, maybe it would be good to continue this on one of the support channels like slack rather than this issue. |
Sorry, it was a typo, it's kubectl. For those interested, k8s 1.12 deployment with all the goodies (ingress, dashboard, optional vsphere*, etc) automated with ansible and maintained here: github.com/ReSearchITEng/kubeadm-playbook/ |
@ReSearchITEng, confirm works (1.12.1). |
Hello, Please find the output of the pods:- [user@darshan-p-hegde-89ca8c531 ~]$ kubectl get pods -n kube-system I have described the flannel pod and and the output is below:- Name: kube-flannel-ds-amd64-gm5z7 Normal Scheduled 6m57s default-scheduler Successfully assigned kube-system/kube-flannel-ds-amd64-gm5z7 to darshan-p-hegde-89ca8c531.mylabserver.com Please find the output of the coreos pods:- Warning FailedCreatePodSandBox 7m50s kubelet, darshan-p-hegde-89ca8c531.mylabserver.com Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "5f6770d9dfcb53738a0dd428b86e815d4d85e9b71a76d17b10b1f764f102fb61" network for pod "coredns-576cbf47c7-9r27x": NetworkPlugin cni failed to set up pod "coredns-576cbf47c7-9r27x_kube-system" network: open /run/flannel/subnet.env: no such file or directory |
@hegdedarsh possible that it is a different problem, but I would suggest using a released version https://raw.githubusercontent.com/coreos/flannel/v0.10.0/Documentation/kube-flannel.yml, modify the tolerations and give it a try. |
This fixes the issue for me. Thanks for the PR! |
Adding the toleration in the Flannel yaml works for me also. Tested on v1.12.1 Kubernetes. Thanks. |
I am using the yaml file recommended in this issue. But for me nodePort and "externalIps" doesn't work anymore unless its from the same node that the pods are located on. If i try to telnet via the master ip i get a timeout. Is this a problem with flannel? |
I am on a fresh install of k8s 1.12 and have just tried downloading v0.10 and the tolerations seem to exist already. So I applied the yml
It tried creating the flannel pod but came up with 'Error' and eventually "CrashLoopBackOff". |
just here to say that using https://raw.githubusercontent.com/coreos/flannel/v0.10.0/Documentation/kube-flannel.yml with the toleration's set as the below works on Kubernetes 1.12.3 with kubeadm install:
|
Thanks . it worked for me after applying above changes to flannel config to v1.12.3. |
There hasn't been a release of flannel for a year and we need to upgrade to Kubernetes 1.12. Are there plans to have a new release anytime soon? If not, it's not a problem, we can always branch and fix it ourselves. Thanks |
There is a release planned soon. Can we have a PR that updates kube-flannel.yml with the correct tolerations? |
Thanks! Wasn't it fixed here? 13a990b |
What's currently on master works just fine. But yes a release would be
good. :)
…On Sat, 26 Jan 2019, 01:27 vmendi ***@***.*** wrote:
Thanks!
Wasn't it fix here? 13a990b
<13a990b>
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1044 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AKoi5lm86HZYEaJtGADmS0Ry7cd2179aks5vG67zgaJpZM4W9xBu>
.
|
I can certify that with the latest release v0.11.0, flannel works with kubernetes 1.12.5 out of the box :) |
Yes, although you must know the commit to fetch the fixed manifest. Typically, I obtain the manifest by using the tag, e.g. for v0.10.0, I use https://raw.githubusercontent.com/coreos/flannel/v0.10.0/Documentation/kube-flannel.yml Of course, the manifest does not include the fix, since it is the manifest that existed when v0.10.0 was released. I humbly ask the maintainers to consider making fixes like this easier to find. (In my experience, a common way to make such fixes easy to find is to cherry-pick them to a release branch. I realize the flannel repo does not use release branches. I don't have insight into why that's the case.) |
For anyone who wants to patch the v0.10.0 DaemonSet to tolerate all taints with the
|
I strongly disagree that flannel should tolerate all taints, because there are nodes it should certainly not run on, (e.g. windows nodes). |
Seems like a new behavior with kubeadm, after I created a master, I see two taints on the master node:
But https://raw.githubusercontent.com/coreos/flannel/v0.10.0/Documentation/kube-flannel.yml has toleration only to
I added a toleration to
kube-flannel.yml
to solve the issue:Expected Behavior
The docs should work with flannel out of the box
https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
Current Behavior
Possible Solution
Maybe instead it should use a toleration without a key?
Steps to Reproduce (for bugs)
https://raw.githubusercontent.com/coreos/flannel/v0.10.0/Documentation/kube-flannel.yml
from the docs.Context
Your Environment
v0.10.0
Linux master1 4.4.0-134-generic #160-Ubuntu SMP Wed Aug 15 14:58:00 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
,"Ubuntu 16.04.5 LTS"
The text was updated successfully, but these errors were encountered: