Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flannel cross platform support #4299

Open
wants to merge 9 commits into
base: master
from

Conversation

Projects
None yet
5 participants
@nmiculinic
Copy link
Contributor

commented Feb 25, 2019

  • Work in progress since I have to test resulting manifests (( waiting for the cluster to spin up... and debugging other issues on the fly ))

It's adapted from: https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml

  • CNI installation has been moved from parallel container to initContainer per official kube-flannel.yml template
  • Also the image has been changed to use only the single image, not specific one for flannel CNI (( which does pretty much the same thing ))

Note:
This breaks custom overridden flannel_image_tag unless they follow flannel naming convention (( tag name-arch )). See PR for details. Probably better solution would be multiarch flannel image manifest, but this should do for now.

nmiculinic added some commits Feb 25, 2019

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Feb 25, 2019

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: nmiculinic
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: riverzhang

If they are not already assigned, you can assign the PR to them by writing /assign @riverzhang in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@nmiculinic

This comment has been minimized.

Copy link
Contributor Author

commented Feb 25, 2019

Related to #4294 #4065

@nmiculinic nmiculinic changed the title [WIP] Flannel cross [WIP] Flannel cross platform support Feb 25, 2019

@lwolf

This comment has been minimized.

Copy link

commented Feb 25, 2019

Probably better solution would be multiarch flannel image manifest, but this should do for now.

FYI I have a github+travic-ci job set up to pack official flannel images into the manifest (arm+arm64+amd64).
https://github.com/lwolf/docker-multiarch
https://hub.docker.com/r/lwolf/flannel

@nmiculinic

This comment has been minimized.

Copy link
Contributor Author

commented Feb 25, 2019

Can we integrate it somehow into upstream?

@lwolf

This comment has been minimized.

Copy link

commented Feb 25, 2019

Definitely, if there are no restrictions like - "use only official upstream docker images" or something.
Feel free to test it in your branch

@nmiculinic

This comment has been minimized.

Copy link
Contributor Author

commented Feb 25, 2019

I meant the flannel docker image upstream, but yeah. I'll try it out

@nmiculinic nmiculinic changed the title [WIP] Flannel cross platform support Flannel cross platform support Feb 25, 2019

@nmiculinic

This comment has been minimized.

Copy link
Contributor Author

commented Feb 25, 2019

Removed WIP status, tested in on cluster and it's working

@Miouge1

This comment has been minimized.

Copy link
Member

commented Feb 26, 2019

I ran into similar problem with Calico (See PR #4253). I suppose that Flannel is not distributed on docker.io with multi arch detection?

@nmiculinic

This comment has been minimized.

Copy link
Contributor Author

commented Feb 26, 2019

This is the most official looking image I could find: https://hub.docker.com/r/quayio/coreos-flannel/ ... and it's not updated nor maintained. To the best of my knowledge, flannel isn't distributed on docker.io

@nmiculinic

This comment has been minimized.

Copy link
Contributor Author

commented Feb 28, 2019

What would it take to get this merged? What needs to be changed/added or should I close the PR?

@@ -6,8 +6,15 @@
src: "{{item.file}}.j2"
dest: "{{kube_config_dir}}/{{item.file}}"
with_items:
- {name: flannel, file: cni-flannel-rbac.yml, type: sa}
- {name: kube-flannel, file: cni-flannel.yml, type: ds}
- { name: flannel, file: cni-flannel-rbac.yml, type: sa }

This comment has been minimized.

Copy link
@Miouge1

Miouge1 Mar 7, 2019

Member

We don't need to introduce a space between { and name?

This comment has been minimized.

Copy link
@nmiculinic

nmiculinic Mar 8, 2019

Author Contributor

Fixed

- { name: flannel, file: cni-flannel-rbac.yml, type: sa }
- { name: kube-flannel, file: cni-flannel.yml, type: ds }
vars:
flannel_arches:

This comment has been minimized.

Copy link
@Miouge1

Miouge1 Mar 7, 2019

Member

What do you think about putting this as a group_var?

This comment has been minimized.

Copy link
@nmiculinic

nmiculinic Mar 8, 2019

Author Contributor

Sure, that seems reasonable

---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: kube-flannel
name: kube-flannel-ds-{{ arch }}

This comment has been minimized.

Copy link
@Miouge1

Miouge1 Mar 7, 2019

Member

When upgrading a cluster this can be a problem since we will end up with kube-flannel and kube-flannel-ds-amd64 on the same nodes? Can we do something about this?

This comment has been minimized.

Copy link
@nmiculinic

nmiculinic Mar 8, 2019

Author Contributor

Hmmm...not sure. Options:

(1) leave amd64 without -arch prefix and hope for the best
(2) make breaking change and put it into changelog
(3) put multiarch flannel behind feature flag which is turned off by default.

This comment has been minimized.

Copy link
@rhockenbury

rhockenbury Mar 9, 2019

Contributor

Would there be anything wrong with adding a task to delete the old daemonset (kube-flannel) after verifying the new one is running? It seems that's the approach taken in a number of places in this codebase when upgrading components.

This comment has been minimized.

Copy link
@nmiculinic

nmiculinic Apr 17, 2019

Author Contributor

@rhockenbury can you give me an example where this approach is taken and I shall apply the same trick here?

This comment has been minimized.

This comment has been minimized.

Copy link
@Miouge1

Miouge1 Apr 17, 2019

Member

Another option could be to add the arch prefix only not non amd64

This comment has been minimized.

Copy link
@nmiculinic

nmiculinic Apr 18, 2019

Author Contributor

Ok, I think I added proper cleaning in pre-upgrade stage

nmiculinic added some commits Mar 8, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.