Add Calico v2.5 support for Kubernetes v1.8+ #3623

KashifSaadat · 2017-10-13T14:54:14Z

Added support for Canal (Calico) v2.5.1, which is required to work with Kubernetes v1.8.0+.

Older versions of Calico relied on ThirdPartyResources API to store it's config data, however this is now fully deprecated in Kubernetes v1.8 and has moved over to CustomResourceDefinitions (CRD). Calico v2.5+ has been updated to use CRD, however there is a manual upgrade process involved to migrate the configuration data across: https://github.com/projectcalico/calico/blob/master/upgrade/v2.5/README.md

KashifSaadat · 2017-10-13T15:00:14Z

/assign @chrislovecnm
/assign @justinsb

Some Notes:

I've tested with Kubernetes v1.7.2 and v1.8.0, both successful
I've included the changes from other recent PRs to add configurable options into the Canal manifest file (enabling Prometheus metrics, chain insert mode, etc)
On raising this PR I noticed there are failing integration and bootstrapchannelbuilder tests related to a recent Weave upgrade (PR Update Weave Net to version 2.0.5 #3614). I'm not sure how this has happened as the Travis CI tests should have picked this up and failed? I was looking to fix the tests in this PR but got a bit puzzled as it's failing to find a weave manifest for K8s v1.7, but the file exists. Any ideas?
Attempted to update the Calico networking manifest also but couldn't get it working. Would it be ok to raise an issue for this and add support separately?
We will need to include within the release notes about migration steps to Kubernetes v1.8+ when using Canal/Calico networking. Is there anywhere in the repo that you'd like me to update for a reference to this document: https://github.com/gunjan5/calico/blob/master/upgrade/v2.5/README.md

chrislovecnm · 2017-10-13T18:14:08Z

On raising this PR I noticed there are failing integration and bootstrapchannelbuilder tests related to a recent Weave upgrade (PR #3614). I'm not sure how this has happened as the Travis CI tests should have picked this up and failed? I was looking to fix the tests in this PR but got a bit puzzled as it's failing to find a weave manifest for K8s v1.7, but the file exists. Any ideas?

Ick ... checking now

chrislovecnm · 2017-10-13T18:16:55Z

@KashifSaadat which test?

KashifSaadat · 2017-10-13T18:27:32Z

Sorry ignore that!! It seems my git checkout must have screwed up a little, I had the bootstrapchannelbuilder changes but I think it was actually missing the 1.7 manifest file somehow. I've cleared up and done a fresh checkout and the tests are passing now. 👍

chrislovecnm · 2017-10-13T18:33:29Z

I am guessing that the makefile did not regenerate go-bindata :)

KashifSaadat · 2017-10-13T18:34:40Z

Ah yes that could have been it!

chrislovecnm

Question about resources. I think we want QOS on these pods guaranteed :)

chrislovecnm · 2017-10-13T19:08:54Z

upup/models/cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.8.yaml.template

+          securityContext:
+            privileged: true
+          resources:
+            requests:


did we want to set limits and request the same to get QOS?

Sure that sounds sensible, will update.

chrislovecnm · 2017-10-13T19:09:30Z

upup/models/cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.8.yaml.template

+              name: cni-net-dir
+        # This container runs flannel using the kube-subnet-mgr backend
+        # for allocating subnets.
+        - name: kube-flannel


missing resources section

This wasn't defined in the existing canal templates. I'm not sure exactly what the limits should be set to here, so I'll use what was defined for the flannel networking templates.

May want to talk with @justinsb about it. Can we file an issue and revisit?

The canal / calico guys should tell us what to use ;)

chrislovecnm · 2017-10-13T19:10:25Z

upup/models/cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.8.yaml.template

+
+# Calico Roles
+# Pulled from https://docs.projectcalico.org/v2.5/getting-started/kubernetes/installation/hosted/rbac-kdd.yaml
+kind: ClusterRole


@caseydavenport / @liggitt you mind reviewing?

chrislovecnm · 2017-10-13T19:12:17Z

Attempted to update the Calico networking manifest also but couldn't get it working. Would it be ok to raise an issue for this and add support separately?

Just push another WIP PR, we can ping @caseydavenport and his amigos :)

chrislovecnm · 2017-10-14T05:45:19Z

Fixes #3352

liggitt · 2017-10-14T13:16:26Z

upup/models/cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.8.yaml.template

+    resources:
+      - pods/status
+    verbs:
+      - update


Should probably be using patch to avoid dropping unknown fields (see flannel use of patch for nodes/status)

As Casey mentioned I'm unable to change this currently and would need to wait for it to be resolved upstream in Calico. Tested just to make sure and my deployments were failing.

liggitt · 2017-10-14T13:16:51Z

upup/models/cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.8.yaml.template

+    verbs:
+      - get
+      - list
+      - update


Calico updates node specs?

Yeah it does (though like you mention for pods/status it should really be using patch - there's an open issue against Calico to do that).

Calico stores some of its per-node configuration, etc. as annotations on the node.

KashifSaadat · 2017-10-16T11:26:36Z

Should be ready for another review now :)

chrislovecnm · 2017-10-17T02:19:03Z

/lgtm

k8s-github-robot · 2017-10-17T02:19:44Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chrislovecnm

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

~~OWNERS~~ [chrislovecnm]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

k8s-github-robot · 2017-10-17T02:20:06Z

/test all [submit-queue is verifying that this PR is safe to merge]

k8s-github-robot · 2017-10-17T02:51:08Z

Automatic merge from submit-queue.

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 13, 2017

k8s-ci-robot assigned chrislovecnm and justinsb Oct 13, 2017

chrislovecnm reviewed Oct 13, 2017

View reviewed changes

liggitt reviewed Oct 14, 2017

View reviewed changes

Add Calico v2.5 support for Kubernetes v1.8+

6d624c7

KashifSaadat force-pushed the canal-v2.5 branch from 769b99f to 6d624c7 Compare October 16, 2017 11:24

chrislovecnm approved these changes Oct 17, 2017

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 17, 2017

k8s-github-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 17, 2017

k8s-github-robot merged commit 1d5b281 into kubernetes:master Oct 17, 2017

KashifSaadat mentioned this pull request Nov 1, 2017

Canal should be updated to use Calico v2.5 #3352

Closed

while1eq1 mentioned this pull request Aug 21, 2019

Update weave to 2.5.2 #7444

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Calico v2.5 support for Kubernetes v1.8+ #3623

Add Calico v2.5 support for Kubernetes v1.8+ #3623

KashifSaadat commented Oct 13, 2017 •

edited

Loading

KashifSaadat commented Oct 13, 2017 •

edited

Loading

chrislovecnm commented Oct 13, 2017 •

edited

Loading

chrislovecnm commented Oct 13, 2017

KashifSaadat commented Oct 13, 2017

chrislovecnm commented Oct 13, 2017

KashifSaadat commented Oct 13, 2017

chrislovecnm left a comment

chrislovecnm Oct 13, 2017

KashifSaadat Oct 16, 2017

chrislovecnm Oct 13, 2017

KashifSaadat Oct 16, 2017

chrislovecnm Oct 17, 2017

chrislovecnm Oct 17, 2017

chrislovecnm Oct 13, 2017

chrislovecnm commented Oct 13, 2017

chrislovecnm commented Oct 14, 2017

liggitt Oct 14, 2017

KashifSaadat Oct 16, 2017

liggitt Oct 14, 2017

caseydavenport Oct 15, 2017

KashifSaadat commented Oct 16, 2017

chrislovecnm commented Oct 17, 2017

k8s-github-robot commented Oct 17, 2017

k8s-github-robot commented Oct 17, 2017

k8s-github-robot commented Oct 17, 2017

Add Calico v2.5 support for Kubernetes v1.8+ #3623

Add Calico v2.5 support for Kubernetes v1.8+ #3623

Conversation

KashifSaadat commented Oct 13, 2017 • edited Loading

KashifSaadat commented Oct 13, 2017 • edited Loading

chrislovecnm commented Oct 13, 2017 • edited Loading

chrislovecnm commented Oct 13, 2017

KashifSaadat commented Oct 13, 2017

chrislovecnm commented Oct 13, 2017

KashifSaadat commented Oct 13, 2017

chrislovecnm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chrislovecnm commented Oct 13, 2017

chrislovecnm commented Oct 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KashifSaadat commented Oct 16, 2017

chrislovecnm commented Oct 17, 2017

k8s-github-robot commented Oct 17, 2017

k8s-github-robot commented Oct 17, 2017

k8s-github-robot commented Oct 17, 2017

KashifSaadat commented Oct 13, 2017 •

edited

Loading

KashifSaadat commented Oct 13, 2017 •

edited

Loading

chrislovecnm commented Oct 13, 2017 •

edited

Loading