Fix Flannel nonMasqueradeCIDR

[mikesplain]

Fixes #3950

And looks like we forgot to bump the version in bootstrapchannelbuilder.go(moved to #3953) (thanks @jkemp101)

[chrislovecnm]

@mikesplain so this needs to be two PRs. First of this can cause breaking changes with upgrades using nonMasqCIDR properly.

We are going to need boostrapchannelbuilder.go bumped for 1.8.0, if we can, and we need to figure out migration for the CIDR.

[chrislovecnm]

/hold

As we chatted about - how do we upgrade flannel between cidrs

[KashifSaadat]

Hey @chrislovecnm, we should probably get this in pretty soon.

I'm not sure this would have ever worked correctly unless a user's Cluster Spec had the samenonMasqueradeCIDR defined, because nodes fail to provision otherwise (clusterDNS is set incorrectly, causing DNS lookups to fail and you get the placeholder address 203.0.113.123 in logs).

[chrislovecnm]

How does a migration work between cidrs with flannel? Before we get this in we need to figure out how to transition users.

[KashifSaadat]

I'm not sure whether there would be a migration for existing users on flannel. Might be wrong but I think these are the current two scenarios for existing users:

• Kops ClusterSpec configured with different nonMasqueradeCIDR: Provisioning cluster will fail because clusterDNS etc will be set to addresses different to what flannel would provision (based on the value hardcoded in the manifest file) - I had failures on the kubelet services across nodes, failing to resolve
• Kops ClusterSpec configured with nonMasqueradeCIDR: 100.64.0.0/10 (matching manifest file): Cluster would be in working condition and the PR should have no impact as the addresses will stay the same

So in the second scenario there wouldn't be any migration to perform, or am I missing something here? I don't know much about flannel :(

[justinsb]

Ah - good point @KashifSaadat . So nobody could have a working cluster anyway with flannel + a different non-masquerade cidr.

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: justinsb

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

• OWNERS [justinsb]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

[justinsb]

Send a bootstrapchannelbuilder bump in #4064

[chrislovecnm]

/hold cancel

[k8s-github-robot]

/test all [submit-queue is verifying that this PR is safe to merge]

[k8s-github-robot]

Automatic merge from submit-queue.