KubeDNS ConfigMap Addon was updated upstream, which breaks against previous releases of CDK

[chuckbutler]

During a deployment, the kubedns maps are working with whats in teh charm store, however the addon template apparently landed a change which has broken when building from source

error: error validating "/etc/kubernetes/addons/kubedns-controller.yaml": error validating data: found invalid field optional for v1.ConfigMapVolumeSource; if you choose to ignore these errors, turn validation off with --validate=false

I know this was pending an actual re-publish of the container, but that's not whats listed as broken here, its the presence of the config-map being listed as optional. We've either botched a translation or the template is broken upstream, and we should really fix this before the next CDK rev.

[Cynerva]

Thanks @chuckbutler. Here's the upstream PR that made this change: kubernetes/kubernetes#40382

The optional field for configmaps was implemented recently: kubernetes/kubernetes#39981

So, this is a versioning problem. We're pulling in a fancy new addon definition before we have the kubernetes binaries to support it.

This might be a good argument for packaging the addons as a resource, which we can update independently, rather than pulling them into the charm at build time.

An alternative would be to give UpdateAddonsTactic the option to checkout a non-master branch, though I'm not a big fan of that. Seems like it'd be an easy option to miss, and we won't know the charm is broken until we try to deploy it. Yeah, I'm leaning toward taking out the tactic and making a resource.

[Cynerva]

This issue causes a hook failure, by the way, so it's a blocker for kubernetes-master development. @chuckbutler @mbruzek @wwwtyro

In case somebody needs a quick fix, see Cynerva/kubernetes@0071f7a which disables UpdateAddonsTactic and adds in usable addon definitions directly.

[Cynerva]

Here's a branch that removes UpdateAddonsTactic and ships addons as a resource instead: Cynerva:gkk/addons-resource

More work would be needed to create a jenkins job for the addons resource, but I should be able to knock that out quickly if we decide this is the way to go.

@chuckbutler and @mbruzek I'd like to hear your opinions before I take any further action on this issue. Open to ideas!

[chuckbutler]

@Cynerva i don't know that we're ready to abandon the add-on tactic just yet...

I think this is simply a case of having landed a broken change in master, and we need to raise awareness that this is a broken change and we should then see if we can't press for having a proper fix vetted in k8s/k8s rather than change our build routine because an add-on manifest was updated.

I imagine if this is broken for us, its broken for everyone

We also have an opportunity to re-build the bins for k8s and stuff them in the edge channel for use with this new addon template... which would explain why its broken for us, but not in tip...

[Cynerva]

Ha okay, apologies if I was too gung-ho about this :)

I think this is simply a case of having landed a broken change in master, and we need to raise awareness that this is a broken change and we should then see if we can't press for having a proper fix vetted in k8s/k8s rather than change our build routine because an add-on manifest was updated.

I disagree. The new addon is broken against the 1.5.2 release, but I don't think it's broken against kubernetes/master builds (though I haven't tested this). I don't think it'd be fair of us to ask for cluster/addons on master to always support the latest stable release - it's a development branch, after all.

Perhaps I'm making too many assumptions about a discussion that hasn't occurred yet, but I figure they'll just suggest we grab from an earlier branch. At which point, we'll need more control than UpdateAddonsTactic gives us currently.

I'd like to strive for that level of control anyway, to be honest. It doesn't sit well with me that kubernetes-master development is blocked without hacking on the code to make it work.

[chuckbutler]

His basically means to me that we should be hacking on the next tagged release or be building against the non 1.5.2 release as that's already cut. Our tests passed it and we should be working on what's next

…

On Feb 8, 2017, at 10:56 AM, Cynerva ***@***.***> wrote: Ha okay, apologies if I was too gung-ho about this :) I think this is simply a case of having landed a broken change in master, and we need to raise awareness that this is a broken change and we should then see if we can't press for having a proper fix vetted in k8s/k8s rather than change our build routine because an add-on manifest was updated. I disagree. The new addon is broken against the 1.5.2 release, but I don't think it's broken against kubernetes/master builds (though I haven't tested this). I don't it'd be fair of us to ask for cluster/addons on master to always support the latest stable release - it's a development branch, after all. Perhaps I'm making too many assumptions about a discussion that hasn't occurred yet, but I figure they'll just suggest we grab from an earlier branch. At which point, we'll need more control than UpdateAddonsTactic gives us currently. I'd like to strive for that level of control anyway, to be honest. It doesn't sit well with me that kubernetes-master development is blocked without hacking on the code to make it work. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[chuckbutler]

This was fixed with 41251. If you need to set a specific tag for addons, export KUBE_VERSION before issuing the build.