Make CoreDNS the default DNS server

[rajansandeep]

This will install CoreDNS as the default DNS server instead of kube-dns

Fixes #6500

[rajansandeep]

This minimum change should do the trick.
I hope I'm not missing something obvious!

[zetaab]

seems okay for me, please fix tests

[zetaab]

Thanks @rajansandeep
/lgtm

[hakman]

/test pull-kops-verify-staticcheck

[rifelpet]

@justinsb can probably weigh in more here, but typically with changes like this we try to avoid modifying existing k8s clusters that arent also upgrading their k8s version. This way if someone updates Kops and runs update cluster without changing anything in their manifest, there should be minimal changes reported.

To make this kind of change we do one or both of two things:

1. Update the default for newly created clusters only. This means that kops create cluster would populate the provider field with CoreDNS but update cluster wouldn't change anything.
2. Update the default for newer k8s versions only. If we were targeting this change for Kops 1.17, we could have a check that only defaults to CoreDNS if the cluster's k8s version is >= 1.17. This makes the change more explicit since it is tied to a k8s version.

Thoughts?

[rajansandeep]

For a start, I've rebased the PR.
I'll make further changes after we decide on the proposal made by @rifelpet

[rifelpet]

We discussed this during office hours yesterday and determined that it will be best to default to CoreDNS only for clusters >= kubernetes 1.18. This way way upgrading kops by itself will not change the default, only upgrading kubernetes itself.

Heres an example of how you could get that information:

kops/upup/pkg/fi/cloudup/apply_cluster.go

Lines 282 to 289 in e29a04e

	kv, err := k8sversion.Parse(cluster.Spec.KubernetesVersion)
	if err != nil {
	return err
	}
	// check if we should recommend turning off anonymousAuth on k8s versions gte than 1.10
	// we do 1.10 since this is a really critical issues and 1.10 has it
	if kv.IsGTE("1.10") {

[rajansandeep]

@rifelpet okay, I'll update the PR to reflect these changes.

[olemarkus]

How is kv, err := k8sversion.Parse(cluster.Spec.KubernetesVersion); kv.IsGTE vs b.cluster.IsKubernetesGTE?
I tend to prefer the latter.

[rifelpet]

I'm wondering about the upgrade path for this. Given the note in the docs regarding switching from KubeDNS to CoreDNS, and to my knowledge kops doesnt have a simple way of removing manifests and knowing when it is safe to remove the KubeDNS deployment, we might need to resort to having a Required Action item in the 1.18 release notes.

[olemarkus]

Any updates on this? Would be nice to get this in soon.

[rajansandeep]

Sorry @rifelpet had to push a change due to a small nit.

[rifelpet]

/lgtm
/assign @justinsb

[rajansandeep]

Any updates?

[rdrgmnzs]

From the Kops office hours 05/08 we decided to wait to make CoreDNS the default DNS until Kops 1.19 and to introduce node-local-dns along with it to help avoid some of the iptables DNAT issues that can pop up.

[olemarkus]

By introducing node-local-dns we mean make it enabled by default?

[k8s-ci-robot]

@rajansandeep: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
pull-kops-verify-cloudformation	bb1fb76	link	/test pull-kops-verify-cloudformation
pull-kops-verify-hashes	bb1fb76	link	/test pull-kops-verify-hashes

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rajansandeep, zetaab

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [zetaab]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hakman]

Did a quick rebase in case anyone feels that this should be merged into 1.19.

[olemarkus]

It should definitely make it into 1.20.

/lgtm