Enable CoreDNS as a DNS plugin for Kubernetes

[johnbelamaric]

Feature Description

• One-line feature description (can be used as a release note): Enable CoreDNS as a DNS plugin for Kubernetes
• Primary contact (assignee): @johnbelamaric
• Responsible SIGs: sig-network, sig-cluster-lifecycle
• Design proposal link (community repo): kubernetes/community#1100
• Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred: @bowei @thockin
• Approver (likely from SIG/area to which feature belongs): @thockin
• Feature target (which target equals to which milestone):
  • Alpha release target (1.9)
  • Beta release target (1.10)
  • Stable release target (1.11)

[johnbelamaric]

cc @luxas @jbeda @mattmoyer @miekg

[cmluciano]

@kubernetes/sig-network-feature-requests

[klausenbusk]

What is the argument behind this switch?

[johnbelamaric]

Hi Kristian,

Here's a little background. CoreDNS is another CNCF project and is the successor to SkyDNS, which kube-dns is based on. It is a flexible, extensible authoritative DNS server and we have built a direct integration to the Kubernetes API. It can serve as cluster DNS, complying with the dns spec. We started discussions with @thockin, @matchstick and @bowei last year around KubeCon and they are open to it in principle, but of course it needs to be proven to be the right choice.

As for reasons to switch, CoreDNS has fewer moving parts than kube-dns, since it is a single executable and single process. It is written in Go so it is memory-safe (kube-dns includes dnsmasq which is not). It supports a number of use cases that kube-dns doesn't. As a general-purpose authoritative DNS server it has a lot of functionality that kube-dns couldn't reasonably be expected to add.

You may also want to check out the intro or coredns.io. There is also a webinar I did for CNCF a couple months back.

[smarterclayton]

I would expect a proposal for this.

[johnbelamaric]

Ok, will do. I assume that means based on https://github.com/kubernetes/community/blob/master/contributors/design-proposals/Design_Proposal_TEMPLATE.md

[johnbelamaric]

Ok, here is the proposal. Sorry for all those commits referencing this.

kubernetes/community#1100

[kargakis]

Not sure this was meant to close

/reopen

[johnbelamaric]

This has changed now, we need to split into two features: CoreDNS available as an option, and CoreDNS as default. Can we just rename the issue and create a new one for making it default (targeting 1.12)?

[justaugustus]

@johnbelamaric / @cmluciano -- I've renamed the description of this feature and created a new one (#566) around enabling CoreDNS as the default DNS plugin. PTAL and let me know if there are any details I need to update on that feature issue.

[johnbelamaric]

@justaugustus thanks!

[idvoretskyi]

@justaugustus what's the difference between the current one and #566?

[johnbelamaric]

@idvoretskyi the current one does not enable it by default. It just declares it a GA feature that can be enabled. Enabling it by default (on new installs and on upgrades) will come later.

[idvoretskyi]

@johnbelamaric it's clear, thanks.

[luxas]

I'm not sure two feature issues are needed for that though. If CoreDNS is GA in v1.11 it's up to every deployment tool (kubeadm, kops, GKE, AKS, EKS, etc.) to enable it by default in their own pace. You can't dictate a default for an option for all of the installer options, as some of them are unreachable to the OSS world. I'd go with just this one. If you want to track the enabling of CoreDNS as a feature for a specific deployment like GKE, then you might optionally do that in a separate issue.

Does that make sense to you?

[johnbelamaric]

@luxas I see your point. However, don't we need to provide a recommendation at some point as to which of the options to use? What if we want to deprecate kube-dns at some point?

This "replace a component" is new territory, so we're trying to figure out the best process.

[luxas]

When CoreDNS is GA, I'd expect it to be the default for new clusters as it's the new thing.
That's the implicit recommendation from the feature status for me at least.
If CoreDNS is ready to technically be default in all new clusters in v1.11, I'd graduate it to GA now and deprecate kube-dns. kube-dns would be supported as an option for backwards-compability for a year (approx. four releases; until v1.15) though, as of GA: 1 year or 2 releases (whichever is longer) in https://kubernetes.io/docs/reference/deprecation-policy/

[johnbelamaric]

@luxas I expect there will be additional features (if there aren't already) in which you choose between several GA options that provide similar functionality. For example, whether to use iptables or ipvs for kube-proxy. I think we need a model for managing those situations.

[AishSundar]

@johnbelamaric I am the CI Signal lead for 1.11 and also work on Conformance testing program for K8s. I see this feature is going to Stable in 1.11.
I also see you have some e2e test changes for this feature. I am following up to see if and which of those tests should we promote to the conformance suite in 1.11.
As part of the process to increase conformance coverage, outlined by Conformance WG and Sig-Arch, we expect features going into stable/GA to have representation in Conformance suite. Your update on the same will help us evaluate this feature better.

[smarterclayton]

I would expect the existing DNS conformance tests to cover this. I would not expect use of coredns to be required to be conformant. All that said - are our dns conformance tests sufficient in scope to cover the dns minimum requirements?

[fturib]

@AishSundar : CoreDNS is replacing kube-dns. As such we ensured already that CoreDNS is conform to existing DNS Conformance test.
We have a suite test for CoreDNS running here : http://k8s-testgrid.appspot.com/sig-network-gce#gci-gce-coredns (that is larger than Conformance)

As such, CoreDNS has already a representation in Conformance suite (the same as kube-dns, the feature it replaces).

We also ensured that CoreDNS is compliant to all DNS test running in e2e. that is why we have some e2e test changes. But those are not part of the conformance suite.

Adapt existing test to CoreDNS, verifying the configuration by Configmap change : kubernetes/kubernetes#63265

Create a new test for DNS, by adding a specific DNS related test on scalability (asked by sig-networking) : kubernetes/kubernetes#63820

Extends all DNS tests for IPv6 use case : kubernetes/kubernetes#59894

[johnbelamaric]

@smarterclayton CoreDNS is conformant in the default configuration.

[smarterclayton]

Thanks

…

On Thu, May 24, 2018 at 9:04 AM, John Belamaric ***@***.***> wrote: @smarterclayton <https://github.com/smarterclayton> CoreDNS is conformant in the default configuration. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#427 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABG_p5HuTwCmZPQTctXv2FUS15nuzKC6ks5t1q_bgaJpZM4PU9YV> .

[AishSundar]

Thanks @fturib for the clarification and PR pointers here

[justaugustus]

@johnbelamaric --
We're doing one more sweep of the 1.11 Features tracking spreadsheet.
Would you mind filling in any incomplete / blank fields for this feature's line item?

[johnbelamaric]

@justaugustus I updated it to "Draft". We have several PRs in various states of merged and under review for the docs. There is already a link in there to the tracking issue.

[justaugustus]

@johnbelamaric thanks for the update!
Looks like everything is fine on that tracking issue. My only note would be that that tracking issue would be better suited as an issue in k/k or k/website, not the Kubeadm project.

[justaugustus]

Closing this as the feature is GA in 1.11. Please feel free to reopen if there is still a need to track this.
/close