Switch default DNS plugin to CoreDNS

[justaugustus]

Feature Description

• One-line feature description (can be used as a release note): Switch default DNS plugin to CoreDNS
• Primary contact (assignee): @johnbelamaric
• Responsible SIGs: sig-network, sig-cluster-lifecycle
• Design proposal link (community repo): Add coredns proposal community#1100
• Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred: @bowei @thockin
• Approver (likely from SIG/area to which feature belongs): @thockin
• Feature target (which target equals to which milestone):
  • Stable release target: 1.12

xref: #427

/kind feature
/sig cluster-lifecycle
/sig network
/assign @johnbelamaric

[k8s-ci-robot]

@justaugustus: GitHub didn't allow me to assign the following users: johnbelamaric.

Note that only kubernetes members and repo collaborators can be assigned.

In response to this:

Feature Description

• One-line feature description (can be used as a release note): Switch default DNS plugin to CoreDNS
• Primary contact (assignee): @johnbelamaric
• Responsible SIGs: sig-network, sig-cluster-lifecycle
• Design proposal link (community repo): Add coredns proposal community#1100
• Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred: @bowei @thockin
• Approver (likely from SIG/area to which feature belongs): @thockin
• Feature target (which target equals to which milestone):
• Alpha release target: 1.12

/kind feature
/sig cluster-lifecycle
/sig network
/assign @johnbelamaric

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[luxas]

Hi, thanks for filing this feature!
I have a couple of questions/concerns though.
Why is this marked alpha? It sounds more like it should be stable/GA when it's a default.
I don't think this even needs to be a separate features issue. The common practice is to use one issue (in this case #427) to track the full lifecycle.
If your goal is to have CoreDNS as the default DNS plugin in v1.12, I'd just adjust #427 to be "stable/GA target: v1.12"

Cheers 😄

[stp-ip]

The reasoning behind the split was, that in order to make it default later on the actual plugin needs to be stable/GA already. That was the prerequisite after a bigger discussion. The result was the split into two feature issues and making the default alpha first. (Alpha might not be the best thing so.)

[stp-ip]

Reading all your other comments on the other thread (#427 (comment)) your comment is much clearer in my head. As CoreDNS is such a central place it make still make sense to track the state of being the default.

[justaugustus]

@johnbelamaric @cmluciano --

It looks like this feature is currently in the Kubernetes 1.12 Milestone.

If that is still accurate, please ensure that this issue is up-to-date with ALL of the following information:

• One-line feature description (can be used as a release note):
• Primary contact (assignee):
• Responsible SIGs:
• Design proposal link (community repo):
• Link to e2e and/or unit tests:
• Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred:
• Approver (likely from SIG/area to which feature belongs):
• Feature target (which target equals to which milestone):
  • Alpha release target (x.y)
  • Beta release target (x.y)
  • Stable release target (x.y)

Set the following:

• Description
• Assignee(s)
• Labels:
  • stage/{alpha,beta,stable}
  • sig/*
  • kind/feature

Please note that the Features Freeze is July 31st, after which any incomplete Feature issues will require an Exception request to be accepted into the milestone.

In addition, please be aware of the following relevant deadlines:

• Docs deadline (open placeholder PRs): 8/21
• Test case freeze: 8/28

Please make sure all PRs for features have relevant release notes included as well.

Happy shipping!

/cc @justaugustus @kacole2 @robertsandoval @rajendar38

[luxas]

I assume this is beta in v1.12? kubeadm has used it on a GA/beta level already since v1.11 fwiw

[justaugustus]

@johnbelamaric / @cmluciano / @luxas / @thockin --
So this is a case where the "graduation" of the feature is actually flipping the bit for it to be the default DNS plugin.
As CoreDNS is already GA, what should we do here?

• Straight to GA?
• Set to Beta to soak for a release and then graduate to GA?
• Something else entirely?

/assign @johnbelamaric
/remove-stage alpha

[stp-ip]

As the point for switching the flip was mainly to separate GA of CoreDNS and the actual config switch it seems like moving to beta is another step, that might not be necessary especially considering it's already GA/beta in kubeadm.

Should go GA in v1.12 in my view fwiw.

[justaugustus]

Just to clarify for passerbys...
Adding CoreDNS as a DNS plugin was tracked on #427. There is a process for moving features through stages (alpha --> beta --> stable), which again was tracked on #427.

This Feature issue is unique in the fact that here, we're not actually tracking the development state of the feature (as it's already been developed), but instead we're tracking the process of deciding in which release it should be enabled as the default DNS plugin. AFAIK, we don't have an official graduation process for "flipping a bit".

While developing kubeadm is a separate concern, @stp-ip is right; this has already had some soak time as CoreDNS was selected as the default DNS plugin for kubeadm in Kubernetes 1.11.

FWIW, I would also vote to mark this as GA for 1.12, but I'll wait for the powers that be (@kubernetes/sig-network-misc) to make a call on that.

[chrisohaver]

How does this process relate to the KEP process?

The KEP for graduating CoreDNS to default DNS server, is here ...
https://github.com/kubernetes/community/blob/master/keps/sig-network/0012-20180518-coredns-default-proposal.md

[cmluciano]

I agree that switching to default based on kubeadm results is great progress. Do we have any preliminary results from cloud-providers or other large deploys on stability.

It may be a "flipping in bit" in docs, but often features are not used in k8s until they reach beta stage.

[justaugustus]

@johnbelamaric @cmluciano -- we still need to make a call on this. I'm tentatively setting this to GA on the sheet, but let me know if you feel differently.

/stage stable
cc: @kacole2 @wadadli @robertsandoval @rajendar38

[chrisohaver]

The KEP lists the following criteria for graduating coredns to "default"...

• Add CoreDNS image in a Kubernetes repository (To Be Defined) and ensure a workflow/process to update the CoreDNS versions in the future.
• Have a certain number (To Be Defined) of clusters of significant size (To Be Defined) adopting and running CoreDNS as their default DNS.

The first is, I believe, satisfied.
For the second, we (CoreDNS team + CNCF) are conducting a survey of k8s+coredns users to gauge coredns adoption in large clusters. Just pushed it out recently. We do not have survey results yet.

[fturib]

@justaugustus : I try to push an email on this subject with Tim, John, I will add @cmluciano .. but I cannot find any email for yourself. Can your provide one ? Thanks !

[fturib]

for info, the survey for feedback on CoreDNS adoption is available here : https://www.surveymonkey.com/r/SKZQSLK

[johnbelamaric]

Going GA with this feature means deprecating kube-dns, I think we need to wait until 1.13 at the earliest for this, to gather enough data.

[chrisohaver]

I’m looking into the auto scale and debugging articles.

[johnbelamaric]

@justaugustus I think we can wait a cycle or so to deprecate kube-dns. Originally I was pairing the two but that's not strictly necessary.

[justaugustus]

@justaugustus I think we can wait a cycle or so to deprecate kube-dns. Originally I was pairing the two but that's not strictly necessary.

@johnbelamaric -- Got it. Noting again that we need to coalesce a plan for kube-dns deprecation sooner rather than later, as it's a multi-release event.

[tfogo]

Hi @rajansandeep, @johnbelamaric,

How are docs looking for this? Is there a PR open in k/website?

[rajansandeep]

I forgot to refer the PR for this.
kubernetes/website#10228 is ready for review.

[chrisohaver]

... and kubernetes/website#10200 and kubernetes/website#10201.

[tpepper]

warning this is now at risk, see:

kubernetes/kubernetes#68629
kubernetes/kubernetes#68613

[kacole2]

/milestone v1.13

[AishSundar]

@bowei @chrisohaver can you plz update this issue with your level of confidence, whats pending in terms of PR, test and docs for this feature to wrap in 1.13? Considering the last minute Scalability and memory constraints we ran into last cycle, it will be great to land the open PRs sooner in this cycle to we have enough time to watch the CI signals and stabilize.

Currently Code Slush for 1.13 is Nov 9th and Freeze starts on Nov 15th. Thanks

[fturib]

@AishSundar : We have been investigating the scale issue and memory constraint. We found some optimization and know now we can make it. However we need to wrap up the changes and publish a new release of CoreDNS.
I cannot provide a date yet, but I expect to be by Oct 15th.

We then need:

• re-publish the PR for CoreDNS as default, with the new image. (pretty simple - no delay)
• e2e test will validate after that merge. (2 PR already presented - see below)
• about doc, I think all docs expected were finally merged in 1.12 (because still valid whatever CoreDNS is default or not for GCE). We may need to have another look.

I agree completely on the need to have this feature checked-in asap. We had a sync-up with @thockin from SIG-Network last Tuesday on this subject.
Thanks to remind the code slush date. I plan to be much ahead of these dates.

PR needed for validation e2e tests at scale:

• Add option to track dns pods kubernetes#69283 (already merged)
• Track DNS pod resource usage in scalability tests test-infra#9445

[AishSundar]

Thanks @fturib for the detailed status and good to know that we have a path forward. I will check on the status again post Oct mid.

[tfogo]

Hi @fturib, I know we had a bunch of docs land for this in 1.12, but I just wanted to check if there be any updates to the docs necessary for 1.13? The deadline for placeholder PRs for the 1.13 release is November 8. So it's important to make a docs PR as soon as possible if needed.

[AishSundar]

@fturib @chrisohaver @wojtek-t are there any specific docs / release notes to callout any metrics / resource changes for CoreDNS

@marpaia as FYI for release notes

[fturib]

@AishSundar, @tfogo : let me sync-up on this topic with @chrisohaver, @rajansandeep and come later today or tomorrow on any need for the doc for this feature. Thanks!

[fturib]

We have few updates for some docs. Only minor changes and a pointer for a documentation page create by @chrisohaver for tuning CoreDNS when resources are limited on the cluster.

There is no resource change for CoreDNS.

@rajansandeep is preparing a PR for these changes in kubernetes/website.

[AishSundar]

Thanks @fturib for the confirmation. @rajansandeep once you have the doc PRs plz link them here as well. Thanks.

[rajansandeep]

@AishSundar The PR is here. kubernetes/website#10923

[AishSundar]

@tfogo

[luxas]

v1.13 is now released, and this is hence fully implemented 🎉