New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support running a nodelocal dns cache #70555

Merged
merged 2 commits into from Nov 14, 2018

Conversation

@prameshj
Copy link
Contributor

prameshj commented Nov 1, 2018

This change includes the yaml files and gce startup script changes
to run this addon. It is disabled by default, can be enabled by setting
KUBE_ENABLE_NODELOCAL_DNS=true
An ip address is required for the cache instance to listen for
requests on, default is a link local ip address of value 169.254.25.10

Cluster with nodelocaldns running can be created using:
KUBE_ENABLE_NODELOCAL_DNS=true go run hack/e2e.go -v --up

What type of PR is this?

Uncomment only one, leave it on its own line:

/kind api-change
/kind bug
/kind cleanup
/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

What this PR does / why we need it:
Implementation of https://github.com/kubernetes/community/blob/master/keps/sig-network/0030-nodelocal-dns-cache.md
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Introduces support for running a nodelocal dns cache. It is disabled by default, can be enabled by setting KUBE_ENABLE_NODELOCAL_DNS=true
An ip address is required for the cache instance to listen for requests on, default is a link local ip address of value 169.254.20.10
@MrHohn

This comment has been minimized.

Copy link
Member

MrHohn commented Nov 1, 2018

/ok-to-test

@prameshj prameshj force-pushed the prameshj:nodelocaldnscache branch from eac47d0 to 98069af Nov 2, 2018

@prameshj

This comment has been minimized.

Copy link
Contributor

prameshj commented Nov 2, 2018

/retest

@neolit123

This comment has been minimized.

Copy link
Member

neolit123 commented Nov 2, 2018

/kind feature
/priority important-longterm
/sig network

@prameshj prameshj force-pushed the prameshj:nodelocaldnscache branch from 98069af to 356aa68 Nov 2, 2018

@prameshj

This comment has been minimized.

Copy link
Contributor

prameshj commented Nov 2, 2018

/assign @bowei

@timothysc

This comment has been minimized.

Copy link
Member

timothysc commented Nov 5, 2018

/hold

@roberthbailey - google needs to make certain that these changes are more broadly disseminated and outlined as a default for every deployment.

@prameshj - is there a proposal on this?

@prameshj

This comment has been minimized.

Copy link
Contributor

prameshj commented Nov 13, 2018

/assign @roberthbailey

@roberthbailey
Copy link
Member

roberthbailey left a comment

/lgtm
/approve

Show resolved Hide resolved cluster/centos/util.sh Outdated
Show resolved Hide resolved cluster/addons/dns/nodelocaldns/Makefile Outdated
Show resolved Hide resolved cluster/addons/dns/nodelocaldns/nodelocaldns.yaml.base Outdated
Show resolved Hide resolved cluster/addons/dns/nodelocaldns/nodelocaldns.yaml.in Outdated
Show resolved Hide resolved cluster/addons/dns/nodelocaldns/nodelocaldns.yaml.in Outdated
Show resolved Hide resolved cluster/addons/dns/nodelocaldns/nodelocaldns.yaml.in Outdated
Show resolved Hide resolved cluster/centos/deployAddons.sh Outdated
Show resolved Hide resolved hack/local-up-cluster.sh Outdated
Show resolved Hide resolved hack/local-up-cluster.sh Outdated
Show resolved Hide resolved hack/local-up-cluster.sh Outdated
Show resolved Hide resolved cluster/gce/gci/configure-helper.sh Outdated

@mikedanese mikedanese removed the lgtm label Nov 13, 2018

@mikedanese

This comment has been minimized.

Copy link
Member

mikedanese commented Nov 13, 2018

My biggest concern is this #70555 (comment)

@k8s-ci-robot k8s-ci-robot added size/L and removed size/XL labels Nov 14, 2018

@prameshj prameshj force-pushed the prameshj:nodelocaldnscache branch from 481e979 to 2675fd5 Nov 14, 2018

@mikedanese
Copy link
Member

mikedanese left a comment

Can you verify hack/local-up-cluster.sh with and without enabling local DNS and verify that GCE works with enabling local DNS?

Otherwise
lgtm
/approve

local -r localdns_file="${dst_dir}/dns/nodelocaldns/nodelocaldns.yaml"
mv "${dst_dir}/dns/nodelocaldns/nodelocaldns.yaml.sed" "${localdns_file}"
# Replace the sed configurations with variable values.
sed -i -e "s/\\\$DNS_DOMAIN/${DNS_DOMAIN}/g" "${localdns_file}"

This comment has been minimized.

@mikedanese

mikedanese Nov 14, 2018

Member

Just a suggestion, I'd recommend sticking with the DNS_DOMAIN format so you don't have to worry about escaping.

@@ -704,7 +706,11 @@ function start_kubelet {
mkdir -p "/var/lib/kubelet" &>/dev/null || sudo mkdir -p "/var/lib/kubelet"
# Enable dns
if [[ "${ENABLE_CLUSTER_DNS}" = true ]]; then

This comment has been minimized.

@mikedanese

mikedanese Nov 14, 2018

Member

The usage of == vs = in this file is confusing but I see we do this elsewhere in this file.

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Nov 14, 2018

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bowei, madhusudancs, mikedanese, prameshj, roberthbailey

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@prameshj

This comment has been minimized.

Copy link
Contributor

prameshj commented Nov 14, 2018

/retest

Removed .salt template , using .sed template
Removed default config options from yaml.
Removed unused yaml files

@prameshj prameshj force-pushed the prameshj:nodelocaldnscache branch from 2675fd5 to 6d7c5e9 Nov 14, 2018

@prameshj

This comment has been minimized.

Copy link
Contributor

prameshj commented Nov 14, 2018

@mikedanese Verified that gce cluster comes up with cache enabled.
hack/local-up-cluster.sh worked.
"KUBE_ENABLE_NODELOCAL_DNS=true hack/local-up-cluster.sh" failed with "disallowed by cluster policy" error since node-local-dns runs in privileged mode and the local kubelet does not get the --allow-privileged flag. So this is working as intended.

@mikedanese

This comment has been minimized.

Copy link
Member

mikedanese commented Nov 14, 2018

/lgtm

@prameshj

This comment has been minimized.

Copy link
Contributor

prameshj commented Nov 14, 2018

/retest

@k8s-ci-robot k8s-ci-robot merged commit 79dab47 into kubernetes:master Nov 14, 2018

18 checks passed

cla/linuxfoundation prameshj authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gke Job succeeded.
Details
pull-kubernetes-e2e-kops-aws Job succeeded.
Details
pull-kubernetes-e2e-kubeadm-gce Skipped
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Job succeeded.
Details
pull-kubernetes-local-e2e-containerized Skipped
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
tide In merge pool.
Details

@prameshj prameshj referenced this pull request Nov 20, 2018

Open

kube-dns per node #45363

@krmayankk

This comment has been minimized.

Copy link
Contributor

krmayankk commented Nov 29, 2018

does this work on GKE as well ? How are we solving the issue of connecting to local dnsmasq from the kubernetes pods ?

@prameshj

This comment has been minimized.

Copy link
Contributor

prameshj commented Nov 29, 2018

does this work on GKE as well ? How are we solving the issue of connecting to local dnsmasq from the kubernetes pods ?

Yes, this will be available on GKE once version 1.13 is released. Not sure i understand the second question - The pods will connect to the dns cache instance because that will be the configured server in /etc/resolv.conf. This is the same way it works in OpenSource.

@blakebarnett

This comment has been minimized.

Copy link

blakebarnett commented Nov 29, 2018

@krmayankk It works on GKE because it redirects requests for the well-known DNS service IP (x.x.x.10) to the coredns process running locally as a daemonset using iptables. You can see the code for how it works here: kubernetes/dns#270

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment