kubeadm: disable the kube-proxy DaemonSet on non-Linux nodes

[neolit123]

What this PR does / why we need it:

Windows worker nodes run kube-proxy as a Windows service.
In the future the kube-proxy DaemonSet might run on Windows nodes
too, but for now a temporary measure is needed to disable it.

Add a linux node selector in the kube-proxy manifest spec.
Similar was done already for CoreDNS.

Which issue(s) this PR fixes:

xref kubernetes/kubeadm#1393

Special notes for your reviewer:
NONE

Does this PR introduce a user-facing change?:

kubeadm: disable the kube-proxy DaemonSet on non-Linux nodes. This step is required to support Windows worker nodes.

/kind cleanup
/priority important-longterm
@kubernetes/sig-cluster-lifecycle-pr-reviews
/assign @timothysc @fabriziopandini
cc @PatrickLang

[neolit123]

/sig windows

[SataQiu]

LGTM

[rosti]

My question is how we are going to replace the DaemonSet? I know that the answer is a Windows service ATM. However, do we expect, that users do their own kube-proxy provisioning or should kubeadm do it instead?

It looks good to me, but I want to know if we should be providing some replacement.

[neolit123]

My question is how we are going to replace the DaemonSet? I know that the answer is a Windows service ATM. However, do we expect, that users do their own kube-proxy provisioning or should kubeadm do it instead?

the plan is for kubeadm to not manage kube-proxy on Windows workers until the DaemonSet works.
we are going to have a wrapper around kubeadm that starts kube-proxy as a Windows service.

[rosti]

we are going to have a wrapper around kubeadm that starts kube-proxy as a Windows service.

Now this sounds horrible, unfortunately the alternative is to have an optional add-on per-node that deploys kube-proxy as a service on Windows nodes and this is even worse.
So I guess, that leaving it alone until DaemonSet kube-proxy works on Windows is a good choice for now. It does not imply the introduction of some technical debt for us (other than making sure to remove the node selector later on).

/lgtm
/hold
For @timothysc & @fabriziopandini 's opinion.

[detiber]

Can't say I'm a fan of the approach, but at the same time I don't have a better alternative, so a hesitant +1 from me.

[fabriziopandini]

I'm ok with this PR, but I'm not sure about

we are going to have a wrapper around kubeadm that starts kube-proxy as a Windows service.

On Linux, systemd service creation is not a kubeadm task, but it is something delegated to OS specific packages (deb or rpm); kubeadm task usually is limited to service configuration/start.

My first reaction is that Iwe should follow the same approach also on windows/for kube-proxy.
An OS specific package should install kube-proxy and create the service, then Kubeadm should check the service exists in preflight, then pull the configuration and start the service during join

But TBH I didn't investigated this in detail, so might be I'm missing something 😅

[neolit123]

My first reaction is that Iwe should follow the same approach also on windows/for kube-proxy.
An OS specific package should install kube-proxy and create the service, then Kubeadm should check the service exists in preflight, then pull the configuration and start the service during join

WIndows nodes are blocked on privileged containers at the moment.
so this is the reason kube-proxy is run as a Windows service.

one day kube-proxy will be able to run as a DaemonSet on Windows too, until then we are heavily leaning towards the wrapper approach so that we don't maintain such code in kubeadm.

also i wanted to explain the Windows / kubeadm discussions i had recently with SIG Windows, but i won't be able to attend this week's kubeadm office hours.

[timothysc]

I'm not familiar with windows deployments, but how does service routing work on window then?

[neolit123]

it works somehow, this is more of a question for the SIG Window folk.

[timothysc]

@michmike ^

[benmoss]

kube-proxy still runs and handles service routing, it's just that it needs to run on the host directly and can't be run from inside a privileged container.

[timothysc]

? below

[neolit123]

let's hold this PR regardless, so that we can get the KEP in place that answers the questions.
/approve cancel

[timothysc]

kubernetes/enhancements#994

/approve
/hold cancel

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: timothysc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cmd/kubeadm/OWNERS [timothysc]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[fejta-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

[fejta-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.