Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support running Cloud Node Manager as a Windows Service #800

Closed
JoelSpeed opened this issue Sep 10, 2021 · 5 comments · Fixed by #823
Closed

Support running Cloud Node Manager as a Windows Service #800

JoelSpeed opened this issue Sep 10, 2021 · 5 comments · Fixed by #823
Labels
kind/feature Categorizes issue or PR as related to a new feature.

Comments

@JoelSpeed
Copy link
Contributor

What would you like to be added:

Other components that run on Windows nodes (eg Kubelet or Kube-Proxy) have the ability to be run as a Windows service.
Flags are added that adjust the init process and optionally, initialise the compiled exe as a Windows service.
The actual initialisation is handled here and would be relatively easy to port over to this project or into (pending further discussion) the k8s.io/cloud-provider repository.

We should add support for this at a minimum to the Cloud Node Manager for Azure as this component is required to run on every Node.

Why is this needed:

In some scenarios, it is not possible to run the CNM as a pod within the cluster. We have a requirement to run the CNM as a binary outside of the cluster on Windows nodes, and specifically for this use case, we expect the CNM to run as a Windows service in the same way that Kubelet and Kube Proxy do for us already.

CC @andyzhangx @aravindhp @sebsoto @Fedosin

@JoelSpeed JoelSpeed added the kind/feature Categorizes issue or PR as related to a new feature. label Sep 10, 2021
@elmiko
Copy link

elmiko commented Sep 16, 2021

we discussed this a little bit at the sig cloud provider meeting last night, there aren't any notes in the agenda but the meeting was recorded (agenda).

my general takeaway is that there is a need for the windows specific binary to be made, and there is some specialized initialisation logic that is currently used, but that in the future we might see these pods scheduled to windows nodes through daemonsets with node selectors. i am almost positive i am missing some of the nuance from the discussion, would be happy to have a comment from @andrewsykim or @cheftako

@andrewsykim
Copy link
Member

@feiskyer to share more details

@feiskyer
Copy link
Member

The image mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.1 supports both Windows and Linux nodes:

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
  "manifests": [
    {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "size": 587,
      "digest": "sha256:7a135ed1705995d5b78f6f7a4a7d5c413201ceccdb4b4545e581d9f277e78b48",
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "size": 951,
      "digest": "sha256:20051b3338a59171dea65876441ceacbe01154d488c6f645e651ea95fc7e8df2",
      "platform": {
        "architecture": "amd64",
        "os": "windows",
        "os.version": "10.0.17763.2114"
      }
    }
  ]
}

The actual initialisation is handled here and would be relatively easy to port over to this project or into (pending further discussion) the k8s.io/cloud-provider repository.

This is also reasonable; we could add it in the next release.

@JoelSpeed before the new flag is added, is it possible for you to run cnm as daemonsets on both Windows and Linux nodes?

@JoelSpeed
Copy link
Contributor Author

@JoelSpeed before the new flag is added, is it possible for you to run cnm as daemonsets on both Windows and Linux nodes?

Unfortunately we have a requirement that means we cannot run the CNM as a daemonset on Windows nodes, so we need support for the Windows service initialisation code to run the CNM directly on the Windows host

@JoelSpeed
Copy link
Contributor Author

I've implemented a PR to add support for this, it didn't take very long as it's mainly copy/paste, but perhaps we can continue the conversation on the PR, it's relatively small

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants