Skip to content

Commit

Permalink
Updating Windows intro and resource management pages (#34083)
Browse files Browse the repository at this point in the history 
* Updating some Windows docs pages including:
- Fixinig heading levels for /docs/concepts/windows/intro.md
- Misc updates to /docs/concepts/windows/intro.ms
- Updates to /docs/concepts/configuration/windows-resource-management.md
for accuracy

Signed-off-by: Mark Rossetti <marosset@microsoft.com>

* Apply suggestions from code review

Co-authored-by: Qiming Teng <tengqm@outlook.com>
Co-authored-by: Tim Bannister <tim@scalefactory.com>
Co-authored-by: divya-mohan0209 <divya.mohan0209@gmail.com>

Co-authored-by: Qiming Teng <tengqm@outlook.com>
Co-authored-by: Tim Bannister <tim@scalefactory.com>
Co-authored-by: divya-mohan0209 <divya.mohan0209@gmail.com>
  • Loading branch information
@marosset @tengqm
@sftim @divya-mohan0209
4 people committed Jun 6, 2022
1 parent bfdfd60 commit eb88ef7
Show file tree
Hide file tree
Showing 2 changed files with 56 additions and 57 deletions.
62 changes: 29 additions & 33 deletions content/en/docs/concepts/configuration/windows-resource-management.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,52 +32,48 @@ host, and thus privileged containers are not available on Windows.
Containers cannot assume an identity from the host because the Security Account Manager
(SAM) is separate.

## Memory reservations {#resource-management-memory}
## Memory management {#resource-management-memory}

Windows does not have an out-of-memory process killer as Linux does. Windows always
treats all user-mode memory allocations as virtual, and pagefiles are mandatory.

Windows nodes do not overcommit memory for processes running in containers. The
Windows nodes do not overcommit memory for processes. The
net effect is that Windows won't reach out of memory conditions the same way Linux
does, and processes page to disk instead of being subject to out of memory (OOM)
termination. If memory is over-provisioned and all physical memory is exhausted,
then paging can slow down performance.

You can place bounds on memory use for workloads using the kubelet
parameters `--kubelet-reserve` and/or `--system-reserve`; these account
for memory usage on the node (outside of containers), and reduce
[NodeAllocatable](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable).
As you deploy workloads, set resource limits on containers. This also subtracts from
`NodeAllocatable` and prevents the scheduler from adding more pods once a node is full.

{{< note >}}
When you set memory resource limits for Windows containers, you should either set a
limit and leave the memory request unspecified, or set the request equal to the limit.
{{< /note >}}

On Windows, good practice to avoid over-provisioning is to configure the kubelet
with a system reserved memory of at least 2GiB to account for Windows, Kubernetes
and container runtime overheads.

## CPU reservations {#resource-management-cpu}

To account for CPU use by the operating system, the container runtime, and by
Kubernetes host processes such as the kubelet, you can (and should) reserve a
percentage of total CPU. You should determine this CPU reservation taking account of
to the number of CPU cores available on the node. To decide on the CPU percentage to
reserve, identify the maximum pod density for each node and monitor the CPU usage of
the system services running there, then choose a value that meets your workload needs.
## CPU management {#resource-management-cpu}

You can place bounds on CPU usage for workloads using the
kubelet parameters `--kubelet-reserve` and/or `--system-reserve` to
account for CPU usage on the node (outside of containers).
This reduces `NodeAllocatable`.
The cluster-wide scheduler then takes this reservation into account when determining
pod placement.
Windows can limit the amount of CPU time allocated for different processes but cannot
guarantee a minimum amount of CPU time.

On Windows, the kubelet supports a command-line flag to set the priority of the
On Windows, the kubelet supports a command-line flag to set the
[scheduling priority](https://docs.microsoft.com/windows/win32/procthread/scheduling-priorities) of the
kubelet process: `--windows-priorityclass`. This flag allows the kubelet process to get
more CPU time slices when compared to other processes running on the Windows host.
More information on the allowable values and their meaning is available at
[Windows Priority Classes](https://docs.microsoft.com/en-us/windows/win32/procthread/scheduling-priorities#priority-class).
To ensure that running Pods do not starve the kubelet of CPU cycles, set this flag to `ABOVE_NORMAL_PRIORITY_CLASS` or above.

## Resource reservation {#resource-reservation}

To account for memory and CPU used by the operating system, the container runtime, and by
Kubernetes host processes such as the kubelet, you can (and should) reserve
memory and CPU resources with the `--kube-reserved` and/or `--system-reserved` kubelet flags.
On Windows these values are only used to calculate the node's
[allocatable](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable) resources.

{{< caution >}}
As you deploy workloads, set resource memory and CPU limits on containers.
This also subtracts from `NodeAllocatable` and helps the cluster-wide scheduler in determining which pods to place on which nodes.

Scheduling pods without limits may over-provision the Windows nodes and in extreme
cases can cause the nodes to become unhealthy.
{{< /caution >}}

On Windows, a good practice is to reserve at least 2GiB of memory.

To determine how much CPU to reserve,
identify the maximum pod density for each node and monitor the CPU usage of
the system services running there, then choose a value that meets your workload needs.
51 changes: 27 additions & 24 deletions content/en/docs/concepts/windows/intro.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,8 @@ Some node features are only available if you use a specific
including:

* HugePages: not supported for Windows containers
* Privileged containers: not supported for Windows containers
* Privileged containers: not supported for Windows containers.
[HostProcess Containers](/docs/tasks/configure-pod-container/create-hostprocess-pod/) offer similar functionality.
* TerminationGracePeriod: requires containerD

Not all features of shared namespaces are supported. See [API compatibility](#api)
Expand Down Expand Up @@ -78,7 +79,7 @@ section refers to several key workload abstractions and how they map to Windows.

* Single or multiple containers per Pod with process isolation and volume sharing
* Pod `status` fields
* Readiness and Liveness probes
* Readiness, liveness, and startup probes
* postStart & preStop container lifecycle hooks
* ConfigMap, Secrets: as environment variables or volumes
* `emptyDir` volumes
Expand Down Expand Up @@ -136,7 +137,7 @@ section refers to several key workload abstractions and how they map to Windows.
Pods, workload resources, and Services are critical elements to managing Windows
workloads on Kubernetes. However, on their own they are not enough to enable
the proper lifecycle management of Windows workloads in a dynamic cloud native
environment. Kubernetes also supports:
environment.

* `kubectl exec`
* Pod and container metrics
Expand All @@ -150,14 +151,15 @@ Some kubelet command line options behave differently on Windows, as described be

* The `--windows-priorityclass` lets you set the scheduling priority of the kubelet process
(see [CPU resource management](/docs/concepts/configuration/windows-resource-management/#resource-management-cpu))
* The `--kubelet-reserve`, `--system-reserve` , and `--eviction-hard` flags update
* The `--kube-reserved`, `--system-reserved` , and `--eviction-hard` flags update
[NodeAllocatable](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable)
* Eviction by using `--enforce-node-allocable` is not implemented
* Eviction by using `--eviction-hard` and `--eviction-soft` are not implemented
* A kubelet running on a Windows node does not have memory
restrictions. `--kubelet-reserve` and `--system-reserve` do not set limits on
kubelet or processes running on the host. This means kubelet or a process on the host
could cause memory resource starvation outside the node-allocatable and scheduler.
* When running on a Windows node the kubelet does not have memory or CPU
restrictions. `--kube-reserved` and `--system-reserved` only subtract from `NodeAllocatable`
and do not guarantee resource provided for workloads.
See [Resource Management for Windows nodes](/docs/concepts/configuration/windows-resource-management/#resource-reservation)
for more information.
* The `MemoryPressure` Condition is not implemented
* The kubelet does not take OOM eviction actions

Expand Down Expand Up @@ -192,7 +194,7 @@ Container exit codes follow the same convention where 0 is success, and nonzero
The specific error codes may differ across Windows and Linux. However, exit codes
passed from the Kubernetes components (kubelet, kube-proxy) are unchanged.

##### Field compatibility for container specifications {#compatibility-v1-pod-spec-containers}
#### Field compatibility for container specifications {#compatibility-v1-pod-spec-containers}

The following list documents differences between how Pod container specifications
work between Windows and Linux:
Expand Down Expand Up @@ -232,7 +234,7 @@ work between Windows and Linux:
default value is `/dev/termination-log`, which does work because it does not
exist on Windows by default.

##### Field compatibility for Pod specifications {#compatibility-v1-pod}
#### Field compatibility for Pod specifications {#compatibility-v1-pod}

The following list documents differences between how Pod specifications work between Windows and Linux:

Expand Down Expand Up @@ -260,17 +262,18 @@ The following list documents differences between how Pod specifications work bet
* You cannot enable `mountPropagation` for volume mounts as this is not
supported on Windows.

##### Field compatibility for Pod security context {#compatibility-v1-pod-spec-containers-securitycontext}
#### Field compatibility for Pod security context {#compatibility-v1-pod-spec-containers-securitycontext}

None of the Pod [`securityContext`](/docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context) fields work on Windows.

### Node problem detector
## Node problem detector

The node problem detector (see
[Monitor Node Health](/docs/tasks/debug/debug-cluster/monitor-node-health/))
is not compatible with Windows.
has preliminary support for Windows.
For more information, visit the project's [GitHub page](https://github.com/kubernetes/node-problem-detector#windows).

### Pause container
## Pause container

In a Kubernetes Pod, an infrastructure or “pause” container is first created
to host the container. In Linux, the cgroups and namespaces that make up a pod
Expand All @@ -293,7 +296,7 @@ The Kubernetes project recommends using the Microsoft maintained image if you ar
deploying to a production or production-like environment that requires signed
binaries.

### Container runtimes {#container-runtime}
## Container runtimes {#container-runtime}

You need to install a
{{< glossary_tooltip text="container runtime" term_id="container-runtime" >}}
Expand All @@ -303,7 +306,7 @@ The following container runtimes work with Windows:

{{% thirdparty-content %}}

#### cri-containerd
### ContainerD

{{< feature-state for_k8s_version="v1.20" state="stable" >}}

Expand All @@ -318,9 +321,10 @@ when using GMSA with containerd to access Windows network shares, which requires
kernel patch.
{{< /note >}}

#### Mirantis Container Runtime {#mcr}
### Mirantis Container Runtime {#mcr}

[Mirantis Container Runtime](https://docs.mirantis.com/mcr/20.10/overview.html) (MCR) is available as a container runtime for all Windows Server 2019 and later versions.
[Mirantis Container Runtime](https://docs.mirantis.com/mcr/20.10/overview.html) (MCR)
is available as a container runtime for all Windows Server 2019 and later versions.

See [Install MCR on Windows Servers](https://docs.mirantis.com/mcr/20.10/install/mcr-windows.html) for more information.

Expand Down Expand Up @@ -361,12 +365,10 @@ If you have what looks like a bug, or you would like to
make a feature request, please follow the [SIG Windows contributing guide](https://github.com/kubernetes/community/blob/master/sig-windows/CONTRIBUTING.md#reporting-issues-and-feature-requests) to create a new issue.
You should first search the list of issues in case it was
reported previously and comment with your experience on the issue and add additional
logs. SIG-Windows Slack is also a great avenue to get some initial support and
logs. SIG Windows channel on the Kubernetes Slack is also a great avenue to get some initial support and
troubleshooting ideas prior to creating a ticket.

## {{% heading "whatsnext" %}}

### Deployment tools
## Deployment tools

The kubeadm tool helps you to deploy a Kubernetes cluster, providing the control
plane to manage the cluster it, and nodes to run your workloads.
Expand All @@ -375,9 +377,10 @@ explains how to deploy Windows nodes to your cluster using kubeadm.

The Kubernetes [cluster API](https://cluster-api.sigs.k8s.io/) project also provides means to automate deployment of Windows nodes.

### Windows distribution channels
## Windows distribution channels

For a detailed explanation of Windows distribution channels see the [Microsoft documentation](https://docs.microsoft.com/en-us/windows-server/get-started-19/servicing-channels-19).
For a detailed explanation of Windows distribution channels see the
[Microsoft documentation](https://docs.microsoft.com/en-us/windows-server/get-started-19/servicing-channels-19).

Information on the different Windows Server servicing channels
including their support models can be found at
Expand Down

0 comments on commit eb88ef7

Please sign in to comment.