Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Document the new MinDomainsInPodTopologySpread feature gate #32340

Merged
merged 1 commit into from Apr 12, 2022
Merged

Document the new MinDomainsInPodTopologySpread feature gate #32340

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
28 changes: 27 additions & 1 deletion content/en/docs/concepts/workloads/pods/pod-topology-spread-constraints.md
View file
Open in desktop
Expand Up @@ -73,16 +73,42 @@ You can define one or multiple `topologySpreadConstraint` to instruct the kube-s

- **maxSkew** describes the degree to which Pods may be unevenly distributed.
It must be greater than zero. Its semantics differs according to the value of `whenUnsatisfiable`:

- when `whenUnsatisfiable` equals to "DoNotSchedule", `maxSkew` is the maximum
permitted difference between the number of matching pods in the target
topology and the global minimum
(the minimum number of pods that match the label selector in a topology domain. For example, if you have 3 zones with 0, 2 and 3 matching pods respectively, The global minimum is 0).
(the minimum number of pods that match the label selector in a topology domain.
For example, if you have 3 zones with 0, 2 and 3 matching pods respectively,
The global minimum is 0).
- when `whenUnsatisfiable` equals to "ScheduleAnyway", scheduler gives higher
precedence to topologies that would help reduce the skew.

- **minDomains** indicates a minimum number of eligible domains.
A domain is a particular instance of a topology. An eligible domain is a domain whose
nodes match the node selector.

- The value of `minDomains` must be greater than 0, when specified.
- When the number of eligible domains with match topology keys is less than `minDomains`,
Pod topology spread treats "global minimum" as 0, and then the calculation of `skew` is performed.
The "global minimum" is the minimum number of matching Pods in an eligible domain,
or zero if the number of eligible domains is less than `minDomains`.
- When the number of eligible domains with matching topology keys equals or is greater than
`minDomains`, this value has no effect on scheduling.
- When `minDomains` is nil, the constraint behaves as if `minDomains` is 1.
- When `minDomains` is not nil, the value of `whenUnsatisfiable` must be "`DoNotSchedule`".

tengqm marked this conversation as resolved.
Show resolved Hide resolved
{{< note >}}
The `minDomains` field is an alpha field added in 1.24. You have to enable the
`MinDomainsInPodToplogySpread` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
in order to use it.
{{< /note >}}

- **topologyKey** is the key of node labels. If two Nodes are labelled with this key and have identical values for that label, the scheduler treats both Nodes as being in the same topology. The scheduler tries to place a balanced number of Pods into each topology domain.

- **whenUnsatisfiable** indicates how to deal with a Pod if it doesn't satisfy the spread constraint:
- `DoNotSchedule` (default) tells the scheduler not to schedule it.
- `ScheduleAnyway` tells the scheduler to still schedule it while prioritizing nodes that minimize the skew.

- **labelSelector** is used to find matching Pods. Pods that match this label selector are counted to determine the number of Pods in their corresponding topology domain. See [Label Selectors](/docs/concepts/overview/working-with-objects/labels/#label-selectors) for more details.

When a Pod defines more than one `topologySpreadConstraint`, those constraints are ANDed: The kube-scheduler looks for a node for the incoming Pod that satisfies all the constraints.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add the example to describe minDomains?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer adding the example as a mermaid diagram in a separate PR.

Expand Down
3 changes: 3 additions & 0 deletions content/en/docs/reference/command-line-tools-reference/feature-gates.md
View file
Open in desktop
Expand Up @@ -149,6 +149,7 @@ different Kubernetes components.
| `MemoryManager` | `false` | Alpha | 1.21 | 1.21 |
| `MemoryManager` | `true` | Beta | 1.22 | |
| `MemoryQoS` | `false` | Alpha | 1.22 | |
| `MinDomainsInPodTopologySpread` | `false` | Alpha | 1.24 | |
| `MixedProtocolLBService` | `false` | Alpha | 1.20 | 1.23 |
| `MixedProtocolLBService` | `true` | Beta | 1.24 | |
| `NetworkPolicyEndPort` | `false` | Alpha | 1.21 | 1.21 |
Expand Down Expand Up @@ -989,6 +990,8 @@ Each feature gate is designed for enabling/disabling a specific feature:
NUMA topology.
- `MemoryQoS`: Enable memory protection and usage throttle on pod / container using
cgroup v2 memory controller.
- `MinDomainsInPodTopologySpread`: Enable `minDomains` in Pod
[topology spread constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/).
- `MixedProtocolLBService`: Enable using different protocols in the same `LoadBalancer` type
Service instance.
- `MountContainers`: Enable using utility containers on host as the volume mounter.
Expand Down