-
Notifications
You must be signed in to change notification settings - Fork 1.8k
OSDOCS-11411: Adding ROSA-compatible multi-arch scheduling content #83997
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
kcarmichael08
merged 1 commit into
openshift:main
from
laubai:osdocs-11411-multiarch-scheduling-post-hcp-merge
Nov 26, 2024
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
:_mod-docs-content-type: CONCEPT | ||
[id="multi-architecture-scheduling_{context}"] | ||
= Scheduling workloads on clusters with multi-architecture compute machines | ||
|
||
When you deploy workloads on a cluster with compute nodes that use different architectures, you must align pod architecture with the architecture of the underlying node. Your workload may also require additional configuration to particular resources depending on the underlying node architecture. | ||
|
||
ifndef::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[] | ||
You can use the Multiarch Tuning Operator to enable architecture-aware scheduling of workloads on clusters with multi-architecture compute machines. The Multiarch Tuning Operator implements additional scheduler predicates in the pods specifications based on the architectures that the pods can support at creation time. | ||
endif::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
:_mod-docs-content-type: ASSEMBLY | ||
//This assembly duplicates the following file to avoid symbolic links: | ||
// * post_installation_configuration/configuring-multi-arch-compute-machines/multi-architecture-compute-managing.adoc | ||
|
||
:context: rosa-multi-arch-compute-managing | ||
[id="rosa-multi-arch-managing"] | ||
= Managing a cluster with multi-architecture compute machines | ||
include::_attributes/common-attributes.adoc[] | ||
|
||
toc::[] | ||
|
||
Managing a cluster that has nodes with multiple architectures requires you to consider node architecture as you monitor the cluster and manage your workloads. This requires you to take additional considerations into account when you schedule workloads in a multi-architecture cluster. | ||
|
||
include::modules/multi-architecture-scheduling.adoc[leveloffset=+1] | ||
|
||
include::modules/multi-architecture-scheduling-examples.adoc[leveloffset=+2] | ||
|
||
// TODO OSDOCS-11411: Add back when HCP split in Nodes doc is complete | ||
// .Additional resources | ||
// * xref ../nodes/scheduling/nodes-scheduler-node-affinity.adoc#nodes-scheduler-node-affinity[Controlling pod placement on nodes using node affinity] |
33 changes: 33 additions & 0 deletions
33
snippets/multi-arch-schedule-affinity-taint-toleration.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
:_mod-docs-content-type: SNIPPET | ||
|
||
Using node affinity with taints and tolerations:: When a scheduler computes the set of nodes to schedule a pod, tolerations can broaden the set while node affinity restricts the set. If you set a taint on nodes that have a specific architecture, you must also add a toleration to workloads that you want to be scheduled there. | ||
laubai marked this conversation as resolved.
Show resolved
Hide resolved
|
||
+ | ||
.Example deployment with node affinity and toleration set | ||
-- | ||
[source,yaml] | ||
---- | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: # ... | ||
spec: | ||
# ... | ||
template: | ||
# ... | ||
spec: | ||
affinity: | ||
nodeAffinity: | ||
requiredDuringSchedulingIgnoredDuringExecution: | ||
nodeSelectorTerms: | ||
- matchExpressions: | ||
- key: kubernetes.io/arch | ||
operator: In | ||
values: | ||
- amd64 | ||
- arm64 | ||
tolerations: | ||
- key: "multiarch.openshift.io/arch" | ||
value: "arm64" | ||
operator: "Equal" | ||
effect: "NoSchedule" | ||
---- | ||
-- |
12 changes: 12 additions & 0 deletions
12
snippets/multi-arch-schedule-default-toleration-namespace.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
:_mod-docs-content-type: SNIPPET | ||
|
||
Creating a default toleration in a namespace:: When a node or machine set has a taint, only workloads that tolerate that taint can be scheduled. You can annotate a namespace so all of the workloads get the same default toleration by running the following command: | ||
laubai marked this conversation as resolved.
Show resolved
Hide resolved
|
||
+ | ||
.Example default toleration set on a namespace | ||
-- | ||
[source,terminal] | ||
---- | ||
$ oc annotate namespace my-namespace \ | ||
'scheduler.alpha.kubernetes.io/defaultTolerations'='[{"operator": "Exists", "effect": "NoSchedule", "key": "multiarch.openshift.io/arch"}]' | ||
---- | ||
-- |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
:_mod-docs-content-type: SNIPPET | ||
|
||
Using `nodeAffinity` to schedule nodes with specific architectures:: You can allow a workload to be scheduled on only a set of nodes with architectures supported by its images, you can set the `spec.affinity.nodeAffinity` field in your pod's template specification. | ||
laubai marked this conversation as resolved.
Show resolved
Hide resolved
|
||
+ | ||
.Example deployment with node affinity set | ||
-- | ||
[source,yaml] | ||
---- | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: # ... | ||
spec: | ||
# ... | ||
template: | ||
# ... | ||
spec: | ||
affinity: | ||
nodeAffinity: | ||
requiredDuringSchedulingIgnoredDuringExecution: | ||
nodeSelectorTerms: | ||
- matchExpressions: | ||
- key: kubernetes.io/arch | ||
operator: In | ||
values: <1> | ||
- amd64 | ||
- arm64 | ||
---- | ||
<1> Specify the supported architectures. Valid values include `amd64`,`arm64`, or both values. | ||
[source,yaml] | ||
-- |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
:_mod-docs-content-type: SNIPPET | ||
|
||
Tainting each node for a specific architecture:: You can taint a node to avoid the node scheduling workloads that are incompatible with its architecture. When your cluster uses a `MachineSet` object, you can add parameters to the `.spec.template.spec.taints` field to avoid workloads being scheduled on nodes with non-supported architectures. | ||
laubai marked this conversation as resolved.
Show resolved
Hide resolved
|
||
+ | ||
Before you add a taint to a node, you must scale down the `MachineSet` object or remove existing available machines. For more information, see _Modifying a compute machine set_. | ||
+ | ||
.Example machine set with taint set | ||
-- | ||
[source,yaml] | ||
---- | ||
apiVersion: machine.openshift.io/v1beta1 | ||
kind: MachineSet | ||
metadata: # ... | ||
spec: | ||
# ... | ||
template: | ||
# ... | ||
spec: | ||
# ... | ||
taints: | ||
- effect: NoSchedule | ||
key: multiarch.openshift.io/arch | ||
value: arm64 | ||
---- | ||
-- | ||
+ | ||
You can also set a taint on a specific node by running the following command: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc adm taint nodes <node-name> multiarch.openshift.io/arch=arm64:NoSchedule | ||
---- |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
:_mod-docs-content-type: SNIPPET | ||
|
||
Tolerating architecture taints in workloads:: When a node or machine set has a taint, only workloads that tolerate that taint can be scheduled. You can configure your workload with a `toleration` so that it is scheduled on nodes with specific architecture taints. | ||
laubai marked this conversation as resolved.
Show resolved
Hide resolved
|
||
+ | ||
.Example deployment with toleration set | ||
-- | ||
[source,yaml] | ||
---- | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: # ... | ||
spec: | ||
# ... | ||
template: | ||
# ... | ||
spec: | ||
tolerations: | ||
- key: "multiarch.openshift.io/arch" | ||
value: "arm64" | ||
operator: "Equal" | ||
effect: "NoSchedule" | ||
---- | ||
This example deployment can be scheduled on nodes and machine sets that have the `multiarch.openshift.io/arch=arm64` taint specified. | ||
-- |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.