diff --git a/_topic_maps/_topic_map_rosa_hcp.yml b/_topic_maps/_topic_map_rosa_hcp.yml index 144087b6d85f..d25a15f3c316 100644 --- a/_topic_maps/_topic_map_rosa_hcp.yml +++ b/_topic_maps/_topic_map_rosa_hcp.yml @@ -252,6 +252,8 @@ Topics: File: nodes-cluster-resource-configure - Name: Configuring PID limits File: rosa-configuring-pid-limits +- Name: Managing multi-architecture clusters + File: rosa-multi-arch-cluster-managing --- Name: Security and compliance Dir: security diff --git a/modules/multi-architecture-scheduling-examples.adoc b/modules/multi-architecture-scheduling-examples.adoc index 252de47f968b..d786a0efd813 100644 --- a/modules/multi-architecture-scheduling-examples.adoc +++ b/modules/multi-architecture-scheduling-examples.adoc @@ -4,136 +4,23 @@ :_mod-docs-content-type: CONCEPT [id="multi-architecture-scheduling-examples_{context}"] - = Sample multi-architecture node workload deployments -Before you schedule workloads on a cluster with compute nodes of different architectures, consider the following use cases: +Scheduling a workload to an appropriate node based on architecture works in the same way as scheduling based on any other node characteristic. +ifndef::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[] +Consider the following options when determining how to schedule your workloads. +endif::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[] -Using node affinity to schedule workloads on a node:: You can allow a workload to be scheduled on only a set of nodes with architectures supported by its images, you can set the `spec.affinity.nodeAffinity` field in your pod's template specification. -+ -.Example deployment with the `nodeAffinity` set to certain architectures -[source,yaml] ----- -apiVersion: apps/v1 -kind: Deployment -metadata: # ... -spec: - # ... - template: - # ... - spec: - affinity: - nodeAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - nodeSelectorTerms: - - matchExpressions: - - key: kubernetes.io/arch - operator: In - values: <1> - - amd64 - - arm64 ----- -<1> Specify the supported architectures. Valid values include `amd64`,`arm64`, or both values. +include::snippets/multi-arch-schedule-nodeaffinity.adoc[leveloffset=+0] -Tainting every node for a specific architecture:: You can taint a node to avoid workloads that are not compatible with its architecture to be scheduled on that node. In the case where your cluster is using a `MachineSet` object, you can add parameters to the `.spec.template.spec.taints` field to avoid workloads being scheduled on nodes with non-supported architectures. +ifndef::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[] -* Before you can taint a node, you must scale down the `MachineSet` object or remove available machines. You can scale down the machine set by using one of following commands: -+ -[source,terminal] ----- -$ oc scale --replicas=0 machineset -n openshift-machine-api ----- -+ -Or: -+ -[source,terminal] ----- -$ oc edit machineset -n openshift-machine-api ----- -For more information on scaling machine sets, see "Modifying a compute machine set". +include::snippets/multi-arch-schedule-taint.adoc[leveloffset=+0] -+ --- -.Example `MachineSet` with a taint set -[source,yaml] ----- -apiVersion: machine.openshift.io/v1beta1 -kind: MachineSet -metadata: # ... -spec: - # ... - template: - # ... - spec: - # ... - taints: - - effect: NoSchedule - key: multi-arch.openshift.io/arch - value: arm64 ----- -You can also set a taint on a specific node by running the following command: -[source,terminal] ----- -$ oc adm taint nodes multi-arch.openshift.io/arch=arm64:NoSchedule ----- --- +include::snippets/multi-arch-schedule-default-toleration-namespace.adoc[leveloffset=+0] -Creating a default toleration:: You can annotate a namespace so all of the workloads get the same default toleration by running the following command: -+ -[source,terminal] ----- -$ oc annotate namespace my-namespace \ - 'scheduler.alpha.kubernetes.io/defaultTolerations'='[{"operator": "Exists", "effect": "NoSchedule", "key": "multi-arch.openshift.io/arch"}]' ----- +include::snippets/multi-arch-schedule-toleration.adoc[leveloffset=+0] -Tolerating architecture taints in workloads:: On a node with a defined taint, workloads will not be scheduled on that node. However, you can allow them to be scheduled by setting a toleration in the pod's specification. -+ -.Example deployment with a toleration -[source,yaml] ----- -apiVersion: apps/v1 -kind: Deployment -metadata: # ... -spec: - # ... - template: - # ... - spec: - tolerations: - - key: "multi-arch.openshift.io/arch" - value: "arm64" - operator: "Equal" - effect: "NoSchedule" ----- -+ -This example deployment can also be allowed on nodes with the `multi-arch.openshift.io/arch=arm64` taint specified. +include::snippets/multi-arch-schedule-affinity-taint-toleration.adoc[leveloffset=+0] -Using node affinity with taints and tolerations:: When a scheduler computes the set of nodes to schedule a pod, tolerations can broaden the set while node affinity restricts the set. If you set a taint to the nodes of a specific architecture, the following example toleration is required for scheduling pods. -+ -.Example deployment with a node affinity and toleration set. -[source,yaml] ----- -apiVersion: apps/v1 -kind: Deployment -metadata: # ... -spec: - # ... - template: - # ... - spec: - affinity: - nodeAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - nodeSelectorTerms: - - matchExpressions: - - key: kubernetes.io/arch - operator: In - values: - - amd64 - - arm64 - tolerations: - - key: "multi-arch.openshift.io/arch" - value: "arm64" - operator: "Equal" - effect: "NoSchedule" ----- \ No newline at end of file +endif::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[] diff --git a/modules/multi-architecture-scheduling.adoc b/modules/multi-architecture-scheduling.adoc new file mode 100644 index 000000000000..9f0655cc3252 --- /dev/null +++ b/modules/multi-architecture-scheduling.adoc @@ -0,0 +1,9 @@ +:_mod-docs-content-type: CONCEPT +[id="multi-architecture-scheduling_{context}"] += Scheduling workloads on clusters with multi-architecture compute machines + +When you deploy workloads on a cluster with compute nodes that use different architectures, you must align pod architecture with the architecture of the underlying node. Your workload may also require additional configuration to particular resources depending on the underlying node architecture. + +ifndef::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[] +You can use the Multiarch Tuning Operator to enable architecture-aware scheduling of workloads on clusters with multi-architecture compute machines. The Multiarch Tuning Operator implements additional scheduler predicates in the pods specifications based on the architectures that the pods can support at creation time. +endif::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[] diff --git a/post_installation_configuration/configuring-multi-arch-compute-machines/multi-architecture-compute-managing.adoc b/post_installation_configuration/configuring-multi-arch-compute-machines/multi-architecture-compute-managing.adoc index 1b813312691c..edf20985e70e 100644 --- a/post_installation_configuration/configuring-multi-arch-compute-machines/multi-architecture-compute-managing.adoc +++ b/post_installation_configuration/configuring-multi-arch-compute-machines/multi-architecture-compute-managing.adoc @@ -6,28 +6,23 @@ include::_attributes/common-attributes.adoc[] toc::[] -== Scheduling workloads on clusters with multi-architecture compute machines +Managing a cluster that has nodes with multiple architectures requires you to consider node architecture as you monitor the cluster and manage your workloads. This requires you to take additional considerations into account when you +ifndef::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[] +configure cluster resource requirements and behavior, or +endif::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[] +schedule workloads in a multi-architecture cluster. -Deploying a workload on a cluster with compute nodes of different architectures requires attention and monitoring of your cluster. There might be further actions you need to take in order to successfully place pods in the nodes of your cluster. - -You can use the Multiarch Tuning Operator to enable architecture-aware scheduling of workloads on clusters with multi-architecture compute machines. The Multiarch Tuning Operator implements additional scheduler predicates in the pods specifications based on the architectures that the pods can support at creation time. For more information, see xref:../../post_installation_configuration/configuring-multi-arch-compute-machines/multiarch-tuning-operator.adoc#multiarch-tuning-operator[Managing workloads on multi-architecture clusters by using the Multiarch Tuning Operator]. - -For more information on node affinity, scheduling, taints and tolerations, see the following documentation: - -ifndef::openshift-dedicated,openshift-rosa[] -* xref:../../nodes/scheduling/nodes-scheduler-taints-tolerations.adoc#nodes-scheduler-taints-tolerations[Controlling pod placement using node taints]. -endif::openshift-dedicated,openshift-rosa[] - -* xref:../../nodes/scheduling/nodes-scheduler-node-affinity.adoc#nodes-scheduler-node-affinity[Controlling pod placement on nodes using node affinity] - -* xref:../../nodes/scheduling/nodes-scheduler-about.adoc#nodes-scheduler-about[Controlling pod placement using the scheduler] +include::modules/multi-architecture-scheduling.adoc[leveloffset=+1] include::modules/multi-architecture-scheduling-examples.adoc[leveloffset=+2] .Additional resources - +* xref:../../post_installation_configuration/configuring-multi-arch-compute-machines/multiarch-tuning-operator.adoc#multiarch-tuning-operator[Managing workloads on multi-architecture clusters by using the Multiarch Tuning Operator] +* xref:../../nodes/scheduling/nodes-scheduler-taints-tolerations.adoc#nodes-scheduler-taints-tolerations[Controlling pod placement using node taints] +* xref:../../nodes/scheduling/nodes-scheduler-node-affinity.adoc#nodes-scheduler-node-affinity[Controlling pod placement on nodes using node affinity] +* xref:../../nodes/scheduling/nodes-scheduler-about.adoc#nodes-scheduler-about[Controlling pod placement using the scheduler] * xref:../../machine_management/modifying-machineset.adoc#machineset-modifying_modifying-machineset[Modifying a compute machine set] include::modules/multi-architecture-enabling-64k-pages.adoc[leveloffset=+1] -include::modules/multi-architecture-import-imagestreams.adoc[leveloffset=+1] \ No newline at end of file +include::modules/multi-architecture-import-imagestreams.adoc[leveloffset=+1] diff --git a/rosa_cluster_admin/rosa-multi-arch-cluster-managing.adoc b/rosa_cluster_admin/rosa-multi-arch-cluster-managing.adoc new file mode 100644 index 000000000000..608d725f00c8 --- /dev/null +++ b/rosa_cluster_admin/rosa-multi-arch-cluster-managing.adoc @@ -0,0 +1,20 @@ +:_mod-docs-content-type: ASSEMBLY +//This assembly duplicates the following file to avoid symbolic links: +// * post_installation_configuration/configuring-multi-arch-compute-machines/multi-architecture-compute-managing.adoc + +:context: rosa-multi-arch-compute-managing +[id="rosa-multi-arch-managing"] += Managing a cluster with multi-architecture compute machines +include::_attributes/common-attributes.adoc[] + +toc::[] + +Managing a cluster that has nodes with multiple architectures requires you to consider node architecture as you monitor the cluster and manage your workloads. This requires you to take additional considerations into account when you schedule workloads in a multi-architecture cluster. + +include::modules/multi-architecture-scheduling.adoc[leveloffset=+1] + +include::modules/multi-architecture-scheduling-examples.adoc[leveloffset=+2] + +// TODO OSDOCS-11411: Add back when HCP split in Nodes doc is complete +// .Additional resources +// * xref ../nodes/scheduling/nodes-scheduler-node-affinity.adoc#nodes-scheduler-node-affinity[Controlling pod placement on nodes using node affinity] \ No newline at end of file diff --git a/snippets/multi-arch-schedule-affinity-taint-toleration.adoc b/snippets/multi-arch-schedule-affinity-taint-toleration.adoc new file mode 100644 index 000000000000..b6f4b7ccbd5d --- /dev/null +++ b/snippets/multi-arch-schedule-affinity-taint-toleration.adoc @@ -0,0 +1,33 @@ +:_mod-docs-content-type: SNIPPET + +Using node affinity with taints and tolerations:: When a scheduler computes the set of nodes to schedule a pod, tolerations can broaden the set while node affinity restricts the set. If you set a taint on nodes that have a specific architecture, you must also add a toleration to workloads that you want to be scheduled there. ++ +.Example deployment with node affinity and toleration set +-- +[source,yaml] +---- +apiVersion: apps/v1 +kind: Deployment +metadata: # ... +spec: + # ... + template: + # ... + spec: + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: kubernetes.io/arch + operator: In + values: + - amd64 + - arm64 + tolerations: + - key: "multiarch.openshift.io/arch" + value: "arm64" + operator: "Equal" + effect: "NoSchedule" +---- +-- diff --git a/snippets/multi-arch-schedule-default-toleration-namespace.adoc b/snippets/multi-arch-schedule-default-toleration-namespace.adoc new file mode 100644 index 000000000000..be4d2bee293a --- /dev/null +++ b/snippets/multi-arch-schedule-default-toleration-namespace.adoc @@ -0,0 +1,12 @@ +:_mod-docs-content-type: SNIPPET + +Creating a default toleration in a namespace:: When a node or machine set has a taint, only workloads that tolerate that taint can be scheduled. You can annotate a namespace so all of the workloads get the same default toleration by running the following command: ++ +.Example default toleration set on a namespace +-- +[source,terminal] +---- +$ oc annotate namespace my-namespace \ + 'scheduler.alpha.kubernetes.io/defaultTolerations'='[{"operator": "Exists", "effect": "NoSchedule", "key": "multiarch.openshift.io/arch"}]' +---- +-- diff --git a/snippets/multi-arch-schedule-nodeaffinity.adoc b/snippets/multi-arch-schedule-nodeaffinity.adoc new file mode 100644 index 000000000000..591398f38f23 --- /dev/null +++ b/snippets/multi-arch-schedule-nodeaffinity.adoc @@ -0,0 +1,30 @@ +:_mod-docs-content-type: SNIPPET + +Using `nodeAffinity` to schedule nodes with specific architectures:: You can allow a workload to be scheduled on only a set of nodes with architectures supported by its images, you can set the `spec.affinity.nodeAffinity` field in your pod's template specification. ++ +.Example deployment with node affinity set +-- +[source,yaml] +---- +apiVersion: apps/v1 +kind: Deployment +metadata: # ... +spec: + # ... + template: + # ... + spec: + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: kubernetes.io/arch + operator: In + values: <1> + - amd64 + - arm64 +---- +<1> Specify the supported architectures. Valid values include `amd64`,`arm64`, or both values. +[source,yaml] +-- diff --git a/snippets/multi-arch-schedule-taint.adoc b/snippets/multi-arch-schedule-taint.adoc new file mode 100644 index 000000000000..d3dea3ceb167 --- /dev/null +++ b/snippets/multi-arch-schedule-taint.adoc @@ -0,0 +1,32 @@ +:_mod-docs-content-type: SNIPPET + +Tainting each node for a specific architecture:: You can taint a node to avoid the node scheduling workloads that are incompatible with its architecture. When your cluster uses a `MachineSet` object, you can add parameters to the `.spec.template.spec.taints` field to avoid workloads being scheduled on nodes with non-supported architectures. ++ +Before you add a taint to a node, you must scale down the `MachineSet` object or remove existing available machines. For more information, see _Modifying a compute machine set_. ++ +.Example machine set with taint set +-- +[source,yaml] +---- +apiVersion: machine.openshift.io/v1beta1 +kind: MachineSet +metadata: # ... +spec: + # ... + template: + # ... + spec: + # ... + taints: + - effect: NoSchedule + key: multiarch.openshift.io/arch + value: arm64 +---- +-- ++ +You can also set a taint on a specific node by running the following command: ++ +[source,terminal] +---- +$ oc adm taint nodes multiarch.openshift.io/arch=arm64:NoSchedule +---- diff --git a/snippets/multi-arch-schedule-toleration.adoc b/snippets/multi-arch-schedule-toleration.adoc new file mode 100644 index 000000000000..558ec5eea5ac --- /dev/null +++ b/snippets/multi-arch-schedule-toleration.adoc @@ -0,0 +1,24 @@ +:_mod-docs-content-type: SNIPPET + +Tolerating architecture taints in workloads:: When a node or machine set has a taint, only workloads that tolerate that taint can be scheduled. You can configure your workload with a `toleration` so that it is scheduled on nodes with specific architecture taints. ++ +.Example deployment with toleration set +-- +[source,yaml] +---- +apiVersion: apps/v1 +kind: Deployment +metadata: # ... +spec: + # ... + template: + # ... + spec: + tolerations: + - key: "multiarch.openshift.io/arch" + value: "arm64" + operator: "Equal" + effect: "NoSchedule" +---- +This example deployment can be scheduled on nodes and machine sets that have the `multiarch.openshift.io/arch=arm64` taint specified. +--