diff --git a/modules/hcp-topology-spread-constraint.adoc b/modules/hcp-topology-spread-constraint.adoc index aff970371ad6..97f2d3ba5f4a 100644 --- a/modules/hcp-topology-spread-constraint.adoc +++ b/modules/hcp-topology-spread-constraint.adoc @@ -16,7 +16,7 @@ In some scenarios, node pool VMs might run on the same node, which can cause ava .Procedure -* Open the `KubeDescheduler` custom resource (CR) by entering the following command, and then modify the `KubeDescheduler` CR to use the `SoftTopologyAndDuplicates` and `DevKubeVirtRelieveAndMigrate` profiles so that you maintain the `topologySpreadConstraint` constraint settings. +* Open the `KubeDescheduler` custom resource (CR) by entering the following command, and then modify the `KubeDescheduler` CR to use the `SoftTopologyAndDuplicates` and `KubeVirtRelieveAndMigrate` profiles so that you maintain the `topologySpreadConstraint` constraint settings. + The `KubeDescheduler` CR named `cluster` runs in the `openshift-kube-descheduler-operator` namespace. + @@ -39,7 +39,7 @@ spec: deschedulingIntervalSeconds: 30 # <1> profiles: - SoftTopologyAndDuplicates # <2> - - DevKubeVirtRelieveAndMigrate # <3> + - KubeVirtRelieveAndMigrate # <3> profileCustomizations: devDeviationThresholds: AsymmetricLow devActualUtilizationProfile: PrometheusCPUCombined diff --git a/modules/nodes-descheduler-profiles.adoc b/modules/nodes-descheduler-profiles.adoc index 2ebf9080456b..b461be8466a8 100644 --- a/modules/nodes-descheduler-profiles.adoc +++ b/modules/nodes-descheduler-profiles.adoc @@ -90,23 +90,17 @@ Do not enable `CompactAndScale` with any of the following profiles: `LifecycleAn endif::nodes[] ifdef::virt[] -Use the `DevKubeVirtRelieveAndMigrate` or `LongLifecycle` profile to enable the descheduler on a virtual machine. +Use the `KubeVirtRelieveAndMigrate` or `LongLifecycle` profile to enable the descheduler on a virtual machine. [IMPORTANT] ==== -You can not have both `DevKubeVirtRelieveAndMigrate` and `LongLifeCycle` enabled at the same time. +You cannot have both `KubeVirtRelieveAndMigrate` and `LongLifeCycle` enabled at the same time. ==== endif::virt[] -`DevKubeVirtRelieveAndMigrate`:: This profile is an enhanced version of the `LongLifeCycle` profile. -+ --- -:FeatureName: The `DevKubeVirtRelieveAndMigrate` profile -include::snippets/technology-preview.adoc[] -:!FeatureName: --- +`KubeVirtRelieveAndMigrate`:: This profile is an enhanced version of the `LongLifeCycle` profile. -The `DevKubeVirtRelieveAndMigrate` profile evicts pods from high-cost nodes to reduce overall resource expenses and enable workload migration. It also periodically rebalances workloads to help maintain similar spare capacity across nodes, which supports better handling of sudden workload spikes. Nodes can experience the following costs: +The `KubeVirtRelieveAndMigrate` profile evicts pods from high-cost nodes to reduce overall resource expenses and enable workload migration. It also periodically rebalances workloads to help maintain similar spare capacity across nodes, which supports better handling of sudden workload spikes. Nodes can experience the following costs: -- * **Resource utilization**: Increased resource pressure raises the overhead for running applications. @@ -132,14 +126,14 @@ spec: deschedulingIntervalSeconds: 30 mode: "Automatic" profiles: - - DevKubeVirtRelieveAndMigrate + - KubeVirtRelieveAndMigrate profileCustomizations: devEnableSoftTainter: true devDeviationThresholds: AsymmetricLow devActualUtilizationProfile: PrometheusCPUCombined ---- -The `DevKubeVirtRelieveAndMigrate` profile requires PSI metrics to be enabled on all worker nodes. You can enable this by applying the following `MachineConfig` custom resource (CR): +The `KubeVirtRelieveAndMigrate` profile requires PSI metrics to be enabled on all worker nodes. You can enable this by applying the following `MachineConfig` custom resource (CR): .Example `MachineConfig` CR [source,yaml] @@ -155,6 +149,11 @@ spec: - psi=1 ---- +[NOTE] +==== +The name of the `MachineConfig` object is significant because machine configs are processed in lexicographical order. By default, a config that starts with `98-` disables PSI. To ensure that PSI is enabled, name your config with a higher prefix, such as `99-openshift-machineconfig-worker-psi-karg`. +==== + You can use this profile with the `SoftTopologyAndDuplicates` profile to also rebalance pods based on soft topology constraints, which can be useful in hosted control plane environments. // Show LongLifecycle profile both for virt and nodes diff --git a/modules/virt-configuring-descheduler-evictions.adoc b/modules/virt-configuring-descheduler-evictions.adoc new file mode 100644 index 000000000000..2521b6f00c01 --- /dev/null +++ b/modules/virt-configuring-descheduler-evictions.adoc @@ -0,0 +1,76 @@ +// Module included in the following assemblies: +// +// virt/virtual_machines/advanced_vm_management/virt-enabling-descheduler-evictions.adoc + +:_mod-docs-content-type: PROCEDURE +[id="virt-configuring-descheduler-evictions_{context}"] += Configuring descheduler evictions for virtual machines + +After the descheduler is installed and configured, all migratable virtual machines (VMs) are eligible for eviction by default. You can configure the descheduler to manage VM evictions across the cluster and optionally exclude specific VMs from eviction. + +.Prerequisites + +* Install the descheduler in the {product-title} web console or OpenShift CLI (`oc`). + +.Procedure + +. Stop the VM. + +. Configure the `KubeDescheduler` object with the `KubeVirtRelieveAndMigrate` profile and enable background evictions for improved VM eviction stability during live migration: ++ +[source,yaml] +---- +apiVersion: operator.openshift.io/v1 +kind: KubeDescheduler +metadata: + name: cluster + namespace: openshift-kube-descheduler-operator +spec: + deschedulingIntervalSeconds: 60 + profiles: + - KubeVirtRelieveAndMigrate + mode: Automatic +---- ++ +. Optional: To evict pods, set the `mode` field value to `Automatic`. By default, the descheduler does not evict pods. + +. Optional: Configure limits for the number of parallel evictions to improve stability in large clusters. ++ +The descheduler can limit the number of concurrent evictions per node and across the cluster by using the `evictionLimits` field. Set these limits to match the migration limits configured in the `HyperConverged` custom resource (CR). ++ +[source,yaml] +---- +spec: + evictionLimits: + node: 2 + total: 5 +---- ++ +Set values that correspond to the migration limits in the `HyperConverged` CR: ++ +[source,yaml] +---- +spec: + liveMigrationConfig: + parallelMigrationsPerCluster: 5 + parallelOutboundMigrationsPerNode: 2 +---- + +. Optional: To exclude the VM from eviction, add the `descheduler.alpha.kubernetes.io/prefer-no-eviction` annotation to the `spec.template.metadata.annotations` field. The change is applied dynamically and is propagated to the `VirtualMachineInstance` (VMI) object and the `virt-launcher` pod. ++ +Only the presence of the annotation is checked. The value is not evaluated, so `"true"` and `"false"` have the same effect. ++ +[source,yaml] +---- +apiVersion: kubevirt.io/v1 +kind: VirtualMachine +spec: + template: + metadata: + annotations: + descheduler.alpha.kubernetes.io/prefer-no-eviction: "true" +---- + +. Start the VM. + +The VM is now configured according to the descheduler settings. diff --git a/modules/virt-enabling-descheduler-evictions.adoc b/modules/virt-enabling-descheduler-evictions.adoc deleted file mode 100644 index 566044ca0a0b..000000000000 --- a/modules/virt-enabling-descheduler-evictions.adoc +++ /dev/null @@ -1,52 +0,0 @@ -// Module included in the following assemblies: -// -// virt/virtual_machines/advanced_vm_management/virt-enabling-descheduler-evictions.adoc - -:_mod-docs-content-type: PROCEDURE -[id="virt-enabling-descheduler-evictions_{context}"] -= Enabling descheduler evictions on a virtual machine (VM) - -After the descheduler is installed, you can enable descheduler evictions on your VM by adding an annotation to the `VirtualMachine` custom resource (CR). - -.Prerequisites - -* Install the descheduler in the {product-title} web console or OpenShift CLI (`oc`). -* Ensure that the VM is not running. - -.Procedure - -. Before starting the VM, add the `descheduler.alpha.kubernetes.io/evict` annotation to the `VirtualMachine` CR: -+ -[source,yaml] ----- -apiVersion: kubevirt.io/v1 -kind: VirtualMachine -spec: - template: - metadata: - annotations: - descheduler.alpha.kubernetes.io/evict: "true" ----- - -. Configure the `KubeDescheduler` object with the `LongLifecycle` profile and enable background evictions for improved VM eviction stability during live migration: -+ -[source,yaml] ----- -apiVersion: operator.openshift.io/v1 -kind: KubeDescheduler -metadata: - name: cluster - namespace: openshift-kube-descheduler-operator -spec: - deschedulingIntervalSeconds: 3600 - profiles: - - LongLifecycle <1> - mode: Predictive <2> - profileCustomizations: - devEnableEvictionsInBackground: true <3> ----- -<1> You can only set the `LongLifecycle` profile. This profile balances resource usage between nodes. -<2> By default, the descheduler does not evict pods. To evict pods, set `mode` to `Automatic`. -<3> Enabling `devEnableEvictionsInBackground` allows evictions to occur in the background, improving stability and mitigating oscillatory behavior during live migrations. - -The descheduler is now enabled on the VM. \ No newline at end of file diff --git a/virt/managing_vms/advanced_vm_management/virt-enabling-descheduler-evictions.adoc b/virt/managing_vms/advanced_vm_management/virt-enabling-descheduler-evictions.adoc index 73329a27f627..a7add269849a 100644 --- a/virt/managing_vms/advanced_vm_management/virt-enabling-descheduler-evictions.adoc +++ b/virt/managing_vms/advanced_vm_management/virt-enabling-descheduler-evictions.adoc @@ -14,7 +14,7 @@ include::modules/nodes-descheduler-profiles.adoc[leveloffset=+1] include::modules/nodes-descheduler-installing.adoc[leveloffset=+1] -include::modules/virt-enabling-descheduler-evictions.adoc[leveloffset=+1] +include::modules/virt-configuring-descheduler-evictions.adoc[leveloffset=+1] [role="_additional-resources"] [id="additional-resources_enabling-descheduler-evictions"]