Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions modules/hcp-topology-spread-constraint.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ In some scenarios, node pool VMs might run on the same node, which can cause ava

.Procedure

* Open the `KubeDescheduler` custom resource (CR) by entering the following command, and then modify the `KubeDescheduler` CR to use the `SoftTopologyAndDuplicates` and `DevKubeVirtRelieveAndMigrate` profiles so that you maintain the `topologySpreadConstraint` constraint settings.
* Open the `KubeDescheduler` custom resource (CR) by entering the following command, and then modify the `KubeDescheduler` CR to use the `SoftTopologyAndDuplicates` and `KubeVirtRelieveAndMigrate` profiles so that you maintain the `topologySpreadConstraint` constraint settings.
+
The `KubeDescheduler` CR named `cluster` runs in the `openshift-kube-descheduler-operator` namespace.
+
Expand All @@ -39,7 +39,7 @@ spec:
deschedulingIntervalSeconds: 30 # <1>
profiles:
- SoftTopologyAndDuplicates # <2>
- DevKubeVirtRelieveAndMigrate # <3>
- KubeVirtRelieveAndMigrate # <3>
profileCustomizations:
devDeviationThresholds: AsymmetricLow
devActualUtilizationProfile: PrometheusCPUCombined
Expand Down
23 changes: 11 additions & 12 deletions modules/nodes-descheduler-profiles.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -90,23 +90,17 @@ Do not enable `CompactAndScale` with any of the following profiles: `LifecycleAn

endif::nodes[]
ifdef::virt[]
Use the `DevKubeVirtRelieveAndMigrate` or `LongLifecycle` profile to enable the descheduler on a virtual machine.
Use the `KubeVirtRelieveAndMigrate` or `LongLifecycle` profile to enable the descheduler on a virtual machine.

[IMPORTANT]
====
You can not have both `DevKubeVirtRelieveAndMigrate` and `LongLifeCycle` enabled at the same time.
You cannot have both `KubeVirtRelieveAndMigrate` and `LongLifeCycle` enabled at the same time.
====
endif::virt[]

`DevKubeVirtRelieveAndMigrate`:: This profile is an enhanced version of the `LongLifeCycle` profile.
+
--
:FeatureName: The `DevKubeVirtRelieveAndMigrate` profile
include::snippets/technology-preview.adoc[]
:!FeatureName:
--
`KubeVirtRelieveAndMigrate`:: This profile is an enhanced version of the `LongLifeCycle` profile.

The `DevKubeVirtRelieveAndMigrate` profile evicts pods from high-cost nodes to reduce overall resource expenses and enable workload migration. It also periodically rebalances workloads to help maintain similar spare capacity across nodes, which supports better handling of sudden workload spikes. Nodes can experience the following costs:
The `KubeVirtRelieveAndMigrate` profile evicts pods from high-cost nodes to reduce overall resource expenses and enable workload migration. It also periodically rebalances workloads to help maintain similar spare capacity across nodes, which supports better handling of sudden workload spikes. Nodes can experience the following costs:

--
* **Resource utilization**: Increased resource pressure raises the overhead for running applications.
Expand All @@ -132,14 +126,14 @@ spec:
deschedulingIntervalSeconds: 30
mode: "Automatic"
profiles:
- DevKubeVirtRelieveAndMigrate
- KubeVirtRelieveAndMigrate
profileCustomizations:
devEnableSoftTainter: true
devDeviationThresholds: AsymmetricLow
devActualUtilizationProfile: PrometheusCPUCombined
----

The `DevKubeVirtRelieveAndMigrate` profile requires PSI metrics to be enabled on all worker nodes. You can enable this by applying the following `MachineConfig` custom resource (CR):
The `KubeVirtRelieveAndMigrate` profile requires PSI metrics to be enabled on all worker nodes. You can enable this by applying the following `MachineConfig` custom resource (CR):

.Example `MachineConfig` CR
[source,yaml]
Expand All @@ -155,6 +149,11 @@ spec:
- psi=1
----

[NOTE]
====
The name of the `MachineConfig` object is significant because machine configs are processed in lexicographical order. By default, a config that starts with `98-` disables PSI. To ensure that PSI is enabled, name your config with a higher prefix, such as `99-openshift-machineconfig-worker-psi-karg`.
====

You can use this profile with the `SoftTopologyAndDuplicates` profile to also rebalance pods based on soft topology constraints, which can be useful in hosted control plane environments.

// Show LongLifecycle profile both for virt and nodes
Expand Down
76 changes: 76 additions & 0 deletions modules/virt-configuring-descheduler-evictions.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
// Module included in the following assemblies:
//
// virt/virtual_machines/advanced_vm_management/virt-enabling-descheduler-evictions.adoc

:_mod-docs-content-type: PROCEDURE
[id="virt-configuring-descheduler-evictions_{context}"]
= Configuring descheduler evictions for virtual machines

After the descheduler is installed and configured, all migratable virtual machines (VMs) are eligible for eviction by default. You can configure the descheduler to manage VM evictions across the cluster and optionally exclude specific VMs from eviction.

.Prerequisites

* Install the descheduler in the {product-title} web console or OpenShift CLI (`oc`).

.Procedure

. Stop the VM.

. Configure the `KubeDescheduler` object with the `KubeVirtRelieveAndMigrate` profile and enable background evictions for improved VM eviction stability during live migration:
+
[source,yaml]
----
apiVersion: operator.openshift.io/v1
kind: KubeDescheduler
metadata:
name: cluster
namespace: openshift-kube-descheduler-operator
spec:
deschedulingIntervalSeconds: 60
profiles:
- KubeVirtRelieveAndMigrate
mode: Automatic
----
+
. Optional: To evict pods, set the `mode` field value to `Automatic`. By default, the descheduler does not evict pods.

. Optional: Configure limits for the number of parallel evictions to improve stability in large clusters.
+
The descheduler can limit the number of concurrent evictions per node and across the cluster by using the `evictionLimits` field. Set these limits to match the migration limits configured in the `HyperConverged` custom resource (CR).
+
[source,yaml]
----
spec:
evictionLimits:
node: 2
total: 5
----
+
Set values that correspond to the migration limits in the `HyperConverged` CR:
+
[source,yaml]
----
spec:
liveMigrationConfig:
parallelMigrationsPerCluster: 5
parallelOutboundMigrationsPerNode: 2
----

. Optional: To exclude the VM from eviction, add the `descheduler.alpha.kubernetes.io/prefer-no-eviction` annotation to the `spec.template.metadata.annotations` field. The change is applied dynamically and is propagated to the `VirtualMachineInstance` (VMI) object and the `virt-launcher` pod.
+
Only the presence of the annotation is checked. The value is not evaluated, so `"true"` and `"false"` have the same effect.
+
[source,yaml]
----
apiVersion: kubevirt.io/v1
kind: VirtualMachine
spec:
template:
metadata:
annotations:
descheduler.alpha.kubernetes.io/prefer-no-eviction: "true"
----

. Start the VM.

The VM is now configured according to the descheduler settings.
52 changes: 0 additions & 52 deletions modules/virt-enabling-descheduler-evictions.adoc

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ include::modules/nodes-descheduler-profiles.adoc[leveloffset=+1]

include::modules/nodes-descheduler-installing.adoc[leveloffset=+1]

include::modules/virt-enabling-descheduler-evictions.adoc[leveloffset=+1]
include::modules/virt-configuring-descheduler-evictions.adoc[leveloffset=+1]

[role="_additional-resources"]
[id="additional-resources_enabling-descheduler-evictions"]
Expand Down