Skip to content

Oadp velero pod affinity #1690

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 15, 2025
+320 −3
Merged

Oadp velero pod affinity #1690

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions api/v1alpha1/dataprotectionapplication_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,9 @@ type VeleroConfig struct {
// Velero args are settings to customize velero server arguments. Overrides values in other fields.
// +optional
Args *VeleroServerArgs `json:"args,omitempty"`
// LoadAffinityConfig is the config for data path load affinity.
// +optional
LoadAffinityConfig []*LoadAffinity `json:"loadAffinity,omitempty"`
}

// PodConfig defines the pod configuration options
Expand Down
11 changes: 11 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

53 changes: 53 additions & 0 deletions bundle/manifests/oadp.openshift.io_dataprotectionapplications.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1079,6 +1079,59 @@ spec:
itemOperationSyncFrequency:
description: How often to check status on async backup/restore operations after backup processing. Default value is 2m.
type: string
loadAffinity:
description: LoadAffinityConfig is the config for data path load affinity.
items:
description: |-
LoadAffinity is the config for data path load affinity.
Used by the Node-Agent, that needs to match the DataMover and the RepositoryMaintenance pods.
properties:
nodeSelector:
description: NodeSelector specifies the label selector to match nodes
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
type: object
type: array
logLevel:
description: Velero server's log level (use debug for the most logging, leave unset for velero default)
enum:
Expand Down
53 changes: 53 additions & 0 deletions config/crd/bases/oadp.openshift.io_dataprotectionapplications.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1079,6 +1079,59 @@ spec:
itemOperationSyncFrequency:
description: How often to check status on async backup/restore operations after backup processing. Default value is 2m.
type: string
loadAffinity:
description: LoadAffinityConfig is the config for data path load affinity.
items:
description: |-
LoadAffinity is the config for data path load affinity.
Used by the Node-Agent, that needs to match the DataMover and the RepositoryMaintenance pods.
properties:
nodeSelector:
description: NodeSelector specifies the label selector to match nodes
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
type: object
type: array
logLevel:
description: Velero server's log level (use debug for the most logging, leave unset for velero default)
enum:
Expand Down
10 changes: 7 additions & 3 deletions docs/design/DPA-and-node-selectors.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,12 @@ This design enhances flexibility by allowing Affinity and Anti-Affinity settings
- Guarantee that DataMover pods are scheduled on the same nodes as NodeAgent pods.
- Ensure backward compatibility with existing configurations.
- Ensure user experience is not degraded by the changes by providing Reconcile warnings and errors when the existing node selector configurations are used and may cause issues with the new Affinity and Anti-Affinity settings.
- Extend current Velero specific `spec.configuration.velero.podConfig` with the `spec.configuration.velero.loadAffinity` DPA CRD. The current nodeSelector from the `spec.configuration.velero.podConfig` will be kept as is.
The new `spec.configuration.velero.loadAffinity` setting will be used to configure the Velero pod affinity settings and applied to the `veleroDeployment.Spec.Template.Spec.Affinity`.

## Non Goals
- Removing or deprecating existing node selector configurations.
- Use advanced Affinity settings that are not specified in teh Veleros' loadAffinity CRD such as `requiredDuringSchedulingIgnoredDuringExecution` and `preferredDuringSchedulingIgnoredDuringExecution`.
- Extend current Velero specific `spec.configuration.velero.podConfig` with the `spec.configuration.velero.loadAffinity` DPA CRD. The current nodeSelector from the `spec.configuration.velero.podConfig` will be used to schedule the Velero pod.
- Use advanced Affinity settings that are not specified in the Veleros' loadAffinity CRD such as `requiredDuringSchedulingIgnoredDuringExecution` and `preferredDuringSchedulingIgnoredDuringExecution`.

## High-Level Design
The proposed change introduces a unified nodeSelector structure for all relevant components within the DPA custom resource.
Expand Down Expand Up @@ -229,7 +230,10 @@ The `spec.configuration.velero.loadAffinity` section will be used to schedule th
The use of `spec.configuration.velero.loadAffinity` will **NOT** be validated against the `spec.configuration.velero.podConfig.nodeSelector` field. This is not required restriction, because the `spec.configuration.velero.loadAffinity` is not used only to schedule the Velero pod and no dependent workloads are scheduled using the **ConfigMap** mechanism.

### Velero `spec.configuration.velero.loadAffinity` field
The `spec.configuration.velero.loadAffinity` will be added to the DPA CRD.
The `spec.configuration.velero.loadAffinity` field will be added to the DPA CRD to allow configuring affinity and anti-affinity settings for the Velero pod. This configuration will be applied to `veleroDeployment.Spec.Template.Spec.Affinity`.

Velero pod scheduling can also be influenced by the `spec.configuration.velero.podConfig.nodeSelector` field. When both fields are used, the Velero pod will be scheduled according to standard OpenShift behavior, which supports combined use of **affinity** and **nodeSelector** to control Velero pod placement.

The updated schema will be as follows:

```yaml
Expand Down
12 changes: 12 additions & 0 deletions internal/controller/velero.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (
"github.com/sirupsen/logrus"
"github.com/vmware-tanzu/velero/pkg/install"
"github.com/vmware-tanzu/velero/pkg/util/boolptr"
"github.com/vmware-tanzu/velero/pkg/util/kube"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"
Expand Down Expand Up @@ -253,6 +254,17 @@ func (r *DataProtectionApplicationReconciler) customizeVeleroDeployment(veleroDe
veleroDeployment.Spec.Template.Spec.NodeSelector = dpa.Spec.Configuration.Velero.PodConfig.NodeSelector
}
}

if dpa.Spec.Configuration.Velero.LoadAffinityConfig != nil {
veleroAffinityStruct := make([]*kube.LoadAffinity, len(dpa.Spec.Configuration.Velero.LoadAffinityConfig))

for i, aff := range dpa.Spec.Configuration.Velero.LoadAffinityConfig {
veleroAffinityStruct[i] = (*kube.LoadAffinity)(aff)
}
affinity := kube.ToSystemAffinity(veleroAffinityStruct)
veleroDeployment.Spec.Template.Spec.Affinity = affinity
}

veleroDeployment.Spec.Template.Spec.Volumes = append(veleroDeployment.Spec.Template.Spec.Volumes,
corev1.Volume{
Name: "certs",
Expand Down
Loading