Skip to content

WIP - OADP-3261: Performance & Scale for OADP 1.3.0 #69703

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

WIP - OADP-3261: Performance & Scale for OADP 1.3.0 #69703

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions _attributes/common-attributes.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,10 @@ endif::openshift-origin[]
:insights-advisor-url: link:https://console.redhat.com/openshift/insights/advisor/[Insights Advisor]
:hybrid-console: Red Hat Hybrid Cloud Console
:hybrid-console-second: Hybrid Cloud Console
// OADP
:oadp-first: OpenShift API for Data Protection (OADP)
:oadp-full: OpenShift API for Data Protection
:oadp-short: OADP
:oc-first: pass:quotes[OpenShift CLI (`oc`)]
:product-registry: OpenShift image registry
:rh-storage-first: Red Hat OpenShift Data Foundation
Expand Down Expand Up @@ -253,3 +255,7 @@ endif::[]
:coo-first: Cluster Observability Operator (COO)
:coo-full: Cluster Observability Operator
:coo-short: COO
//ODF
:odf-first: Red Hat OpenShift Data Foundation (ODF)
:odf-full: Red Hat OpenShift Data Foundation
:odf: ODF
1 change: 1 addition & 0 deletions ...nd_restore/application_backup_and_restore/installing/about-installing-oadp.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,4 @@ include::modules/about-installing-oadp-on-multiple-namespaces.adoc[leveloffset=+
* xref:../../../operators/understanding/olm/olm-understanding-olm.adoc#olm-csv_olm-understanding-olm[Cluster service version]

include::modules/oadp-velero-cpu-memory-requirements.adoc[leveloffset=+1]
include::modules/oadp-backup-restore-for-large-usage.adoc[leveloffset=+2]
1 change: 1 addition & 0 deletions ..._and_restore/application_backup_and_restore/installing/installing-oadp-ocs.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ include::modules/oadp-secrets-for-different-credentials.adoc[leveloffset=+2]
You can configure the Data Protection Application by setting Velero resource allocations or enabling self-signed CA certificates.

include::modules/oadp-setting-resource-limits-and-requests.adoc[leveloffset=+2]
include::modules/oadp-odf-cpu-memory-requirements.adoc[leveloffset=+3]
include::modules/oadp-self-signed-certificate.adoc[leveloffset=+2]

include::modules/oadp-installing-dpa-1-2-and-earlier.adoc[leveloffset=+1]
Expand Down
16 changes: 16 additions & 0 deletions modules/oadp-backup-restore-for-large-usage.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
// Module included in the following assemblies:
// about-installing-oadp.adoc

:_mod-docs-content-type: CONCEPT
[id="oadp-backup-restore-for-large-usage_{context}"]
= NodeAgent CPU for large usage

Increasing `NodeAgent` CPU shows a significant impact on improving backup and restore times when using {oadp}.

It is not recommended to use Kopia without limits in production environments on nodes running production workloads due to Kopia’s aggressive consumption of resources.

However running Kopia with limits that are too low results in CPU limiting and slow-performing backups and restore situations.

The testing undertaken, observed that running with around 20 cores limit, and 32 Gi memory limit covered most of our largest cases effectively.

Testing detected no CPU limiting or memory saturation with these resource specifications.
26 changes: 26 additions & 0 deletions modules/oadp-odf-cpu-memory-requirements.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
// Module included in the following assemblies:
//
// * backup_and_restore/application_backup_and_restore/installing/installing-oadp-ocs.adoc

:_mod-docs-content-type: CONCEPT
[id="oadp-odf-cpu-memory-requirements_{context}"]
= Adjust Ceph CPU and memory requirements based on collected data

The following recommendations are based on observations of performance made in the scale and performance lab. The changes are specifically related to {odf-first}. If working with {odf}, consult the appropriate tuning guides for official recommendations.

[id="oadp-odf-config-cpu-memory-requirements_{context}"]
== CPU and memory requirement for configurations

Backup and restore operations require large amounts of Ceph-FS `PersistentVolumes` (PVs), and to avoid Ceph MDS pods restarting with an `out-of-memory` (OOM) error, the following configuration is suggested:

|===
| Configuration types | Request | Max limit

| CPU
| Request changed to 3
| Max limit to 3

| Memory
| Request changed to 8Gi
| Max limit to 128Gi
|===