Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -497,6 +497,7 @@ include::cli:partial$cbcli/nav.adoc[]
*** xref:rest-api:backup-delete-plan.adoc[Delete a Plan]
*** xref:rest-api:backup-get-cluster-info.adoc[Get Information on the Cluster]
*** xref:rest-api:backup-manage-config.adoc[Manage Backup Configuration]
*** xref:rest-api:backup-node-threads.adoc[]

** xref:rest-api:rest-fts.adoc[Search Service API]
*** xref:rest-api:rest-fts-node.adoc[Node Configuration]
Expand Down
4 changes: 4 additions & 0 deletions modules/introduction/partials/new-features-76_2.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,7 @@
** xref:rest-api:backup-get-task-info.adoc[`/api/v1/cluster/self/repository/{repo-state}/{task-name}/taskHistory`]
** xref:rest-api:backup-get-plan-info.adoc[`/api/v1/plan/`]

* You can now set the number of threads each Backup Service node uses when backing up data.
For example, if you find backups cause performance issues on your cluster, you can reduce the number of threads the Backup Service uses.
Reducing the number of threads also reduces the number of concurrent client connections the Backup Service makes to retrieve data.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're mentioning the advantage of reducing the threads on the resources, i think it could be beneficial in mentioning that this will increase the time taken by the backup. The tradeoff is resources or time, and this threads setting gives the customer the choice to fine tune it based on their system and constraints.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. However, I think it would be better to add that to the Thread Usage section in the Backup Service instead of the What's New section. I've made that update and will rebuild the preview site.

See xref:learn:services-and-indexes/services/backup-service.adoc#threads[Thread Usage] for more information.
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= Backup Service
:description: pass:q[The Backup Service schedules full and incremental data backups and merges of previous data-backups.]
:stem: latexmath

[abstract]
{description}
Expand Down Expand Up @@ -195,6 +196,23 @@ When you use 0 to indicate today, the range starts from the time the scheduled
* Example B sets `merge_offset_start` to 4 (four days before today) and `merge_offset_end` to 3 (7 days ago, which is three days before the specified `merge_offset_start`).
Therefore, if today is March 15, the time range is from March 11 to March 8, with both the start and end days included entirely.

[#threads]
== Thread Usage

By default, the Backup Service chooses the number of threads it uses based on the number of CPU cores in the node.
The Backup Service creates one client connection per thread to connect in parallel to nodes when retrieving their data.
The more threads the service uses, the faster it retrieves data.
The formula for the number of threads it uses is stem:[\max(1, cpu\_cores \times 0.75)] (stem:[\tfrac{3}{4}] the number of CPU cores in the system or 1, whichever is larger).

In some cases, you may find that the default number of threads the Backup Service is using is too high.
Depending on resources, a large number of threads can cause performance issues or out-of-memory errors.
If your cluster is experiencing these issues during backup, you can change the number of threads a Backup Service node uses.
You change this setting using the `nodesThreadsMap` REST API endpoint.
See xref:rest-api:backup-node-threads.adoc[] for details on setting the number of threads the Backup Service uses.

Reducing the number of threads the Backup Service uses increases the time it takes to backup data.
You may want to experiment with different thread settings to find a balance between resource use and backup duration for your cluster.

[#see-also]
== See Also

Expand Down
183 changes: 183 additions & 0 deletions modules/rest-api/pages/backup-node-threads.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
= Manage Backup Service Threads
:description: You can change the number of threads a Backup Service node uses when backing up data.
:stem: latexmath

[abstract]
{description}

== HTTP Methods and URIs

.Get all overrides to the thread Backup Service settings
[source, uri]
----
GET /api/v1/nodesThreadsMap
----

.Overwrite all thread settings with new values
[source, uri]
----
POST /api/v1/nodesThreadsMap
----

.Update some thread settings
[source, uri]
----
PATCH /api/v1/nodesThreadsMap
----

== Description

The `nodesThreadMap` endpoint lets you change the number of threads the Backup Service uses on a node.
The default number of threads the Backup Service uses is based on the number of CPU cores in the node: stem:[\max(1, cpu\_cores \times 0.75)].
The number of threads also sets the number of concurrent client connections the service uses to retrieve data from nodes in the cluster.
Each thread creates one connection.
See xref:learn:services-and-indexes/services/backup-service.adoc#threads[Thread Usage] for more information about how the number of threads affects the Backup Service.

== Curl Syntax

.Get the current thread overrides
[source, console]
----
curl -u $USER:$PASSWORD -X GET \
http://$BACKUP_SERVICE_NODE:$BACKUP_SERVICE_PORT/api/v1/nodesThreads
----

.Overwrite all thread settings
[source, console]
----
curl -u Administrator:password -X POST \
http://$BACKUP_SERVICE_NODE:$BACKUP_SERVICE_PORT/api/v1/nodesThreadsMap \
-d <nodes_thread_map>
----

.Update/set some thread settings
[source, console]
----
curl -u Administrator:password -X PATCH \
http://$BACKUP_SERVICE_NODE:$BACKUP_SERVICE_PORT/api/v1/nodesThreadsMap \
-d <nodes_thread_map>
----

.POST Parameters
[cols="2,3,2"]
|===
|Name | Description | Schema

| `nodes_thread_map`
| An object that sets the number of threads for Backup Service nodes.
When you use the `PATCH` method, the changes only apply to the nodes in the map.
Any existing settings that are not in the map remain in effect.
Calling `POST`, removes any existing overrides.
After the call, only the overrides you supply in the map are in effect.
| <<nodes_thread_map_schema,Node Threads Map>>

|===

[#nodes_thread_map_schema]
.Node Thread Map Schema
[source, json]
----
{"nodes_threads_map": {
<backup_node_uuid>:<threads>, . . .
}
}
----

[cols="2,3,2"]
|===
|Name | Description | Schema

| `backup_node_uuid`
| The unique identifier for a node running the Backup Service.
You can get this value from the `/pools/nodes` REST API.
See xref:rest-api:rest-node-get-info.adoc[].
| string

| `threads`
a| The number of threads for the Backup Service to use.

When set to `0`, the Backup Service uses the default number of threads based on the number of CPU cores in the node: stem:[\max(1, cpu\_cores \times 0.75)].
| integer

|===


== Responses
[cols="1,3"]
|===
| Value | Description

| `200 OK`
a| Successful calls to `POST` and `PATCH` just return the response code.

When calling the `GET` method, you receive a JSON object mapping the node UUIDs to thread values.
See <<examples,Examples>> for details.

|`403 Forbidden`
a| User does not have the proper permission to call the API endpoint.

In addition, the call returns a JSON object similar to the following:

[source,json]
----
{
"message": "Forbidden. User needs one of the following permissions",
"permissions": [
"ro_admin"
]
}
----

| `404 Not Found`
a| The resource was not found.

If you call the `GET` method before you have created a nodes threads map by calling `POST` or `PATCH`, you also receive the following JSON message:

[source, json]
----
{
"status":404,
"msg":"Could not find the Nodes Threads Map",
"extras":"element not found"
}
----

|===

== Required Permissions

* `GET`: Full Admin, Backup Full Admin, Read-Only Admin
* `POST` and `PATCH`: Full Admin, Backup Full Admin

[#examples]
== Examples

.Set a backup service node to use a single thread, overwriting any existing overrides
[source, console]
----
curl -s -u Administrator:password -X \
POST http://localhost:8097/api/v1/nodesThreadsMap \
-d '{"nodes_threads_map":{"cb5c77719df4f33131251afdca00531a":1}}'
----

.Get the thread settings
[source, console]
----
curl -s -u Administrator:password -X \
GET http://node3:8097/api/v1/nodesThreadsMap | jq
----

The previous example returns output similar to the following:

[source,json]
----
{
"cb5c77719df4f33131251afdca00531a": 1
}
----

== See Also

* For a an overview of the Backup Service, see xref:learn:services-and-indexes/services/backup-service.adoc[Backup Service].
* For a step-by-step guide to configure and use the Backup Service using the Couchbase Server Web Console, see xref:manage:manage-backup-and-restore/manage-backup-and-restore.adoc[Manage Backup and Restore].
* See xref:learn:services-and-indexes/services/backup-service.adoc#threads[Thread Usage] for more information about how the number of threads affects the Backup Service.
14 changes: 11 additions & 3 deletions modules/rest-api/pages/backup-rest-api.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,17 @@

== APIs in this Section

The Backup Service API provides endpoints categorized as follows: _Cluster_, _Configuration_, _Repository_, _Plan_, _Task_, and _Data_.
For a conceptual overview of the Backup Service, see xref:learn:services-and-indexes/services/backup-service.adoc[Backup Service].
For information on using Couchbase Web Console to configure and use the Backup Service, see xref:manage:manage-backup-and-restore/manage-backup-and-restore.adoc[Manage Backup and Restore].
The Backup Service API endpoints can be grouped into several categories:

* xref:#cluster[Cluster]
* xref:#configuration[Configuration]
* xref:#repository[Repository]
* xref:#plan[Plan]
* xref:#task[Task]
* xref:#data[Data]

For an overview of the Backup Service, see xref:learn:services-and-indexes/services/backup-service.adoc[Backup Service].
For information on using Couchbase Server Web Console to configure and use the Backup Service, see xref:manage:manage-backup-and-restore/manage-backup-and-restore.adoc[Manage Backup and Restore].

All calls require the Full Admin role, and use port `8097`.
Each URI, in Couchbase Server Enterprise Edition Version 7.0, must be prefixed with `/api/v1`.
Expand Down
Loading