diff --git a/modules/learn/pages/services-and-indexes/services/backup-service.adoc b/modules/learn/pages/services-and-indexes/services/backup-service.adoc index 5bca27f8b7..c7ed1682d7 100644 --- a/modules/learn/pages/services-and-indexes/services/backup-service.adoc +++ b/modules/learn/pages/services-and-indexes/services/backup-service.adoc @@ -100,6 +100,10 @@ You can also search the repositories for individual documents that have been bac When restoring data from a backup, you can define filters to choose a subset of the data to restore. You can restore data to its original keyspace or apply a mapping to restore it to a different keyspace. +You do not have to restore backed up data to a bucket using the same xref:learn:buckets-memory-and-storage/storage-engines.adoc[storage engine] as the original bucket. +For example, you can restore data backed up from a bucket that used the Couchstore storage engine to a bucket using Magma. + + [#archiving-and-importing] == Archiving and Importing diff --git a/modules/manage/assets/images/manage-backup-restore/newBucketWithRestoredData.png b/modules/manage/assets/images/manage-backup-restore/newBucketWithRestoredData.png index f64aa73da1..24a7499553 100644 Binary files a/modules/manage/assets/images/manage-backup-restore/newBucketWithRestoredData.png and b/modules/manage/assets/images/manage-backup-restore/newBucketWithRestoredData.png differ diff --git a/modules/manage/assets/images/manage-backup-restore/restoreButton.png b/modules/manage/assets/images/manage-backup-restore/restoreButton.png deleted file mode 100644 index d37e36e91b..0000000000 Binary files a/modules/manage/assets/images/manage-backup-restore/restoreButton.png and /dev/null differ diff --git a/modules/manage/assets/images/manage-backup-restore/restoreDialog.png b/modules/manage/assets/images/manage-backup-restore/restoreDialog.png index 36a158917a..710bf3c1ff 100644 Binary files a/modules/manage/assets/images/manage-backup-restore/restoreDialog.png and b/modules/manage/assets/images/manage-backup-restore/restoreDialog.png differ diff --git a/modules/manage/assets/images/manage-backup-restore/restoreDialogPartiallyComplete.png b/modules/manage/assets/images/manage-backup-restore/restoreDialogPartiallyComplete.png deleted file mode 100644 index a2d5181e1e..0000000000 Binary files a/modules/manage/assets/images/manage-backup-restore/restoreDialogPartiallyComplete.png and /dev/null differ diff --git a/modules/manage/pages/manage-backup-and-restore/manage-backup-and-restore.adoc b/modules/manage/pages/manage-backup-and-restore/manage-backup-and-restore.adoc index 0c0a1d12a7..b92c33ecc0 100644 --- a/modules/manage/pages/manage-backup-and-restore/manage-backup-and-restore.adoc +++ b/modules/manage/pages/manage-backup-and-restore/manage-backup-and-restore.adoc @@ -11,7 +11,7 @@ The data on a Couchbase-Server cluster can be backed up, restored, and archived by means of either of the following: * The *Backup Service*. -This can be configured by means of the *Backup* UI provided by Couchbase Web Console. +This can be configured by means of the *Backup* UI provided by Couchbase Server Web Console. * The xref:backup-restore:cbbackupmgr.adoc[cbbackupmgr] CLI utility. @@ -24,7 +24,7 @@ An overview of the Backup Service is provided in xref:learn:services-and-indexes === The Backup Service and cbbackupmgr Both the Backup Service and `cbbackupmgr` are included in Couchbase Server Enterprise Edition. -Note that from version 7.0, `cbbackupmgr` is also available in Community Edition, but without support for merge, cloud backup, or collection-level restore. +From version 7.0, `cbbackupmgr` is also available in Couchbase Server Community Edition, but without support for merge, cloud backup, or collection-level restore. The following paragraphs summarize the similarities and differences between the Backup Service and `cbbackupmgr` as provided by Enterprise Edition. @@ -34,20 +34,20 @@ For use of `cbbackupmgr`, the Full Admin or the Data Backup & Restore role must The Backup Service — which can be configured by means of the *Backup* facility of Couchbase Web Console, the Couchbase CLI, and the REST API — allows backup, restore, and archiving to be configured for the local cluster; and also permits restore to be configured for a remote cluster. By contrast, `cbbackupmgr` allows backup, restore, and archiving each to be configured either for the local or for a remote cluster: all available options are listed in xref:backup-restore:enterprise-backup-restore.adoc##version-compatibility[Version Compatibility]. -Whereas `cbbackupmgr` performs a specific backup or merge when executed, the Backup Service can be _scheduled_; so that backups and periodic merges are ongoing. +Whereas `cbbackupmgr` performs a specific backup or merge when executed, the Backup Service can be scheduled so that backups and periodic merges are ongoing. The Backup Service therefore supports additional and modified parameters, to allow scheduling to be configured. -Note that both the Backup Service and `cbbackupmgr` allow _full_ and _incremental_ backups. +Both the Backup Service and `cbbackupmgr` allow full and incremental backups. Unlike the Backup Service, `cbbackupmgr` requires a new repository to be created for each new, full backup (successive `cbbackupmgr` backups to the same repository being incremental). Both allow incremental backups, once created, to be merged, and their data deduplicated. Both use the same backup archive structure; allow the contents of backups to be listed; and allow specific documents to be searched for. Both the Backup Service and `cbbackupmgr` support use of AWS S3 storage. -Note that `cbbackupmgr` is available in both Couchbase Server 7.0 Enterprise Edition (_EE_) and 7.0 Community Edition (_CE_). +The `cbbackupmgr` tool is available in both Couchbase Server 7.0 Enterprise Edition (EE) and Couchbase Server Community Edition (CE). However, whereas in EE, `cbbackupmgr` allows backup and restore to be performed with reference to buckets, scopes, and collections; in CE, `cbbackupmgr` allows backup and restore to be performed with reference to buckets only. -For detailed information on how `cbbackupmgr` works (including a detailed description of incremental backup), see the xref:backup-restore:cbbackupmgr.adoc#discussion[Discussion] provided on the page for xref:backup-restore:cbbackupmgr.adoc[cbbackupmgr]. +For detailed information about how `cbbackupmgr` works (including a detailed description of incremental backup), see the xref:backup-restore:cbbackupmgr.adoc#discussion[Discussion] provided on the page for xref:backup-restore:cbbackupmgr.adoc[cbbackupmgr]. The page for xref:backup-restore:cbbackupmgr.adoc[cbbackupmgr] also provides a synopsis of the command, and a description of its basic options. The remainder of the current page describes how to configure and use the Backup Service, using Couchbase Web Console. @@ -59,15 +59,15 @@ For backup, restore, and other related tasks to be scheduled and performed, the The service (as is the case with all other Couchbase services) can be assigned either when a node is initially provisioned as a one-node cluster (as described in xref:manage:manage-nodes/create-cluster.adoc[Create a Cluster]), or when a node is added to an existing cluster (as described in xref:manage:manage-nodes/add-node-and-rebalance.adoc[Add a Node and Rebalance]). Provided that at least one node runs the Backup Service, data for the entire cluster can be backed up, restored, and archived. Locations to be used for saving data must be accessible to all cluster-nodes that are running the Backup Service. -Note also that Couchbase Server must have _read_ and _write_ access to the location. -On Linux, therefore, for a filesystem location, use the `chgrp` command to set the group ID of the folder to `couchbase`; unless a _non-root installation_ has been performed, in which case set the group ID either to the username of the current user, or to a group of which the current user is a member — see xref:install:non-root.adoc[Non-Root Install and Upgrade], for more information. +Note also that Couchbase Server must have read and write access to the location. +On Linux, therefore, for a filesystem location, use the `chgrp` command to set the group ID of the folder to `couchbase`; unless a non-root installation has been performed, in which case set the group ID either to the username of the current user, or to a group of which the current user is a member — see xref:install:non-root.adoc[Non-Root Install and Upgrade], for more information. [#access-the-backup-service-ui] == Access the Backup Service UI To access the Backup Service UI, proceed as follows: -. On Couchbase Web Console, left-click on the *Backup* tab, in the right-hand, vertical navigation bar: +. On Couchbase Web Console, click the *Backup* tab, in the vertical navigation bar: + image::manage-backup-restore/accessBackupTab.png[,100,align=left] + @@ -85,7 +85,7 @@ Currently, all panels are blank. The Backup Service allows backups (and merges) to be scheduled, as _tasks_. This section describes how task-definition and scheduling can be accomplished. -Note that for any given repository, the Backup Service performs one task at a time; with each task maintaining a lock on the repository. +For any given repository, the Backup Service performs one task at a time; with each task maintaining a lock on the repository. Therefore, the administrator-defined interval between tasks should always be sufficient to allow each task to run to completion. If a new task is scheduled to start while a previously started task is still running, the new task cannot run. For information, see xref:learn:services-and-indexes/services/backup-service.adoc#avoiding-task-overlap[Avoiding Task Overlap]. @@ -96,11 +96,11 @@ To schedule one or more backups, proceed as follows: When fully defined, the repository will combine the definitions of one or more backup and related activities, scheduled for one or more buckets, targeted at a storage location accessible to all nodes on the cluster. Each repository must have a name unique among repositories on the cluster. + -To add a repository, left-click on the *ADD REPOSITORY* tab, at the upper right of the screen: +To add a repository, click the *ADD REPOSITORY* tab, at the upper right of the screen: + image::manage-backup-restore/addRepositoryTab.png[,140,align=left] + -This brings up the *Select Plan* dialog, which initially appears as follows: +This opens the *Select Plan* dialog which initially appears as follows: + image::manage-backup-restore/selectPlanDialog.png[,420,align=left] @@ -111,12 +111,12 @@ The *_hourly_backups* plan appears as the default selection. + (For more information, see xref:manage:manage-backup-and-restore/manage-backup-and-restore.adoc#default-plans[Default Plans], below.) + -Left-click on the control that appears at the right-hand side of the *Select plan* dialog's interactive text-field. +Click the control that appears at the right-hand side of the *Select plan* dialog's interactive text-field. A pull-down menu appears, as follows: + image::manage-backup-restore/selectPlanDialogPullDownMenuInitial.png[,420,align=left] + -Three options are thus provided. +Three options are provided. The first two are *_daily_backups* and *_hourly_backups*. The third option is *+ Create new plan*: select this option: + @@ -127,7 +127,7 @@ This establishes the string *+ Create new plan* within the interactive text fiel image::manage-backup-restore/selectPlanDialog2.png[,420,align=left] . Create a custom plan. -In the *Name* field of the *Select Plan* dialog, enter a name for the plan that is to be created. +In the *Name* field of the *Select Plan* dialog, enter a name for the plan that's to be created. The name must be unique across the cluster, can only use the characters `[`, `]`, `A` to `Z`, `a` to `z`, `_` and `-`; and must not start with either `_` or `-`. + Then, optionally, add a description for the plan in the *Description* field: the description can be up to 140 characters in length. @@ -136,19 +136,20 @@ For example, to specify a plan for hourly backups, the following might be entere image::manage-backup-restore/createPlanDialogWithInitialInput.png[,420,align=left] + Next, specify the services for which data will be backed up. -Left-click on the *Services* control: this expands the dialog, and displays a complete list of Couchbase Services, each being accompanied by a checkbox. +Click *Services* to display the list of Couchbase Services. + image::manage-backup-restore/createPlanServicesListInitial.png[,90,align=left] + -To specify that only data for the Data and Index Services should be backed up, uncheck the boxes for all the other services. +To specify that only data for the Data and Index Services should be backed up, clear the boxes for all the other services. + -Next, to specify precise details of what should occur when the backup is run, left-click on the *Add Task* control. +Next, to specify precise details of what should occur when the backup is run, click the *Add Task* control. The dialog now expands, to reveal the following fields: + image::manage-backup-restore/createPlanDialogAddTaskFields.png[,420,align=left] + The fields permit the input of data to specify the details of a particular task. -Note that the dialog permits multiple tasks to be specified, by additional left-clickings of the *Add Task* control; and allows tasks selectively to be removed, by left-clickings of the *Remove Task* control. +The dialog permits multiple tasks to be added by click the *Add Task* control. +It also allows you to remove tasks by click the *Remove Task* control. + In the *Name* field, enter an appropriate name for the task: for example, *hourlyBackup*. + @@ -159,14 +160,14 @@ A pull-down menu appears: + image::manage-backup-restore/periodPullDownMenu.png[,420,align=left] + -From the pull-down menu, select *Hours*, to indicate that the frequency should be determined in units of hours. -(Note that this duly removes from the dialog the day-specification controls associated with *Weekly Calendar*.) +From the pull-down menu, select *Hours*, to set the frequency is in units of hours. +This removes from the dialog the day-specification controls associated with *Weekly Calendar*. + In the *Start Time* field, specify a time of day at which the task is to be run. The time of day must be specified as hours and minutes, separated by a colon. -Note that when the frequency-unit specified is *Minutes*, this field takes no input. +When the frequency-unit specified is *Minutes*, this field takes no input. When the frequency-unit specified is *Hours* (as is the case in the current example), only the numbers signifying minutes (those after the colon) are used. -To ensure that the hourly task is performed on the hour, leave these numbers as *00*. +To make sure that the hourly task is performed on the hour, leave these numbers as *00*. + In the *Type* field, specify the task to be performed, by accessing the control at the right-hand side of the field. This displays the following pull-down menu: @@ -178,11 +179,11 @@ Then, in the *Frequency* field, specify the frequency with which the task should The field only accepts integers: these must be between 1 and 200 inclusive. To specify that the task be performed hourly, enter *1*. + -(Note that an overview of all options for task-scheduling is provided below, in the section xref:manage:manage-backup-and-restore/manage-backup-and-restore.adoc#review-scheduling-options[Review Scheduling Options].) +See xref:manage:manage-backup-and-restore/manage-backup-and-restore.adoc#review-scheduling-options[Review Scheduling Options] for an overview of all task-scheduling options. + To complete specification of the task, determine whether the backup to be performed is *Full* or *Incremental*. -If it is to be *Full*, check the *Full Backup* checkbox. -If it is to be *Incremental* (as should be the case in the current example), leave the checkbox unchecked. +If it's to be *Full*, select *Full Backup*. +If it's to be *Incremental* (as should be the case in the current example), leave *Full Backup* cleared*. + The dialog now appears as follows: + @@ -206,7 +207,7 @@ The *ID* should be a name for the repository. The name must be unique across the cluster, can only use the characters `[`, `]`, `A` to `Z`, `a` to `z`, `_` and `-`; and must not start with either `_`, `-`, `[`, or `]`. For example, `hourlyBackupRepo`. + -The *Bucket* should be the name of either a _Couchbase_ or an _Ephemeral_ bucket, whose data is to be backed up. +The *Bucket* should be the name of either a Couchbase or an Ephemeral bucket, whose data is to be backed up. Selection can be made with a pull-down menu, accessed by means of the control at the right of the field. If a bucket-name is selected, only data from this bucket is backed up. If the default selection, *All buckets*, is used, data from all buckets on the cluster (including all Couchbase and all Ephemeral buckets) is backed up. @@ -216,14 +217,14 @@ For the current example, the sample bucket `travel-sample` is assumed to have be + The value for *Storage Locations* can be specified as *Filesystem* (the default) or *Cloud*. For the current example, *Filesystem* will be used. -Note that if *Cloud* is selected, allowing AWS S3 storage to be used, the dialog expands, and displays additional options: these are described below, in xref:manage:manage-backup-and-restore/manage-backup-and-restore.adoc#use-cloud-storage[Use Cloud Storage]. +If *Cloud* is selected, allowing AWS S3 storage to be used, the dialog expands, and displays additional options: these are described below, in xref:manage:manage-backup-and-restore/manage-backup-and-restore.adoc#use-cloud-storage[Use Cloud Storage]. + The *Location* should be the location of the storage-based archive for the repository. If on the local filesystem, this location must be a pathname accessible to all nodes within the cluster that are running the Backup Service: which is to say, reads from and writes to the location are shared through an NFS mount (or through some other type of shared-folder technology, such as Samba). -Couchbase Server must have _read_ and _write_ access to the location. +Couchbase Server must have read and write access to the location. On Linux, therefore, for a filesystem location, use the `chgrp` command to set the group ID of the folder to `couchbase`; unless a _non-root installation_ has been performed, in which case set the group ID either to the username of the current user, or to a group of which the current user is a member. + -Note that a location should be used for only one repository: when multiple repositories are to be archived, a different location should be used for each. +A location should be used for only one repository: when multiple repositories are to be archived, a different location should be used for each. If appropriate, locations may be specified as subdirectories, within a top-level directory. + When complete, the dialog may look as follows: @@ -239,7 +240,7 @@ The *Backup* screen now appears as follows: image::manage-backup-restore/newRepository.png[,720,align=left] -The newly created repository, *hourlyBackupRepo*, is thus displayed with its associated plan, `HourlyBackupPlan`, with the affected bucket (`travel-sample`) and the next scheduled backup displayed. +The newly created repository, *hourlyBackupRepo*, is displayed with its associated plan, `HourlyBackupPlan`, with the affected bucket (`travel-sample`) and the next scheduled backup displayed. Data Service and Index Service data for `travel-sample` will now be backed up to the specified location on the specified schedule. A repository whose plan is being executed (with data thereby backed up repeatedly, on schedule) is referred to as an _active_ repository. @@ -267,9 +268,9 @@ This displays the *Trigger Backup* dialog, which appears as follows: image::manage-backup-restore/triggerBackup.png[,420,align=left] The immediate backup to be performed will be _incremental_ by default. -To perform a _full_ backup, check the *Perform a full backup* checkbox. +To perform a _full_ backup, select *Perform a full backup*. -Left-click on the *Backup* button, at the lower right of the dialog. +Click the *Backup* button, at the lower right of the dialog. The dialog disappears, and a notification is displayed at the lower left of the console: image::manage-backup-restore/immediateBackupNotification.png[,220,align=left] @@ -297,8 +298,8 @@ The *Inspect Backups* view is selected by default. (Note the left-clicking the *Task History* button displays the *Tasks History* view: this is the same display as that accessed by means of the *Task History button, from the expanded row on the *Repositories* view of the *Backup* screen; and is described in xref:manage:manage-backup-and-restore/manage-backup-and-restore.adoc#inspect-tasks[Inspect Tasks], below.) The main, lower panel of the *Backups* view provides the ID of the repository (in this case, `83f3b752-78e6-49f8-a527-2844c30fbc75`) and its size (here, `235.551MiB`); and also provides a vertically arranged list of all backups that have occurred, with the earliest at the top. -Each backup has its own row; with its start-time, type (_full_ or _incremental_), and size. -To inspect a particular backup in detail, left-click on the control at the left-hand side of the row: +Each backup has its own row; with its start-time, type (full or incremental), and size. +To inspect a particular backup in detail, click the control at the left-hand side of the row: image::manage-backup-restore/examineBackup.png[,360,align=left] @@ -309,16 +310,16 @@ image::manage-backup-restore/examineBackupExpanded.png[,720,align=left] The displayed data includes the UUID for the source cluster. Also specified are the numbers of *Eventing Functions* written for the Eventing Service, and the number of *Full Text Search Aliases* for the Search Service (here, the numbers are both zero). -Each bucket that has been backed up (in this case, the `travel-sample` bucket alone), appears on its assigned row in a table that specifies its size; along with the number of items, mutations, and tombstones that have been included in the backup. -The row also lists the numbers of backed up indexes for the Index, Search, and Analytics Services; and the number of backed up Views. -Additionally, in a searchable sub-panel, each _scope_ that the bucket contains is individually listed (these here being the _default_ and _inventory_ scopes, and four _tenant_agent_ scopes); with the number of mutations and tombstones listed, per scope. +Each backed-up bucket appears on a table showing its size and the number of items, mutations, and tombstones that have been included in the backup. +The row also lists the numbers of backed up indexes for the Index, Search, and Analytics Services plus the number of backed up Views. +A searchable sub-panel lists each scope that the bucket contains along with the number of mutations and tombstones they contain. -To inspect the individual collections within a displayed scope, left-click on the row for the scope. +To inspect the individual collections within a displayed scope, click the row for the scope. The row expands vertically, as follows: image::manage-backup-restore/examineBackupExpandedScope.png[,720,align=left] -Thus, left-clicking on the row for the `inventory` scope has displayed the individual collections within the scope; and thereby shows the mutations and tombstones for each collection. +Clicking on the row for the `inventory` scope displays the individual collections within the scope with the mutations and tombstones for each collection. Collections can be searched for, based on strings entered into the *filter collections* field, which is located to the upper right of the collections panel. The upper panel of the *Data* screen provides interactive fields labelled *Key* and *Search Path*. @@ -329,7 +330,9 @@ For example, by accessing the control at the left-hand side of the *Start* field image::manage-backup-restore/specifyStartingBackupForSearch.png[,280,align=left] For example, type a known document key into the *Key* field — such as `airline_10`. -Then, enter the bucket name into the *Search Path* field: note that this requires explicit specification of both _scope_ and _collection_; unless default scope and collection have been used, in which case, explicit specification of the defaults is optional — for example, `travel-sample._default._default`. +Then, enter the bucket name into the *Search Path* field. +You must explicitly specify both the scope and collection unless you're using the default scope and collection. In that case, explicit;y setting the defaults is optional. +For example, `travel-sample._default._default`. When a search is expressed to include all backups of the bucket for the `inventory` scope and `airline` collection, the panels appear as follows: @@ -340,7 +343,7 @@ The *Examine* screen is now displayed: image::manage-backup-restore/examineScreen.png[,720,align=left] -Note that the controls adjacent to the *Diff* button, near the top of the screen, allow different backups to be selected, so that the differences between the document-versions they contain can be individually examined: +The controls adjacent to the *Diff* button, near the top of the screen, allow different backups to be selected, so that the differences between the document-versions they contain can be individually examined: image::manage-backup-restore/diffSelector.png[,420,align=left] @@ -371,7 +374,7 @@ image::manage-backup-restore/deleteBackupConfirmation.png[,420,align=left] Enter the backup name into the interactive text field, and left-click on *Delete*, to continue with deletion. The backup is deleted. -Note that once it has been deleted, it cannot be restored. +Once it has been deleted, it cannot be restored. [#inspect-tasks] == Inspect Tasks @@ -403,14 +406,14 @@ The selected row is expanded vertically, as follows: image::manage-backup-restore/expandedTaskRow.png[,480,align=left] -The details of the task are thus displayed as a JSON document. +The details of the task are displayed as a JSON document. The details include counts of items, vBuckets, and bytes received from the operation. The `node_runs` subdocument provides information specific to each node in the cluster. [#schedule-merges] == Schedule Merges -A _merge_ allows multiple backups to be combined as one; with _deduplication_ occurring. +A merge allows multiple backups to be combined as one; with deduplication occurring. Merges are supported for filesystem-based repositories: however, merges are _not_ supported for cloud-based repositories. If a merge is scheduled for a cloud-based repository, the Backup Service skips the task. @@ -418,7 +421,7 @@ An immediate merge cannot be triggered for a cloud-based repository. Merges can be scheduled as _tasks_, to be applied to backed up data within a defined repository. This section describes how task-definition and scheduling for merges can be accomplished. -Note that for any given repository, the Backup Service performs one task at a time; with each task maintaining a lock on the repository. +For any given repository, the Backup Service performs one task at a time; with each task maintaining a lock on the repository. Therefore, the administrator-defined interval between tasks should always be sufficient to allow each task to run to completion. If a new task is scheduled to start while a previously started task is still running, the new task cannot run. For information, see xref:learn:services-and-indexes/services/backup-service.adoc#avoiding-task-overlap[Avoiding Task Overlap]. @@ -431,7 +434,7 @@ When the *Select Plan* dialog is displayed, choose *+ Create new plan*. . In the redisplayed *Select Plan* dialog, specify a *Name* and a *Description* for the plan. Then, specify the *Services* whose data should be backed up. + -Note that a merge can only be scheduled as part of a plan that also schedules backup: the merge will be applied to backups within the defined repository. +A merge can only be scheduled as part of a plan that also schedules backup: the merge will be applied to backups within the defined repository. . Left-click on *Add Task*, and add a *Backup* task. For example: @@ -445,19 +448,19 @@ For example: + image::manage-backup-restore/mergeTask.png[,420,align=left] + -The *Type* of the task *MergeTask* has thus been specified as *Merge*, with a frequency of four hours. +The *Type* of the task *MergeTask* has been specified as *Merge*, with a frequency of four hours. Note the fields *Merge Offset Start* and *Merge Offset End*, which respectively specify the relative start and end points of each merge that will be performed. An offset start of *0* indicates that each merge will start with backups made on the current day, if such backups exist. An offset end of *2* indicates that each merge will end with backups that were made 2 days before the specified start-day, if such backups exist. If backups were not made every day during the specified period, as many as can be found will be merged. + -Note that a detailed, diagrammatic explanation of *Merge Offset Start* and *Merge Offset End* is provided in xref:learn:services-and-indexes/services/backup-service.adoc#specifying-merge-offsets[Specifying Merge Offsets]. +A detailed, diagrammatic explanation of *Merge Offset Start* and *Merge Offset End* is provided in xref:learn:services-and-indexes/services/backup-service.adoc#specifying-merge-offsets[Specifying Merge Offsets]. + Left-click on the *Next* button: + image::manage-backup-restore/nextButton.png[,140,align=left] -. When the *Create Repository* dialog appears, enter the *ID* of the repository you are creating, the name of the *Bucket* that is being backed up, the appropriate value of *Storage Locations* (here, *Filesystem*), and the on-disk location of the repository-archive. +. When the *Create Repository* dialog appears, enter the *ID* of the repository you're creating, the name of the *Bucket* that is being backed up, the appropriate value of *Storage Locations* (here, *Filesystem*), and the on-disk location of the repository-archive. (Note that this on-disk location must be accessible to _all_ Backup Service nodes in the cluster.) For example: + @@ -527,93 +530,154 @@ The details in the expanded row confirm that five backups were merged by the ope [#restore-backups] == Restore Backups -One or more backups can be _restored_ to the cluster; which means that the data in the backups is copied back into the buckets from which it was originally backed up, or into other buckets. -Proceed as follows: +You can restore a backup to the same bucket or buckets that you originally backed up or to a different set of buckets. +You can also restore a backup to a different cluster. +The buckets you restore data to do not have to use the same xref:learn:buckets-memory-and-storage/storage-engines.adoc[storage engine] as the original buckets. +You can restore a backup of data from a bucket using the Couchstore storage engine to one using Magma. +You can also restore a Magma-backed bucket backup to a Couchstore bucket. -. In the *Repositories* view of the *Backup* screen, select the repository from which data is to be restored, and left-click on the row for the repository, in order to expand it vertically. -Then, left-click on the *Restore* button: -+ -image:manage-backup-restore/restoreButton.png[,140,align=left] -+ -The *Restore* dialog is now displayed: -+ -image:manage-backup-restore/restoreDialog.png[,420,align=left] +To restore a backup: -. Use the *Restore* dialog to specify which backup or backups should be restored. -In the *Cluster* field, enter the IP address of the cluster at which the data-restoration is targeted. -Enter username and password for the target cluster in the *User* and *Password* fields, and then use the controls at the right-hand sides of the *Start* and *End* fields to select the first and last backups in the series that is to be restored. -The dialog now appears as follows: +. Select menu:Backup[Repositories] then expand the repository containing the data you want to restore. +. Click btn:[Restore]. +The *Restore* dialog opens: + -image:manage-backup-restore/restoreDialogPartiallyComplete.png[,420,align=left] +image:manage-backup-restore/restoreDialog.png[,420,align=left] -. Open the *Services* tab, on the *Restore* dialog, and specify the services whose data is to be restored — unchecking the checkbox for each service whose data is not required. +. In the *Cluster* field, enter the URL of a node in the cluster where you want to restore the data. +Include the REST API port--by default, 8091 for unencrypted HTTP and 18901 for secure HTTPS connections. +. Choose the method you want to use to authenticate with the target cluster. +You can use either Plain (a username and password) or a client certificate and key. +After making your choice, supply the credentials for the target cluster. +. In the *Start* and *End* fields, choose the start and end range of backups you want to restore. +. If you want to restore users and groups, expand *Users* and click *Restore users and User Groups*. +Also choose whether the backed-up users and groups overwrite any identically named existing ones. +. If you want to select which service's data gets restored, expand the *Services* section and select or clear services you want. For example: + image:manage-backup-restore/restoreUncheckCheckboxes.png[,240,align=left] -. Open the *Advanced Restore Options* tab, on the *Restore* dialog. -The dialog expands vertically, revealing the following fields: +. Expand the *Advanced Restore Options* if you want to: + -image:manage-backup-restore/restoreAdvancedOptionsInitial.png[,420,align=left] -+ -These fields allow selection of documents to be restored on the basis of the data they contain. -Documents that meet the specified criteria are included in the data-restoration; those that do not are omitted from it. -+ -Use of these fields is optional: if all data in the specified backups is to be restored, leave these fields blank. -If only some data should be restored, proceed as follows: -+ -In the *Filter Keys* field, add a _regular expression_ that must be matched by a document's _key_, if the document is to be included in the restoration. -For example, `^airline` ensures that only a document whose key begins with the string `airline` is included. + * Filter what data Couchbase Server restores. + * Restore a bucket's data to a different bucket. + * Control how the restore handles xref:learn:data/expiration.adoc[expiration] TTL values. + * Configure details about the bucket and collections being restored. + -In the *Filter Values* field, add a regular expression that must be matched by a _value_ within the document, if the document is to be included in the restoration. -For example, `MIL*` ensures that only a document that contains at least one key-value pair whose value contains the string `MIL` followed by zero of more characters is to be included in the restoration. -(See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions[Regular Expressions^] and https://www.regular-expressions.info/[Regular-Expressions.info^], for further information.) +All the fields in this section are optional. +See <> for more information. + +. Click btn:[Restore] to start the restore process. +A green pop-up briefly appears to verify that the restore task has started. + +To monitor an ongoing restore, click the btn:[Task History] button in the repository's entry in the *Repositories* tab. +The active restore task appears under the *Tasks* section. + +image:manage-backup-restore/newBucketWithRestoredData.png[,,align=left] + +After the restore tasks finishes, you can see whether it succeeded or failed under the *Results* section. + +NOTE: If the restore task completes while you're viewing *Task History*, it does not appear under the *Results* section until you click btn:[Refresh Tasks]. + +To learn how to restore a backup using the command line, see xref:backup-restore:cbbackupmgr-restore.adoc[]. + +[#advanced_restore_options] +=== Advanced Restore Options + +Expanding the *Restore* dialog's *Advanced Restore Options* section shows you fields where you can control: + +* Data filtering +* How TTL values are interpreted +* Whether the restore creates missing buckets or removes some scopes or collections. + +Once you expand the *Advanced Restore Options* section, a set of fields appears: + +image:manage-backup-restore/restoreAdvancedOptionsInitial.png[,420,align=left] + +The fields in this section are: + +Filter Keys:: +Lets you enter a regular expression the restore task uses to filter the key values. +The restore task only restores a document if its key matches the regular expression. + -In the *Map Data* field, indicate whether the data is to be restored to its original or to a different bucket. -If this field is left blank, data is restored to its original bucket: note that this bucket must continue to exist on the cluster. -If data is to be restored to a different bucket, that bucket must either already have been defined on the cluster, or must be created by means of the *Auto-create bucket* option, described below. +For example, if you enter `^airline` in this field, then the restore task only restores documents whose key begins with the string `airline`. + +Filter Values:: +Lets you enter a regular expression the restore task uses to filter documents based on their data. +The restore task only restores a document if one of its values matches the regular expression. + -For example, if data to be restored from `travel-sample` should be restored to `ts`, enter `travel-sample=ts`. +For example, if you enter `MIL*` in this field, the restore task only restores a document if has a value that contains the string `MIL` followed by zero of more characters. + +Map Data:: +Lets you have the restore task restore a backed-up bucket's data to a different bucket. +If you leave this field blank, the restore task restores data into same bucket from which it was backed up. + -Use the *Include Data* and *Exclude Data* fields to indicate the subset of buckets whose data is to be restored. -For example, if backups to be restored were made when the cluster had four buckets defined, named `bucket1`, `bucket2`, `bucket3`, and `bucket4`, entering `bucket1,bucket4` in the *Include Data* field ensures that only data from `bucket1` and `bucket4` is restored; while entering `bucket2,bucket3` in the *Exclude Data* field ensures that data from `bucket2` and `bucket3` is _not_ restored. Note that these options are intended for use on backups that included all buckets on the cluster: they are not required when the backup was made of one bucket only. +If you want a bucket's data to be saved in a bucket of a different name, enter the original bucket's name, an equal sign (`=`) and the target bucket's name. +For example to restore all data backed up from the `travel-sample` bucket into a bucket named `ts`, enter `travel-sample=ts` into the *Map Data* field. + -Note that the *Include Data* and *Exclude Data* fields also allow the _scopes_ and _collections_ within buckets to be specified. -To specify a scope within a bucket, use the syntax _bucket-name_._scope-name_. -To specify a collection within a scope within a bucket, use the syntax _bucket-name_._scope-name_._collection-name_. -For example, entering `bucket1.scope1` in the *Include Data* field would ensure that only data from the scope `scope1` within `bucket1` is restored; while entering `bucket2.scope1.collection1` in the *Exclude Data* field would ensure that data from `collection1`, within `scope1` in `bucket2`, is _not_ be restored. -(For an overview of scopes and collections, see xref:learn:data/scopes-and-collections.adoc[Scopes and Collections].) +The target bucket must exist on the target cluster or you must enable <>. + +Include Data:: +Exclude Data:: +These fields let you limit the restoration to a subset of the buckets, scopes, and collections in the backup. +The *Include Data* has the restore task restore just the buckets, collections, and scopes that you list in this field. +The *Exclude Data* field restores all data in the backup except the buckets or collections you list in this field. + -The *Replace TTL* field allows a new _expiration_ value to be established for restored documents. -The dropdown menu provides the options *none* (the default), which means that no new expiration value is established for any document; *all*, which means that a new expiration value is established for every restored document; and *expired*, which means that a new expiration value is established for every document that has expired. -The new expiration value must be specified by means of the *Replace TTL-with* field: the value must either be specified as an RFC3339 time stamp (such as `2006-01-02T15:04:05-07:00`); or must be `0`, which means that each affected document is restored with no expiration value established. -For more information, see xref:learn:data/expiration.adoc[Expiration]. +To include or exclude buckets, add their names in a comma-separated list to the *Include Data* or *Exclude Data* fields. +For example, suppose the backups you're restoring contain four buckets named `bucket1`, `bucket2`, `bucket3`, and `bucket4`. +Then entering `bucket1,bucket4` in the *Include Data* field has the restore task restore just the data from `bucket1` and `bucket4`. +In this case, you could instead enter `bucket2,bucket3` in the *Exclude Data* field to get the same result. + -Check the *Force Updates* field to ensure that data restored from the specified backup overwrites the current values on the cluster when the current values are the more recent. -If the *Force Updates* checkbox is not checked, current values are not overwritten if more recent. +You can specify a scope to be included in or excluded from the restore by listing its bucket name, followed by a period, and then the scope name. +Similarly, to include or exclude a collection, specify the name of its bucket, scope, and its collection name joined by periods. +For example, to exclude the `route` collection in the `travel-sample` bucket's `inventory` scope, enter `travel-sample.inventory.route` in *Exclude Data*. + -Check the *Auto-remove Collections* checkbox to omit from the restoration any scope or collection that has been removed from the cluster since the backup was performed. -(Note that if a data-containing, administrator-created collection is backed up, but is then deleted from the cluster with all its data, the deleted data will not be restored by the *Restore* operation: however, the empty collection _will_ be restored by the *Restore* operation, unless the *Auto-remove Collections* checkbox is checked, prior to the *Restore* operation.) +See xref:learn:data/scopes-and-collections.adoc[] for an overview of scopes and collections. + +Replace TTL:: +Replace TTL with:: +These fields let you choose how the restore task handles time to live (TTL) values in the documents it's restoring. +The *Replace TTL* list controls when the restore task applies the date you enter into the *Replace TTL with* field to the documents it's restoring. +The settings in this list are: + -Check the *Auto-create Buckets* checkbox to create any buckets to which the restoration has been mapped that do not yet exist on the target cluster. +* *none*: The restore task does not change the TTL value in the value in the backup. +If the document's expiration time is in the past, Couchbase Server marks it as deleted soon after the restore task restores it. +* *expired*: If a document being restored has an expiration date in the past, the restore task sets its TTL to the value you supply in *Replace TTL with*. +* *all*: The restore task applies the new TTL you supply in *Replace TTL with* to all documents it restores. +It even applies the new value to restored documents that had a TTL of `0` (no expiration) in the backup. + -For example, the *Restore* dialog may now appear as follows: +The value you supply in *Replace TTL with* field must be either: + -image:manage-backup-restore/restoreDialogComplete.png[,420,align=left] +* `0` : No TTL value is set for the document. +The document does not expire unless the bucket or collection containing it has a non-zero `maxTTL` value. +See xref:learn:data/expiration.adoc[]. +* A string containing an http://https://www.rfc-editor.org/rfc/rfc3339[RFC3339^] time stamp. +All documents to which the restore task applies this value will expire when on the date and time you set. + -Values are thus specified for filtering documents on a basis of both key and value. -The data to be restored from `travel-sample` is specified to be restored to a bucket named `ts`, which has not previously been created: therefore, the *Auto-create Buckets* checkbox has been checked. +NOTE: The *Replace TTL with* field does not prevent you from entering a timestamp in the past. +Entering a date in the past results in any documents that the restore task applies the field's value to being deleted by Couchbase Server soon after restoration. -. Left-click on *Restore*. -This triggers the specified restoration. -The dialog disappears; and a green restore-notification appears, at the lower left of the console. +Force Updates:: +By default, the restore task does not overwrite an existing document that has a more recent modification time than its backed up version. +Select *Force Updates* to have the restore task always overwrite existing documents with the version in the backup even if the existing document is more recent. -Subsequent to the operation, its results can be checked; by means of the *Buckets* screen of Couchbase Web Console, which might now appear as follows: +Auto-remove Collections:: +When checked, the restore task drops scopes and collections that currently exist in buckets but had been dropped prior to the backup's creation. +The restore task knows which scopes and collections have been dropped because the backup contains the tombstones of these dropped objects. +For a scope or collection to be dropped when you enable *Auto-remove Collections*, its ID must match the ID of a dropped scope or collection as well as matching its name. +Just matching the name of a deleted scope or collection is not enough to have the restore task drop it. ++ +NOTE: This option is only useful for situations where you're dropping and recreating buckets. +For example, suppose you make a backup of a bucket where you had dropped scopes or collections. +Then, later, you drop the bucket and recreate it and its scopes and collections (including the ones you had previously deleted) in precisely the same order that you had created them in the original bucket. +In this case, the scopes and collections will have the same IDs that they had in the original bucket and therefore in the backup. +Finally, if you restore the backup to the bucket with *Auto-remove Collections* selected, the restore task deletes scopes and collections that match the IDs of deleted ones in the backup. -image:manage-backup-restore/newBucketWithRestoredData.png[,720,align=left] +[#auto-create-buckets] +Auto-create Buckets:: +By default, the restore task exits with an error message if a bucket being restored from the backup does not currently exist in the cluster. +Selecting *Auto-create Buckets* has the restore task create any missing buckets. -A new bucket, named `ts`, has thus been created. -Its item-count indicates that it contains only a subset of the documents contained in `travel-sample`, in accordance with the filtering specified for the restore operation. [#pause-backups] == Pause Backups @@ -1042,7 +1106,7 @@ Provider:: This should remain set as `GCP`. Cloud Bucket:: -The name of the bucket on the `GCP` service you are backing up to. +The name of the bucket on the `GCP` service you're backing up to. Cloud Auth Type:: This can be either `ID and Key` or `Instance Metadata Service`. @@ -1071,7 +1135,7 @@ You will require a different set of options depending on which one cloud authent [%collapsible] ===== -If you are using a GCP virtual machine to hold your backup, then you can make use of the GCP VM service account with the `Metadata Service` authorization type. +If you're using a GCP virtual machine to hold your backup, then you can make use of the GCP VM service account with the `Metadata Service` authorization type. . Ensure that the service account that are using on https://cloud.google.com/[Google Cloud] has `Access scopes` set to `Set access for each API`. +