diff --git a/src/current/_includes/v24.3/sidebar-data/cross-cluster-replication.json b/src/current/_includes/v24.3/sidebar-data/cross-cluster-replication.json index 925e3e2aca9..8b76ae44d8b 100644 --- a/src/current/_includes/v24.3/sidebar-data/cross-cluster-replication.json +++ b/src/current/_includes/v24.3/sidebar-data/cross-cluster-replication.json @@ -46,6 +46,12 @@ "/${VERSION}/set-up-physical-cluster-replication.html" ] }, + { + "title": "Upgrade a Cluster Running PCR", + "urls": [ + "/${VERSION}/upgrade-with-pcr.html" + ] + }, { "title": "Fail Over from a Primary to a Standby Cluster", "urls": [ diff --git a/src/current/_includes/v25.2/sidebar-data/cross-cluster-replication.json b/src/current/_includes/v25.2/sidebar-data/cross-cluster-replication.json index 925e3e2aca9..8b76ae44d8b 100644 --- a/src/current/_includes/v25.2/sidebar-data/cross-cluster-replication.json +++ b/src/current/_includes/v25.2/sidebar-data/cross-cluster-replication.json @@ -46,6 +46,12 @@ "/${VERSION}/set-up-physical-cluster-replication.html" ] }, + { + "title": "Upgrade a Cluster Running PCR", + "urls": [ + "/${VERSION}/upgrade-with-pcr.html" + ] + }, { "title": "Fail Over from a Primary to a Standby Cluster", "urls": [ diff --git a/src/current/_includes/v25.3/sidebar-data/cross-cluster-replication.json b/src/current/_includes/v25.3/sidebar-data/cross-cluster-replication.json index d9ee279f835..d6d9d215952 100644 --- a/src/current/_includes/v25.3/sidebar-data/cross-cluster-replication.json +++ b/src/current/_includes/v25.3/sidebar-data/cross-cluster-replication.json @@ -46,6 +46,12 @@ "/${VERSION}/set-up-physical-cluster-replication.html" ] }, + { + "title": "Upgrade a Cluster Running PCR", + "urls": [ + "/${VERSION}/upgrade-with-pcr.html" + ] + }, { "title": "Fail Over from a Primary to a Standby Cluster", "urls": [ diff --git a/src/current/_includes/v25.4/sidebar-data/cross-cluster-replication.json b/src/current/_includes/v25.4/sidebar-data/cross-cluster-replication.json index d9ee279f835..d6d9d215952 100644 --- a/src/current/_includes/v25.4/sidebar-data/cross-cluster-replication.json +++ b/src/current/_includes/v25.4/sidebar-data/cross-cluster-replication.json @@ -46,6 +46,12 @@ "/${VERSION}/set-up-physical-cluster-replication.html" ] }, + { + "title": "Upgrade a Cluster Running PCR", + "urls": [ + "/${VERSION}/upgrade-with-pcr.html" + ] + }, { "title": "Fail Over from a Primary to a Standby Cluster", "urls": [ diff --git a/src/current/v24.3/physical-cluster-replication-overview.md b/src/current/v24.3/physical-cluster-replication-overview.md index 8c2c57c8bc9..ef53e26703b 100644 --- a/src/current/v24.3/physical-cluster-replication-overview.md +++ b/src/current/v24.3/physical-cluster-replication-overview.md @@ -71,17 +71,9 @@ Statement | Action ## Cluster versions and upgrades -{{site.data.alerts.callout_info}} -The entire standby cluster must be at the same version as, or one version ahead of, the primary's virtual cluster. -{{site.data.alerts.end}} - -When PCR is enabled, upgrade with the following procedure. This upgrades the standby cluster before the primary cluster. Within the primary and standby CockroachDB clusters, the system virtual cluster must be at a cluster version greater than or equal to the virtual cluster: +To safely perform a major version upgrade on a cluster with PCR, you must upgrade the standby cluster before the primary cluster to avoid version incompatibility. If you are using a reader virtual cluster (reader VC), you must upgrade it separately from the primary and standby clusters. -1. [Upgrade the binaries]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-major-version-upgrade) on the primary and standby clusters. Replace the binary on each node of the cluster and restart the node. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the standby's system virtual cluster if auto-finalization is disabled. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary's system virtual cluster if auto-finalization is disabled. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the standby's virtual cluster. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary's virtual cluster. +Follow the process described in [Upgrade a Cluster Running PCR]({% link {{ page.version.version }}/upgrade-with-pcr.md %}). ## Demo video diff --git a/src/current/v24.3/upgrade-cockroach-version.md b/src/current/v24.3/upgrade-cockroach-version.md index 99d79b696be..628ac3ca931 100644 --- a/src/current/v24.3/upgrade-cockroach-version.md +++ b/src/current/v24.3/upgrade-cockroach-version.md @@ -5,7 +5,11 @@ toc: true docs_area: manage --- -This page describes how major-version and patch upgrades work and shows how to upgrade a self-hosted CockroachDB. To upgrade a cluster in CockroachDB {{ site.data.products.cloud }}, refer to [Upgrade a cluster in CockroachDB {{ site.data.products.cloud }}]({% link cockroachcloud/upgrade-cockroach-version.md %}) instead. +This page describes how major-version and patch upgrades work and shows how to upgrade a self-hosted CockroachDB. + +To upgrade a cluster in CockroachDB {{ site.data.products.cloud }}, refer to [Upgrade a cluster in CockroachDB {{ site.data.products.cloud }}]({% link cockroachcloud/upgrade-cockroach-version.md %}) instead. + +To upgrade a cluster with with [physical cluster replication (PCR)]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}), refer to [Upgrade a Cluster Running PCR]({% link {{ page.version.version }}/upgrade-with-pcr.md %}). ## Overview diff --git a/src/current/v24.3/upgrade-with-pcr.md b/src/current/v24.3/upgrade-with-pcr.md new file mode 100644 index 00000000000..32201d50fa2 --- /dev/null +++ b/src/current/v24.3/upgrade-with-pcr.md @@ -0,0 +1,141 @@ +--- +title: Upgrade a Cluster Running PCR +summary: Upgrade your primary and standby clusters when using PCR. +toc: true +docs_area: manage +--- + +To safely perform a major version upgrade on a cluster with [**physical cluster replication (PCR)**]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}), you must upgrade the standby cluster before the primary cluster to avoid version incompatibility. You cannot replicate data from a cluster on a newer version to a cluster on an older version. + +This page describes the process for safely performing a major version upgrade on a pair of clusters that have PCR enabled. + +{{site.data.alerts.callout_info}} +The entire standby cluster must be on the same major version as the primary cluster or a major version the primary cluster [can directly upgrade to]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#compatible-versions). Within the primary and standby CockroachDB clusters, the _system virtual cluster (system VC)_ must be at a cluster major version greater than or equal to the _application virtual cluster (app VC)_. +{{site.data.alerts.end}} + +## Minor version upgrades + +Minor versions are not relevant when determining PCR compatibility. There is no need to consider PCR compatibility when upgrading to a specific minor version within a major version, so you can follow the normal [patch upgrade process]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-patch-upgrade). + +## Upgrade primary and standby clusters + +To upgrade your primary and standby clusters: + +1. Ensure that the virtual clusters on both your primary cluster and your standby cluster are [finalized]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) on the current version before beginning the upgrade process. + +1. [Upgrade the binaries]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-major-version-upgrade) on the standby cluster. On each node of the cluster, replace the binary and restart the node. + +1. [Connect]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#connect-to-the-system-virtual-cluster) to the standby cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ shell + cockroach sql --url \ + "postgresql://root@{standby node IP or hostname}:26257?options=-ccluster=system&sslmode=verify-full" \ + --certs-dir "certs" + ~~~ + +1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the standby cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SET CLUSTER SETTING version '{VERSION}'; + ~~~ + + {{site.data.alerts.callout_info}} + If you need to [roll back]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#roll-back-a-major-version-upgrade) an upgrade, you must do so before the upgrade has been finalized. + {{site.data.alerts.end}} + +1. Confirm that finalization is complete on the standby cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SHOW CLUSTER SETTING version; + ~~~ + +1. [Upgrade the binaries]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-major-version-upgrade) on the primary cluster. On each node of the cluster, replace the binary and restart the node. + +1. [Connect]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#connect-to-the-system-virtual-cluster) to the primary cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ shell + cockroach sql --url \ + "postgresql://root@{primary node IP or hostname}:26257?options=-ccluster=system&sslmode=verify-full" \ + --certs-dir "certs" + ~~~ + +1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SET CLUSTER SETTING version '{VERSION}'; + ~~~ + + {{site.data.alerts.callout_info}} + If you need to [roll back]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#roll-back-a-major-version-upgrade) an upgrade, you must do so before the upgrade has been finalized. Rolling back the upgrade on the primary cluster does not roll back the standby cluster. + {{site.data.alerts.end}} + +1. Confirm that finalization is complete on the primary cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SHOW CLUSTER SETTING version; + ~~~ + +1. [Connect]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#connect-to-the-system-virtual-cluster) to the primary cluster's app VC: + + {% include_cached copy-clipboard.html %} + ~~~ shell + cockroach sql --url \ + "postgresql://root@{primary node IP or hostname}:26257?options=-ccluster={app_virtual_cluster_name}&sslmode=verify-full" \ + --certs-dir "certs" + ~~~ + +1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary cluster's app VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SET CLUSTER SETTING version '{VERSION}'; + ~~~ + + Upgrading the primary cluster's app VC also upgrades the standby cluster's app VC, since it replicates from the primary. + +1. Confirm that finalization is complete on the primary cluster's app VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SHOW CLUSTER SETTING version; + ~~~ + +## Upgrade reader VC + +If you have a _reader virtual cluster (reader VC)_, you must drop and recreate it to upgrade after completing the main upgrade process on the primary and standby clusters. Follow these steps to upgrade your reader VC: + +1. After upgrading the app VC on your primary cluster, wait for the replicated time to pass the time at which the upgrade completed. +1. On the standby cluster, stop the reader VC service: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER VIRTUAL CLUSTER STOP SERVICE; + ~~~ + +1. Drop the reader VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + DROP VIRTUAL CLUSTER ; + ~~~ + +1. On the standby cluster, re-create the reader VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER VIRTUAL CLUSTER dest-system SET REPLICATION READ VIRTUAL CLUSTER; + ~~~ + +At this point, the reader VC is on the same version as the standby cluster. + +## Failover and fast failback during upgrade + +If needed, you can perform a [failover]({% link {{ page.version.version }}/failover-replication.md %}) while upgrading your clusters. + +However, after performing a failover you cannot perform a [fast failback]({% link {{ page.version.version }}/failover-replication.md %}#failback) to the original primary cluster while the standby cluster version is newer than the primary cluster version. This is because you cannot replicate data from a cluster on a newer version to a cluster on an older version. \ No newline at end of file diff --git a/src/current/v25.2/physical-cluster-replication-overview.md b/src/current/v25.2/physical-cluster-replication-overview.md index 65b332607ca..9947094fbe3 100644 --- a/src/current/v25.2/physical-cluster-replication-overview.md +++ b/src/current/v25.2/physical-cluster-replication-overview.md @@ -69,17 +69,9 @@ Statement | Action ## Cluster versions and upgrades -{{site.data.alerts.callout_info}} -The entire standby cluster must be at the same version as, or one version ahead of, the primary's virtual cluster. -{{site.data.alerts.end}} - -When PCR is enabled, upgrade with the following procedure. This upgrades the standby cluster before the primary cluster. Within the primary and standby CockroachDB clusters, the system virtual cluster must be at a cluster version greater than or equal to the virtual cluster: +To safely perform a major version upgrade on a cluster with PCR, you must upgrade the standby cluster before the primary cluster to avoid version incompatibility. If you are using a reader virtual cluster (reader VC), you must upgrade it separately from the primary and standby clusters. -1. [Upgrade the binaries]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-major-version-upgrade) on the primary and standby clusters. Replace the binary on each node of the cluster and restart the node. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the standby's system virtual cluster if auto-finalization is disabled. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary's system virtual cluster if auto-finalization is disabled. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the standby's virtual cluster. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary's virtual cluster. +Follow the process described in [Upgrade a Cluster Running PCR]({% link {{ page.version.version }}/upgrade-with-pcr.md %}). ## Demo video diff --git a/src/current/v25.2/upgrade-cockroach-version.md b/src/current/v25.2/upgrade-cockroach-version.md index 99d79b696be..628ac3ca931 100644 --- a/src/current/v25.2/upgrade-cockroach-version.md +++ b/src/current/v25.2/upgrade-cockroach-version.md @@ -5,7 +5,11 @@ toc: true docs_area: manage --- -This page describes how major-version and patch upgrades work and shows how to upgrade a self-hosted CockroachDB. To upgrade a cluster in CockroachDB {{ site.data.products.cloud }}, refer to [Upgrade a cluster in CockroachDB {{ site.data.products.cloud }}]({% link cockroachcloud/upgrade-cockroach-version.md %}) instead. +This page describes how major-version and patch upgrades work and shows how to upgrade a self-hosted CockroachDB. + +To upgrade a cluster in CockroachDB {{ site.data.products.cloud }}, refer to [Upgrade a cluster in CockroachDB {{ site.data.products.cloud }}]({% link cockroachcloud/upgrade-cockroach-version.md %}) instead. + +To upgrade a cluster with with [physical cluster replication (PCR)]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}), refer to [Upgrade a Cluster Running PCR]({% link {{ page.version.version }}/upgrade-with-pcr.md %}). ## Overview diff --git a/src/current/v25.2/upgrade-with-pcr.md b/src/current/v25.2/upgrade-with-pcr.md new file mode 100644 index 00000000000..32201d50fa2 --- /dev/null +++ b/src/current/v25.2/upgrade-with-pcr.md @@ -0,0 +1,141 @@ +--- +title: Upgrade a Cluster Running PCR +summary: Upgrade your primary and standby clusters when using PCR. +toc: true +docs_area: manage +--- + +To safely perform a major version upgrade on a cluster with [**physical cluster replication (PCR)**]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}), you must upgrade the standby cluster before the primary cluster to avoid version incompatibility. You cannot replicate data from a cluster on a newer version to a cluster on an older version. + +This page describes the process for safely performing a major version upgrade on a pair of clusters that have PCR enabled. + +{{site.data.alerts.callout_info}} +The entire standby cluster must be on the same major version as the primary cluster or a major version the primary cluster [can directly upgrade to]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#compatible-versions). Within the primary and standby CockroachDB clusters, the _system virtual cluster (system VC)_ must be at a cluster major version greater than or equal to the _application virtual cluster (app VC)_. +{{site.data.alerts.end}} + +## Minor version upgrades + +Minor versions are not relevant when determining PCR compatibility. There is no need to consider PCR compatibility when upgrading to a specific minor version within a major version, so you can follow the normal [patch upgrade process]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-patch-upgrade). + +## Upgrade primary and standby clusters + +To upgrade your primary and standby clusters: + +1. Ensure that the virtual clusters on both your primary cluster and your standby cluster are [finalized]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) on the current version before beginning the upgrade process. + +1. [Upgrade the binaries]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-major-version-upgrade) on the standby cluster. On each node of the cluster, replace the binary and restart the node. + +1. [Connect]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#connect-to-the-system-virtual-cluster) to the standby cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ shell + cockroach sql --url \ + "postgresql://root@{standby node IP or hostname}:26257?options=-ccluster=system&sslmode=verify-full" \ + --certs-dir "certs" + ~~~ + +1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the standby cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SET CLUSTER SETTING version '{VERSION}'; + ~~~ + + {{site.data.alerts.callout_info}} + If you need to [roll back]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#roll-back-a-major-version-upgrade) an upgrade, you must do so before the upgrade has been finalized. + {{site.data.alerts.end}} + +1. Confirm that finalization is complete on the standby cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SHOW CLUSTER SETTING version; + ~~~ + +1. [Upgrade the binaries]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-major-version-upgrade) on the primary cluster. On each node of the cluster, replace the binary and restart the node. + +1. [Connect]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#connect-to-the-system-virtual-cluster) to the primary cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ shell + cockroach sql --url \ + "postgresql://root@{primary node IP or hostname}:26257?options=-ccluster=system&sslmode=verify-full" \ + --certs-dir "certs" + ~~~ + +1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SET CLUSTER SETTING version '{VERSION}'; + ~~~ + + {{site.data.alerts.callout_info}} + If you need to [roll back]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#roll-back-a-major-version-upgrade) an upgrade, you must do so before the upgrade has been finalized. Rolling back the upgrade on the primary cluster does not roll back the standby cluster. + {{site.data.alerts.end}} + +1. Confirm that finalization is complete on the primary cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SHOW CLUSTER SETTING version; + ~~~ + +1. [Connect]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#connect-to-the-system-virtual-cluster) to the primary cluster's app VC: + + {% include_cached copy-clipboard.html %} + ~~~ shell + cockroach sql --url \ + "postgresql://root@{primary node IP or hostname}:26257?options=-ccluster={app_virtual_cluster_name}&sslmode=verify-full" \ + --certs-dir "certs" + ~~~ + +1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary cluster's app VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SET CLUSTER SETTING version '{VERSION}'; + ~~~ + + Upgrading the primary cluster's app VC also upgrades the standby cluster's app VC, since it replicates from the primary. + +1. Confirm that finalization is complete on the primary cluster's app VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SHOW CLUSTER SETTING version; + ~~~ + +## Upgrade reader VC + +If you have a _reader virtual cluster (reader VC)_, you must drop and recreate it to upgrade after completing the main upgrade process on the primary and standby clusters. Follow these steps to upgrade your reader VC: + +1. After upgrading the app VC on your primary cluster, wait for the replicated time to pass the time at which the upgrade completed. +1. On the standby cluster, stop the reader VC service: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER VIRTUAL CLUSTER STOP SERVICE; + ~~~ + +1. Drop the reader VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + DROP VIRTUAL CLUSTER ; + ~~~ + +1. On the standby cluster, re-create the reader VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER VIRTUAL CLUSTER dest-system SET REPLICATION READ VIRTUAL CLUSTER; + ~~~ + +At this point, the reader VC is on the same version as the standby cluster. + +## Failover and fast failback during upgrade + +If needed, you can perform a [failover]({% link {{ page.version.version }}/failover-replication.md %}) while upgrading your clusters. + +However, after performing a failover you cannot perform a [fast failback]({% link {{ page.version.version }}/failover-replication.md %}#failback) to the original primary cluster while the standby cluster version is newer than the primary cluster version. This is because you cannot replicate data from a cluster on a newer version to a cluster on an older version. \ No newline at end of file diff --git a/src/current/v25.3/physical-cluster-replication-overview.md b/src/current/v25.3/physical-cluster-replication-overview.md index 65b332607ca..9947094fbe3 100644 --- a/src/current/v25.3/physical-cluster-replication-overview.md +++ b/src/current/v25.3/physical-cluster-replication-overview.md @@ -69,17 +69,9 @@ Statement | Action ## Cluster versions and upgrades -{{site.data.alerts.callout_info}} -The entire standby cluster must be at the same version as, or one version ahead of, the primary's virtual cluster. -{{site.data.alerts.end}} - -When PCR is enabled, upgrade with the following procedure. This upgrades the standby cluster before the primary cluster. Within the primary and standby CockroachDB clusters, the system virtual cluster must be at a cluster version greater than or equal to the virtual cluster: +To safely perform a major version upgrade on a cluster with PCR, you must upgrade the standby cluster before the primary cluster to avoid version incompatibility. If you are using a reader virtual cluster (reader VC), you must upgrade it separately from the primary and standby clusters. -1. [Upgrade the binaries]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-major-version-upgrade) on the primary and standby clusters. Replace the binary on each node of the cluster and restart the node. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the standby's system virtual cluster if auto-finalization is disabled. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary's system virtual cluster if auto-finalization is disabled. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the standby's virtual cluster. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary's virtual cluster. +Follow the process described in [Upgrade a Cluster Running PCR]({% link {{ page.version.version }}/upgrade-with-pcr.md %}). ## Demo video diff --git a/src/current/v25.3/read-from-standby.md b/src/current/v25.3/read-from-standby.md index 681fa6aa91d..5dca8061812 100644 --- a/src/current/v25.3/read-from-standby.md +++ b/src/current/v25.3/read-from-standby.md @@ -11,13 +11,13 @@ Use this page to understand how the _read from standby_ feature works and how to ## How the read from standby feature works -PCR utilizes [cluster virtualization]({% link {{ page.version.version }}/cluster-virtualization-overview.md %}) to separate a cluster's control plane from its data plane. A cluster always has one control plane, called a _system virtual cluster (SystemVC)_, and at least one data plane, called an _App Virtual Cluster (AppVC)_. The standby cluster's SystemVC manages the PCR job and other cluster metadata, and is not used for application queries. All data tables, system tables, and cluster settings in the standby cluster's AppVC are identical to the primary cluster's AppVC. The standby cluster's AppVC itself remains offline during replication. +PCR utilizes [cluster virtualization]({% link {{ page.version.version }}/cluster-virtualization-overview.md %}) to separate a cluster's control plane from its data plane. A cluster always has one control plane, called a _system virtual cluster (system VC)_, and at least one data plane, called an _App Virtual Cluster (app VC)_. The standby cluster's system VC manages the PCR job and other cluster metadata, and is not used for application queries. All data tables, system tables, and cluster settings in the standby cluster's app VC are identical to the primary cluster's app VC. The standby cluster's app VC itself remains offline during replication. -When using read from standby, applications can read from the standby cluster, but they do not connect directly to the standby cluster's AppVC. Instead, PCR introduces a _reader virtual cluster (ReaderVC)_. The ReaderVC ensures a clean, isolated environment specifically for serving read queries without interfering with replication or system metadata. It reads continuously from the standby cluster's AppVC using internal pointers, providing access to the replicated data while keeping the AppVC offline. The ReaderVC itself only stores a small amount of metadata and no user data, so it is not expected to take up additional storage space. +When using read from standby, applications can read from the standby cluster, but they do not connect directly to the standby cluster's app VC. Instead, PCR introduces a _reader virtual cluster (reader VC)_. The reader VC ensures a clean, isolated environment specifically for serving read queries without interfering with replication or system metadata. It reads continuously from the standby cluster's app VC using internal pointers, providing access to the replicated data while keeping the app VC offline. The reader VC itself only stores a small amount of metadata and no user data, so it is not expected to take up additional storage space. -The standby cluster's ReaderVC has its own system tables and [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}). The ReaderVC replicates a subset of system tables, including **Users** and **Roles**, from the AppVC, so that existing primary users can authenticate using the same [users and roles]({% link {{ page.version.version }}/security-reference/authorization.md %}) as on the primary cluster's AppVC. Other system tables and cluster settings are set to defaults in the ReaderVC. For more information, consult [Physical Cluster Replication Technical Overview]({% link {{ page.version.version }}/physical-cluster-replication-technical-overview.md %}). +The standby cluster's reader VC has its own system tables and [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}). The reader VC replicates a subset of system tables, including **Users** and **Roles**, from the app VC, so that existing primary users can authenticate using the same [users and roles]({% link {{ page.version.version }}/security-reference/authorization.md %}) as on the primary cluster's app VC. Other system tables and cluster settings are set to defaults in the reader VC. For more information, consult [Physical Cluster Replication Technical Overview]({% link {{ page.version.version }}/physical-cluster-replication-technical-overview.md %}). -In the event of failover, the ReaderVC's response depends on the type of failover. After failover to the latest timestamp, the ReaderVC continues pointing to the AppVC but stops receiving updates. After failover to a point-in-time timestamp, the ReaderVC is destroyed. +In the event of failover, the reader VC's response depends on the type of failover. After failover to the latest timestamp, the reader VC continues pointing to the app VC but stops receiving updates. After failover to a point-in-time timestamp, the reader VC is destroyed. ## Use the read from standby feature ### Before you begin @@ -46,7 +46,7 @@ ALTER VIRTUAL CLUSTER main SET REPLICATION READ VIRTUAL CLUSTER; ~~~ {{site.data.alerts.callout_info}} -The standby cluster's AppVC must have a status of `replicating` before you can create your ReaderVC. Use the [`SHOW VIRTUAL CLUSTERS`]({% link {{ page.version.version }}/show-virtual-cluster.md %}) command to check the status of the AppVC. +The standby cluster's app VC must have a status of `replicating` before you can create your reader VC. Use the [`SHOW VIRTUAL CLUSTERS`]({% link {{ page.version.version }}/show-virtual-cluster.md %}) command to check the status of the app VC. {{site.data.alerts.end}} ### Check the status of your reader virtual cluster @@ -58,7 +58,7 @@ To confirm that your reader virtual cluster is active: SHOW VIRTUAL CLUSTERS; ~~~ -The output shows a `standby-readonly` virtual cluster in addition to the systemVC and AppVC: +The output shows a `standby-readonly` virtual cluster in addition to the system VC and app VC: ~~~ id | name | data_state | service_mode @@ -69,7 +69,7 @@ The output shows a `standby-readonly` virtual cluster in addition to the systemV ~~~ {{site.data.alerts.callout_info}} -The ReaderVC cannot serve reads until after the PCR initial scan is complete. After completing the initial scan, wait until the ReaderVC's `service_mode` is `shared`, then wait about one minute before connecting to the ReaderVC. +The reader VC cannot serve reads until after the PCR initial scan is complete. After completing the initial scan, wait until the reader VC's `service_mode` is `shared`, then wait about one minute before connecting to the reader VC. {{site.data.alerts.end}} ### Run read-only queries on the standby cluster diff --git a/src/current/v25.3/upgrade-cockroach-version.md b/src/current/v25.3/upgrade-cockroach-version.md index 99d79b696be..628ac3ca931 100644 --- a/src/current/v25.3/upgrade-cockroach-version.md +++ b/src/current/v25.3/upgrade-cockroach-version.md @@ -5,7 +5,11 @@ toc: true docs_area: manage --- -This page describes how major-version and patch upgrades work and shows how to upgrade a self-hosted CockroachDB. To upgrade a cluster in CockroachDB {{ site.data.products.cloud }}, refer to [Upgrade a cluster in CockroachDB {{ site.data.products.cloud }}]({% link cockroachcloud/upgrade-cockroach-version.md %}) instead. +This page describes how major-version and patch upgrades work and shows how to upgrade a self-hosted CockroachDB. + +To upgrade a cluster in CockroachDB {{ site.data.products.cloud }}, refer to [Upgrade a cluster in CockroachDB {{ site.data.products.cloud }}]({% link cockroachcloud/upgrade-cockroach-version.md %}) instead. + +To upgrade a cluster with with [physical cluster replication (PCR)]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}), refer to [Upgrade a Cluster Running PCR]({% link {{ page.version.version }}/upgrade-with-pcr.md %}). ## Overview diff --git a/src/current/v25.3/upgrade-with-pcr.md b/src/current/v25.3/upgrade-with-pcr.md new file mode 100644 index 00000000000..65774013236 --- /dev/null +++ b/src/current/v25.3/upgrade-with-pcr.md @@ -0,0 +1,141 @@ +--- +title: Upgrade with Physical Cluster Replication Enabled +summary: Upgrade your primary and standby clusters when using PCR. +toc: true +docs_area: manage +--- + +To safely perform a major version upgrade on a cluster with [**physical cluster replication (PCR)**]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}), you must upgrade the standby cluster before the primary cluster to avoid version incompatibility. You cannot replicate data from a cluster on a newer version to a cluster on an older version. + +This page describes the process for safely performing a major version upgrade on a pair of clusters that have PCR enabled. + +{{site.data.alerts.callout_info}} +The entire standby cluster must be on the same major version as the primary cluster or a major version the primary cluster [can directly upgrade to]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#compatible-versions). Within the primary and standby CockroachDB clusters, the _system virtual cluster (system VC)_ must be at a cluster major version greater than or equal to the _application virtual cluster (app VC)_. +{{site.data.alerts.end}} + +## Minor version upgrades + +Minor versions are not relevant when determining PCR compatibility. There is no need to consider PCR compatibility when upgrading to a specific minor version within a major version, so you can follow the normal [patch upgrade process]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-patch-upgrade). + +## Upgrade primary and standby clusters + +To upgrade your primary and standby clusters: + +1. Ensure that the virtual clusters on both your primary cluster and your standby cluster are [finalized]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) on the current version before beginning the upgrade process. + +1. [Upgrade the binaries]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-major-version-upgrade) on the standby cluster. On each node of the cluster, replace the binary and restart the node. + +1. [Connect]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#connect-to-the-system-virtual-cluster) to the standby cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ shell + cockroach sql --url \ + "postgresql://root@{standby node IP or hostname}:26257?options=-ccluster=system&sslmode=verify-full" \ + --certs-dir "certs" + ~~~ + +1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the standby cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SET CLUSTER SETTING version '{VERSION}'; + ~~~ + + {{site.data.alerts.callout_info}} + If you need to [roll back]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#roll-back-a-major-version-upgrade) an upgrade, you must do so before the upgrade has been finalized. + {{site.data.alerts.end}} + +1. Confirm that finalization is complete on the standby cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SHOW CLUSTER SETTING version; + ~~~ + +1. [Upgrade the binaries]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-major-version-upgrade) on the primary cluster. On each node of the cluster, replace the binary and restart the node. + +1. [Connect]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#connect-to-the-system-virtual-cluster) to the primary cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ shell + cockroach sql --url \ + "postgresql://root@{primary node IP or hostname}:26257?options=-ccluster=system&sslmode=verify-full" \ + --certs-dir "certs" + ~~~ + +1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SET CLUSTER SETTING version '{VERSION}'; + ~~~ + + {{site.data.alerts.callout_info}} + If you need to [roll back]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#roll-back-a-major-version-upgrade) an upgrade, you must do so before the upgrade has been finalized. Rolling back the upgrade on the primary cluster does not roll back the standby cluster. + {{site.data.alerts.end}} + +1. Confirm that finalization is complete on the primary cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SHOW CLUSTER SETTING version; + ~~~ + +1. [Connect]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#connect-to-the-system-virtual-cluster) to the primary cluster's app VC: + + {% include_cached copy-clipboard.html %} + ~~~ shell + cockroach sql --url \ + "postgresql://root@{primary node IP or hostname}:26257?options=-ccluster={app_virtual_cluster_name}&sslmode=verify-full" \ + --certs-dir "certs" + ~~~ + +1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary cluster's app VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SET CLUSTER SETTING version '{VERSION}'; + ~~~ + + Upgrading the primary cluster's app VC also upgrades the standby cluster's app VC, since it replicates from the primary. + +1. Confirm that finalization is complete on the primary cluster's app VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SHOW CLUSTER SETTING version; + ~~~ + +## Upgrade reader VC + +If you have a [_reader virtual cluster (reader VC)_]({% link {{ page.version.version }}/read-from-standby.md %}), you must drop and recreate it to upgrade after completing the main upgrade process on the primary and standby clusters. Follow these steps to upgrade your reader VC: + +1. After upgrading the app VC on your primary cluster, wait for the replicated time to pass the time at which the upgrade completed. +1. On the standby cluster, stop the reader VC service: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER VIRTUAL CLUSTER STOP SERVICE; + ~~~ + +1. Drop the reader VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + DROP VIRTUAL CLUSTER ; + ~~~ + +1. On the standby cluster, re-create the reader VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER VIRTUAL CLUSTER dest-system SET REPLICATION READ VIRTUAL CLUSTER; + ~~~ + +At this point, the reader VC is on the same version as the standby cluster. + +## Failover and fast failback during upgrade + +If needed, you can perform a [failover]({% link {{ page.version.version }}/failover-replication.md %}) while upgrading your clusters. + +However, after performing a failover you cannot perform a [fast failback]({% link {{ page.version.version }}/failover-replication.md %}#failback) to the original primary cluster while the standby cluster version is newer than the primary cluster version. This is because you cannot replicate data from a cluster on a newer version to a cluster on an older version. \ No newline at end of file diff --git a/src/current/v25.4/physical-cluster-replication-overview.md b/src/current/v25.4/physical-cluster-replication-overview.md index 65b332607ca..9947094fbe3 100644 --- a/src/current/v25.4/physical-cluster-replication-overview.md +++ b/src/current/v25.4/physical-cluster-replication-overview.md @@ -69,17 +69,9 @@ Statement | Action ## Cluster versions and upgrades -{{site.data.alerts.callout_info}} -The entire standby cluster must be at the same version as, or one version ahead of, the primary's virtual cluster. -{{site.data.alerts.end}} - -When PCR is enabled, upgrade with the following procedure. This upgrades the standby cluster before the primary cluster. Within the primary and standby CockroachDB clusters, the system virtual cluster must be at a cluster version greater than or equal to the virtual cluster: +To safely perform a major version upgrade on a cluster with PCR, you must upgrade the standby cluster before the primary cluster to avoid version incompatibility. If you are using a reader virtual cluster (reader VC), you must upgrade it separately from the primary and standby clusters. -1. [Upgrade the binaries]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-major-version-upgrade) on the primary and standby clusters. Replace the binary on each node of the cluster and restart the node. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the standby's system virtual cluster if auto-finalization is disabled. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary's system virtual cluster if auto-finalization is disabled. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the standby's virtual cluster. -1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary's virtual cluster. +Follow the process described in [Upgrade a Cluster Running PCR]({% link {{ page.version.version }}/upgrade-with-pcr.md %}). ## Demo video diff --git a/src/current/v25.4/read-from-standby.md b/src/current/v25.4/read-from-standby.md index b1cd65b77be..ab5f1b7036f 100644 --- a/src/current/v25.4/read-from-standby.md +++ b/src/current/v25.4/read-from-standby.md @@ -11,13 +11,13 @@ Use this page to understand how the _read from standby_ feature works and how to ## How the read from standby feature works -PCR utilizes [cluster virtualization]({% link {{ page.version.version }}/cluster-virtualization-overview.md %}) to separate a cluster's control plane from its data plane. A cluster always has one control plane, called a _system virtual cluster (SystemVC)_, and at least one data plane, called an _App Virtual Cluster (AppVC)_. The standby cluster's SystemVC manages the PCR job and other cluster metadata, and is not used for application queries. All data tables, system tables, and cluster settings in the standby cluster's AppVC are identical to the primary cluster's AppVC. The standby cluster's AppVC itself remains offline during replication. +PCR utilizes [cluster virtualization]({% link {{ page.version.version }}/cluster-virtualization-overview.md %}) to separate a cluster's control plane from its data plane. A cluster always has one control plane, called a _system virtual cluster (system VC)_, and at least one data plane, called an _App Virtual Cluster (app VC)_. The standby cluster's system VC manages the PCR job and other cluster metadata, and is not used for application queries. All data tables, system tables, and cluster settings in the standby cluster's app VC are identical to the primary cluster's app VC. The standby cluster's app VC itself remains offline during replication. -When using read from standby, applications can read from the standby cluster, but they do not connect directly to the standby cluster's AppVC. Instead, PCR introduces a _reader virtual cluster (ReaderVC)_. The ReaderVC ensures a clean, isolated environment specifically for serving read queries without interfering with replication or system metadata. It reads continuously from the standby cluster's AppVC using internal pointers, providing access to the replicated data while keeping the AppVC offline. The ReaderVC itself only stores a small amount of metadata and no user data, so it is not expected to take up additional storage space. +When using read from standby, applications can read from the standby cluster, but they do not connect directly to the standby cluster's app VC. Instead, PCR introduces a _reader virtual cluster (reader VC)_. The reader VC ensures a clean, isolated environment specifically for serving read queries without interfering with replication or system metadata. It reads continuously from the standby cluster's app VC using internal pointers, providing access to the replicated data while keeping the app VC offline. The reader VC itself only stores a small amount of metadata and no user data, so it is not expected to take up additional storage space. -The standby cluster's ReaderVC has its own system tables and [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}). The ReaderVC replicates a subset of system tables, including **Users** and **Roles**, from the AppVC, so that existing primary users can authenticate using the same [users and roles]({% link {{ page.version.version }}/security-reference/authorization.md %}) as on the primary cluster's AppVC. Other system tables and cluster settings are set to defaults in the ReaderVC. For more information, consult [Physical Cluster Replication Technical Overview]({% link {{ page.version.version }}/physical-cluster-replication-technical-overview.md %}). +The standby cluster's reader VC has its own system tables and [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}). The reader VC replicates a subset of system tables, including **Users** and **Roles**, from the app VC, so that existing primary users can authenticate using the same [users and roles]({% link {{ page.version.version }}/security-reference/authorization.md %}) as on the primary cluster's app VC. Other system tables and cluster settings are set to defaults in the reader VC. For more information, consult [Physical Cluster Replication Technical Overview]({% link {{ page.version.version }}/physical-cluster-replication-technical-overview.md %}). -In the event of failover, the ReaderVC is destroyed. +In the event of failover, the reader VC is destroyed. ## Use the read from standby feature ### Before you begin @@ -46,7 +46,7 @@ ALTER VIRTUAL CLUSTER main SET REPLICATION READ VIRTUAL CLUSTER; ~~~ {{site.data.alerts.callout_info}} -The standby cluster's AppVC must have a status of `replicating` before you can create your ReaderVC. Use the [`SHOW VIRTUAL CLUSTERS`]({% link {{ page.version.version }}/show-virtual-cluster.md %}) command to check the status of the AppVC. +The standby cluster's app VC must have a status of `replicating` before you can create your reader VC. Use the [`SHOW VIRTUAL CLUSTERS`]({% link {{ page.version.version }}/show-virtual-cluster.md %}) command to check the status of the app VC. {{site.data.alerts.end}} ### Check the status of your reader virtual cluster @@ -58,7 +58,7 @@ To confirm that your reader virtual cluster is active: SHOW VIRTUAL CLUSTERS; ~~~ -The output shows a `standby-readonly` virtual cluster in addition to the systemVC and AppVC: +The output shows a `standby-readonly` virtual cluster in addition to the system VC and app VC: ~~~ id | name | data_state | service_mode @@ -69,7 +69,7 @@ The output shows a `standby-readonly` virtual cluster in addition to the systemV ~~~ {{site.data.alerts.callout_info}} -The ReaderVC cannot serve reads until after the PCR initial scan is complete. After completing the initial scan, wait until the ReaderVC's `service_mode` is `shared`, then wait about one minute before connecting to the ReaderVC. +The reader VC cannot serve reads until after the PCR initial scan is complete. After completing the initial scan, wait until the reader VC's `service_mode` is `shared`, then wait about one minute before connecting to the reader VC. {{site.data.alerts.end}} ### Run read-only queries on the standby cluster diff --git a/src/current/v25.4/upgrade-cockroach-version.md b/src/current/v25.4/upgrade-cockroach-version.md index 99d79b696be..628ac3ca931 100644 --- a/src/current/v25.4/upgrade-cockroach-version.md +++ b/src/current/v25.4/upgrade-cockroach-version.md @@ -5,7 +5,11 @@ toc: true docs_area: manage --- -This page describes how major-version and patch upgrades work and shows how to upgrade a self-hosted CockroachDB. To upgrade a cluster in CockroachDB {{ site.data.products.cloud }}, refer to [Upgrade a cluster in CockroachDB {{ site.data.products.cloud }}]({% link cockroachcloud/upgrade-cockroach-version.md %}) instead. +This page describes how major-version and patch upgrades work and shows how to upgrade a self-hosted CockroachDB. + +To upgrade a cluster in CockroachDB {{ site.data.products.cloud }}, refer to [Upgrade a cluster in CockroachDB {{ site.data.products.cloud }}]({% link cockroachcloud/upgrade-cockroach-version.md %}) instead. + +To upgrade a cluster with with [physical cluster replication (PCR)]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}), refer to [Upgrade a Cluster Running PCR]({% link {{ page.version.version }}/upgrade-with-pcr.md %}). ## Overview diff --git a/src/current/v25.4/upgrade-with-pcr.md b/src/current/v25.4/upgrade-with-pcr.md new file mode 100644 index 00000000000..d6f82dfa17d --- /dev/null +++ b/src/current/v25.4/upgrade-with-pcr.md @@ -0,0 +1,141 @@ +--- +title: Upgrade a Cluster Running PCR +summary: Upgrade your primary and standby clusters when using PCR. +toc: true +docs_area: manage +--- + +To safely perform a major version upgrade on a cluster with [**physical cluster replication (PCR)**]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}), you must upgrade the standby cluster before the primary cluster to avoid version incompatibility. You cannot replicate data from a cluster on a newer version to a cluster on an older version. + +This page describes the process for safely performing a major version upgrade on a pair of clusters that have PCR enabled. + +{{site.data.alerts.callout_info}} +The entire standby cluster must be on the same major version as the primary cluster or a major version the primary cluster [can directly upgrade to]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#compatible-versions). Within the primary and standby CockroachDB clusters, the _system virtual cluster (system VC)_ must be at a cluster major version greater than or equal to the _application virtual cluster (app VC)_. +{{site.data.alerts.end}} + +## Minor version upgrades + +Minor versions are not relevant when determining PCR compatibility. There is no need to consider PCR compatibility when upgrading to a specific minor version within a major version, so you can follow the normal [patch upgrade process]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-patch-upgrade). + +## Upgrade primary and standby clusters + +To upgrade your primary and standby clusters: + +1. Ensure that the virtual clusters on both your primary cluster and your standby cluster are [finalized]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) on the current version before beginning the upgrade process. + +1. [Upgrade the binaries]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-major-version-upgrade) on the standby cluster. On each node of the cluster, replace the binary and restart the node. + +1. [Connect]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#connect-to-the-system-virtual-cluster) to the standby cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ shell + cockroach sql --url \ + "postgresql://root@{standby node IP or hostname}:26257?options=-ccluster=system&sslmode=verify-full" \ + --certs-dir "certs" + ~~~ + +1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the standby cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SET CLUSTER SETTING version '{VERSION}'; + ~~~ + + {{site.data.alerts.callout_info}} + If you need to [roll back]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#roll-back-a-major-version-upgrade) an upgrade, you must do so before the upgrade has been finalized. + {{site.data.alerts.end}} + +1. Confirm that finalization is complete on the standby cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SHOW CLUSTER SETTING version; + ~~~ + +1. [Upgrade the binaries]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#perform-a-major-version-upgrade) on the primary cluster. On each node of the cluster, replace the binary and restart the node. + +1. [Connect]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#connect-to-the-system-virtual-cluster) to the primary cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ shell + cockroach sql --url \ + "postgresql://root@{primary node IP or hostname}:26257?options=-ccluster=system&sslmode=verify-full" \ + --certs-dir "certs" + ~~~ + +1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SET CLUSTER SETTING version '{VERSION}'; + ~~~ + + {{site.data.alerts.callout_info}} + If you need to [roll back]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#roll-back-a-major-version-upgrade) an upgrade, you must do so before the upgrade has been finalized. Rolling back the upgrade on the primary cluster does not roll back the standby cluster. + {{site.data.alerts.end}} + +1. Confirm that finalization is complete on the primary cluster's system VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SHOW CLUSTER SETTING version; + ~~~ + +1. [Connect]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#connect-to-the-system-virtual-cluster) to the primary cluster's app VC: + + {% include_cached copy-clipboard.html %} + ~~~ shell + cockroach sql --url \ + "postgresql://root@{primary node IP or hostname}:26257?options=-ccluster={app_virtual_cluster_name}&sslmode=verify-full" \ + --certs-dir "certs" + ~~~ + +1. [Finalize]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}#finalize-a-major-version-upgrade-manually) the upgrade on the primary cluster's app VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SET CLUSTER SETTING version '{VERSION}'; + ~~~ + + Upgrading the primary cluster's app VC also upgrades the standby cluster's app VC, since it replicates from the primary. + +1. Confirm that finalization is complete on the primary cluster's app VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SHOW CLUSTER SETTING version; + ~~~ + +## Upgrade reader VC + +If you have a [_reader virtual cluster (reader VC)_]({% link {{ page.version.version }}/read-from-standby.md %}), you must drop and recreate it to upgrade after completing the main upgrade process on the primary and standby clusters. Follow these steps to upgrade your reader VC: + +1. After upgrading the app VC on your primary cluster, wait for the replicated time to pass the time at which the upgrade completed. +1. On the standby cluster, stop the reader VC service: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER VIRTUAL CLUSTER STOP SERVICE; + ~~~ + +1. Drop the reader VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + DROP VIRTUAL CLUSTER ; + ~~~ + +1. On the standby cluster, re-create the reader VC: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER VIRTUAL CLUSTER SET REPLICATION READ VIRTUAL CLUSTER; + ~~~ + +At this point, the reader VC is on the same version as the standby cluster. + +## Failover and fast failback during upgrade + +If needed, you can perform a [failover]({% link {{ page.version.version }}/failover-replication.md %}) while upgrading your clusters. + +However, after performing a failover you cannot perform a [fast failback]({% link {{ page.version.version }}/failover-replication.md %}#failback) to the original primary cluster while the standby cluster version is newer than the primary cluster version. This is because you cannot replicate data from a cluster on a newer version to a cluster on an older version. \ No newline at end of file