From b40a658b4e9892e3288c7232fa37c42721d5e224 Mon Sep 17 00:00:00 2001 From: "Kenneth P. J. Dyer" <93145796+kennethdyer@users.noreply.github.com> Date: Thu, 4 Apr 2024 08:53:27 -0500 Subject: [PATCH] DOCSP-36932 RN for 1.7.2 (#265) * DOCSP-35932 RN for 1.7.2 * Fixes per Johnny DuBois * Fixes per Johnny DuBois * Additiona 1.7.2 changes * Fixes per Evgeni * Fixes per Evgeni * Fixes per Evgeni * Fixes per Evgeni * Fixes per Evgeni * Fixes per Evgeni * Fixes orphan warning * Fixes orphan warning * Apply suggestions from code review Fixes per Lauren Co-authored-by: ltran-mdb2 <143426234+ltran-mdb2@users.noreply.github.com> * Fixes per Lauren Co-authored-by: ltran-mdb2 <143426234+ltran-mdb2@users.noreply.github.com> * Fixes per Lauren * Fixes per Lauren * Fixes per Johnny DuBois * Fixes per Johnny DuBois * Fixes per Johnny DuBois * Fixes per Johnny DuBois * Fixes per Johnny DuBois * Fixes per Johnny DuBois * Fixes per Johnny DuBois * Fixes per Johnny DuBois * Fixes per Johnny DuBois Co-authored-by: John DuBois * Fixes per Johnny DuBois * fixes per Tim * fixes per Tim * fixes per Tim --------- Co-authored-by: ltran-mdb2 <143426234+ltran-mdb2@users.noreply.github.com> Co-authored-by: John DuBois --- source/includes/fact-oplog-background.rst | 28 +++----- .../read-preference-connection-string.rst | 9 ++- source/multiple-mongosyncs.txt | 7 ++ source/reference/limitations.txt | 20 +++++- source/reference/mongosync.txt | 6 +- source/release-notes/1.7.txt | 70 ++++++++++++++++++- source/using-mongosync.txt | 2 + 7 files changed, 111 insertions(+), 31 deletions(-) diff --git a/source/includes/fact-oplog-background.rst b/source/includes/fact-oplog-background.rst index 9f2138a06..b78a7876a 100644 --- a/source/includes/fact-oplog-background.rst +++ b/source/includes/fact-oplog-background.rst @@ -8,22 +8,14 @@ on the source cluster, the sync fails and ``mongosync`` exits. .. include:: /includes/fact-applyOps.rst -Starting in version 1.5.0, ``mongosync`` enables Oplog Rollover -Resilience (ORR). With ORR, ``mongosync`` applies changes on the -source cluster to the destination cluster during the initial sync. ORR -increases the resilience of ``mongosync`` to oplog rollover but does not -prevent rollover entirely. - -You might exceed the oplog window if you: - -- Sync from a high write rate source cluster for an extended - period. -- Pause sync for an extended period. - -To increase the size of the ``oplog`` on the source cluster, use -:setting:`~replication.oplogSizeMB`. For more information, see -:ref:`Change Oplog Size ` and -:ref:`Workloads that Might Requre a Large Oplog Size -`. - +During the initial sync, ``mongosync`` may apply operations at a slower +rate due to copying documents concurrently. +After the initial sync, ``mongosync`` applies changes +faster and is more likely to maintain a position in the ``oplog`` +that is close to the real-time writes occurring on the source cluster. + +If you anticipate syncing a large data set, or if you plan to pause +synchronization for an extended period of time, you might exceed the +:term:`oplog window`. Use the :setting:`~replication.oplogSizeMB` setting +to increase the size of the ``oplog`` on the source cluster. diff --git a/source/includes/read-preference-connection-string.rst b/source/includes/read-preference-connection-string.rst index 3d0143876..d8a44285f 100644 --- a/source/includes/read-preference-connection-string.rst +++ b/source/includes/read-preference-connection-string.rst @@ -1,5 +1,4 @@ -You can also specify read preference on a per-connection basis in the -connection string. By default, ``mongosync`` sets the source cluster -read preference to :readmode:`nearest` to distribute reads evenly across -nodes. For more information on setting read preference in connection -strings, see :ref:`connections-read-preference`. \ No newline at end of file + +``mongosync`` requires the :readmode:`primary` read preference +to connect to the source cluster. For more information, see :ref:`connections-read-preference`. + diff --git a/source/multiple-mongosyncs.txt b/source/multiple-mongosyncs.txt index eeaad7c34..39d4ee18c 100644 --- a/source/multiple-mongosyncs.txt +++ b/source/multiple-mongosyncs.txt @@ -18,6 +18,13 @@ There are two ways to synchronize :ref:`sharded clusters loaded clusters, use one ``mongosync`` instance for each shard on the source cluster. +.. important:: + + When the source cluster is a sharded cluster, stop the + balancer and don't run the :dbcommand:`moveChunk` or + :dbcommand:`moveRange` commands until ``mongosync`` reaches + the collection copy phase. + .. _c2c-sharded-config-single: Configure a Single ``mongosync`` Instance diff --git a/source/reference/limitations.txt b/source/reference/limitations.txt index a61f3c149..d5cfc27a3 100644 --- a/source/reference/limitations.txt +++ b/source/reference/limitations.txt @@ -52,6 +52,9 @@ General Limitations - .. include:: /includes/fact-atlas-require-indexes-limitation.rst - ``mongosync`` doesn't sync users or roles. - .. include:: /includes/fact-applyOps.rst +- ``mongosync`` must read from the source cluster using the + :readmode:`primary` read preference. + MongoDB Community Edition ------------------------- @@ -105,11 +108,22 @@ Sharded Clusters - ``mongosync`` only syncs indexes that have consistent index specifications on all shards. -.. note:: + .. note:: + + To check for index inconsistencies, see :ref:`Find Inconsistent + Indexes Across Shards `. + +- You must stop the balancer on a sharded source cluster until + mongosync finishes the collection copy phase. + + .. note:: - To check for index inconsistencies, see :ref:`Find Inconsistent - Indexes Across Shards `. + After stopping the balancer, wait fifteen minutes before + starting ``mongosync``. This gives the cluster time to + finish any in progress chunk migrations. +- You must not run the :dbcommand:`moveChunk` and + :dbcommand:`moveRange` commands on the source cluster. - The shard key cannot be :ref:`refined ` while syncing. - The shard key cannot be modified using :dbcommand:`reshardCollection` diff --git a/source/reference/mongosync.txt b/source/reference/mongosync.txt index ec5608360..04292b7ee 100644 --- a/source/reference/mongosync.txt +++ b/source/reference/mongosync.txt @@ -266,10 +266,8 @@ behavior, see :ref:`read-concern` and :ref:`write-concern`. Read Preference ~~~~~~~~~~~~~~~ -By default, ``mongosync`` sets the source cluster read preference to -:readmode:`nearest` to distribute reads evenly across nodes. To change -the read preference of the source cluster, specify the read preference -mode in the URI of the connection string. For details, see +``mongosync`` requires the :readmode:`primary` read preference +when connecting to the source cluster. For more information, see :ref:`connections-read-preference`. .. _c2c-capped-collections: diff --git a/source/release-notes/1.7.txt b/source/release-notes/1.7.txt index b25159afe..20c519d3c 100644 --- a/source/release-notes/1.7.txt +++ b/source/release-notes/1.7.txt @@ -15,6 +15,74 @@ Release Notes for mongosync 1.7 Patch Releases -------------- +.. _1.7.2-c2c-release-notes: + +1.7.2 Release +~~~~~~~~~~~~~ + +**February 21, 2024** + +Issues Fixed: + +- Fixed bug introduced in v1.5.0 that could result in data + inconsistencies when mongosync is killed or paused during + the initial collection copy phase and then resumed. + + This can only affect migrations if: + + - The migration includes mixed _id types and the destination + cluster runs MongoDB 4.4 or earlier. + + - The migration includes capped collections and the + destination cluster runs MongoDB 6.0 or later. + +- Fixed bug introduced in v1.0.0 where indexes could be created + on the destination cluster that were dropped on the source + cluster during migration. + + This can only affect migrations where the given index is both + created and dropped while mongosync is running. + +- Fixed bug introduced in v1.0.0 that could result in + collections being created on the destination cluster with + incorrect types, options, or indexes. + + This can only affect migrations where collections are dropped + or renamed while mongosync is running and the source or + destintion cluster runs MongoDB 6.0.0 to 6.0.12 or MongoDB + 7.0.0 or 7.0.3. + +- Fixed bug introduced in v1.1.0 that could result in a + mongosync crash if shard key indexes on the source cluster + have inconsistent values across shards for the "background" + index build option. + + This can only affect migrations that involve sharded + collections and source clusters older than MongoDB 6.0. + +Other Changes: + +- Disables the Oplog Rollover Resilience mechanism. + +- Live upgrade to mongosync 1.7.2 is not allowed. + +- Reduces latency of the mongosync API. + +Limitations: + +- mongosync must read from the source cluster using the + :readmode:`primary` read preference. If you try to start mongosync + with a different read preference, it throws an error. + +- You must stop the balancer on a sharded source cluster until + mongosync finishes the collection copy phase. If you try to + start mongosync while the balancer is still running, it throws + an error. + +- You must not run the :dbcommand:`moveChunk` or :dbcommand:`moveRange` + commands on the source cluster. If these are run on the + source cluster, mongosync throws an error. + .. _1.7.1-c2c-release-notes: 1.7.1 Release @@ -64,7 +132,7 @@ Issues Fixed: capped collection during migration, before or while the Collection Copy phase is working on the same capped collection. -- Fixed bug introduced in 1.6 that caused ``mongosync`` to fail during +- Fixed bug introduced in 1.6 that caused ``mongosync`` to fail during initialization if the user specified only an exclusion filter and there were no fully excluded databases. diff --git a/source/using-mongosync.txt b/source/using-mongosync.txt index 11eeaad44..0e995e430 100644 --- a/source/using-mongosync.txt +++ b/source/using-mongosync.txt @@ -1,3 +1,5 @@ +:orphan: + .. place holder for the using mongosync section TOC .. _c2c-running-mongosync: .. _c2c-using-mongosync: