Skip to content

Conversation

rkistner
Copy link
Contributor

@rkistner rkistner commented Nov 6, 2024

This adds an option to use MongoDB postImages when replicating changes, to get proper consistency of updates.

For what this means, see: https://www.mongodb.com/docs/v6.0/reference/command/collMod/#change-streams-with-document-pre--and-post-images

Since this can be a breaking change for existing instances, it is not automatically enabled.

There are three valid options:

  1. post_images: off: The current behavior and the default to prevent breaking existing instances. This may produce updates out of order.
  2. post_images: read_only: This requires changeStreamPreAndPostImages: { enabled: true } to be set on every collection that is referenced in sync rules, otherwise replication will error. Ideal when permissions are restricted.
  3. post_images: auto_configure: This automatically configures the changeStreamPreAndPostImages option on collections if needed. Requires the collMod permission.

TODO:

  • Proper naming of options
  • Handle cases where document postImages have expired, or have only been enabled after the start of the changestream (will need to restart replication).
  • Add check in diagnostics/validation API.

@changeset-bot
Copy link

changeset-bot bot commented Nov 6, 2024

⚠️ No Changeset found

Latest commit: c5ca6e7

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@rkistner rkistner marked this pull request as ready for review November 13, 2024 10:56
@rkistner rkistner requested a review from Rentacookie November 13, 2024 10:56
@rkistner rkistner changed the title MongoDB postImages [MongoDB] postImages Nov 13, 2024
@rkistner rkistner merged commit 80573ac into feat/modular-replication-architecture Nov 13, 2024
15 checks passed
@rkistner rkistner deleted the mongodb-postimages branch November 13, 2024 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants