feat(mongodb): make transaction read concern configurable by xgerman · Pull Request #314 · ExtendDB/extenddb

xgerman · 2026-08-26T04:10:28Z

Problem

The MongoDB storage backend hardcodes readConcern: snapshot on every multi-document transaction it opens (conditional writes, TransactWriteItems, TransactGetItems, idempotency-token checks). Real MongoDB (7.0+) supports this and it's the strongest available isolation level, but some MongoDB-wire-compatible servers do not implement it.

Concretely, DocumentDB (a Postgres-based MongoDB-API-compatible database) rejects transactions started with readConcern: snapshot with Error code 115 (CommandNotSupported), which makes the mongodb backend unusable against it beyond CreateTable.

Change

Adds storage.mongodb.transaction_read_concern (default: "snapshot", preserving current behavior) so operators can set "majority" or "local" to run against backends that don't implement snapshot reads, trading off snapshot isolation for compatibility. Also updates the MongoDB backend design doc (docs/design/13-storage-mongodb.md) to describe the new setting.

As a side effect, this widens ServerComponentsFactory's StorageConfig trait-object bound from an elided reference lifetime to 'static. StorageConfig::as_any()'s downcast is already documented as the intended mechanism for backend-specific settings like this one, but Any::downcast_ref requires the pointee to be provably 'static — an unannotated &dyn StorageConfig doesn't satisfy that, so the downcast wouldn't compile without this change. All existing callers already own 'static data (Box<dyn StorageConfig>), so this is not a breaking change for callers.

Validation

cargo build --workspace, cargo test (all touched crates pass), cargo clippy --workspace -- -D warnings clean.
Verified end-to-end against a real DocumentDB 0.116.0 instance (built from source to pick up an unrelated upstream Postgres-extension fix) with transaction_read_concern = "majority": CreateTable, PutItem, GetItem, and DeleteTable all succeeded via the DynamoDB API surface.
Default behavior (real MongoDB, transaction_read_concern unset) is unchanged — still snapshot.

Trade-offs

Running with majority/local instead of snapshot weakens isolation between concurrent transactions on backends that use this setting — concurrent transactions may observe a less consistent view of the data than under true snapshot reads. This is called out in both the sample config and the design doc.

The MongoDB storage backend hardcodes readConcern: snapshot on every multi-document transaction (conditional writes, TransactWriteItems, TransactGetItems, idempotency tokens). Real MongoDB (7.0+) supports this, but some MongoDB-wire-compatible servers do not. Concretely, DocumentDB (github.com/documentdb/documentdb) rejects transactions started with readConcern: snapshot with 'Error code 115 (CommandNotSupported)', which made the mongodb backend unusable against it beyond CreateTable. Add storage.mongodb.transaction_read_concern (default: "snapshot", preserving current behavior) so operators can set "majority" or "local" to run against backends that don't implement snapshot reads, trading off snapshot isolation for compatibility. Validated end-to-end against DocumentDB 0.116.0 (PutItem/GetItem/CreateTable/DeleteTable all succeed with transaction_read_concern = "majority"). Also widens the StorageConfig trait-object bound used by ServerComponentsFactory from an elided reference lifetime to 'static, since StorageConfig::as_any()'s downcast (already documented as intended for backend-specific settings like this one) requires it -- without it, Any::downcast_ref cannot be proven sound against an unannotated &dyn StorageConfig. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1de8a53e-4145-4deb-8d82-294a4879046e Signed-off-by: German <geeichbe@microsoft.com>

Updates the MongoDB backend design doc (13-storage-mongodb.md) to reflect that transaction read concern is now configurable via storage.mongodb.transaction_read_concern, rather than unconditionally snapshot. Left the RFC (docs/rfcs/0000-mongodb-backend.md) as a point-in-time design record, matching this repo's convention for RFCs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1de8a53e-4145-4deb-8d82-294a4879046e Signed-off-by: German <geeichbe@microsoft.com>

LeeroyHannigan

Thanks for this, the plumbing is thorough: all five transaction sites covered, catalog reads untouched, and the 'static widening checks out. A few things before it lands:

LeeroyHannigan · 2026-09-02T09:42:22Z

+        "snapshot" => Ok(mongodb::options::ReadConcern::snapshot()),
+        "majority" => Ok(mongodb::options::ReadConcern::majority()),
+        "local" => Ok(mongodb::options::ReadConcern::local()),
+        "linearizable" => Ok(mongodb::options::ReadConcern::linearizable()),


Blocking linearizable and available pass startup validation here but MongoDB rejects both inside multi-document transactions, so an operator who sets them gets exactly the opaque runtime failure this gate exists to prevent. Shrinking the allowlist to snapshot | majority | local (or rejecting the two with a message saying they're invalid in transactions) keeps the fail-at-startup promise.

Fixed in 91af441. Startup validation now accepts only snapshot, majority, and local; linearizable, available, and unknown values fail with a clear error listing the transaction-safe choices.

LeeroyHannigan · 2026-09-02T09:42:22Z

+    /// change this if the target deployment's MongoDB-compatible server
+    /// does not support snapshot reads.
+    #[serde(default = "default_transaction_read_concern")]
+    pub transaction_read_concern: String,


Blocking There's no test coverage on this: nothing pins the parser (accept/reject/case) and nothing asserts the configured value actually reaches TransactionOptions, so a refactor reverting one of the five sites back to the literal stays green. A parser unit test plus one majority-mode transaction test would cover it.

Fixed in 91af441. Added parser/default/unknown-field tests and centralized read/write TransactionOptions construction across all five transaction sites. The focused test asserts configured majority reaches both option shapes. The MongoDB crate suite passes (103 tests total).

LeeroyHannigan · 2026-09-02T09:42:22Z

+that request it with `CommandNotSupported` (error code 115); set this to
+`"majority"` or `"local"` to run against such targets. Doing so weakens
+isolation between concurrent transactions relative to `"snapshot"` — see
+§5.1 for what that trades away.


Should-fix Off-default values change externally observable semantics (TransactGetItems stops being a point-in-time snapshot, and under local a condition can be evaluated against data that rolls back on failover), which belongs in differences-from-dynamodb.md. Also this pointer lands on session mechanics, and 5.1 still promises WriteConflict-not-stale-reads unconditionally, which stops being true off snapshot.

Fixed in 91af441. Added the externally observable divergence to differences-from-dynamodb.md and revised the MongoDB design so point-in-time/WriteConflict guarantees are explicitly limited to snapshot; majority/local weakening and local failover rollback risk are documented.

LeeroyHannigan · 2026-09-02T09:42:22Z

+    /// view of the data than they would under snapshot isolation. Only
+    /// change this if the target deployment's MongoDB-compatible server
+    /// does not support snapshot reads.
+    #[serde(default = "default_transaction_read_concern")]


Should-fix The postgres config has deny_unknown_fields but this one doesn't, so a typo'd transaction_read_concern key silently defaults to snapshot, which for a DocumentDB user is the exact failure this knob exists to avoid.

Fixed in 91af441. MongoStorageConfig now uses #[serde(deny_unknown_fields)], with a regression test proving a typoed transaction_read_concern key fails configuration loading instead of falling back to snapshot.

Restrict transaction read concerns to transaction-safe values, centralize transaction options, reject configuration mistakes, and document weaker non-snapshot semantics. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c9b74922-c628-425e-bf1b-484b223c52b5 Signed-off-by: German <geeichbe@microsoft.com>

…n-read-concern' into geeichbe/configurable-transaction-read-concern

xgerman · 2026-09-02T21:34:03Z

Addressed all review feedback in 91af441 and pushed the branch after integrating current main.

Validation completed:

cargo fmt --all -- --check
cargo test -p extenddb-storage-mongodb (103 tests)
Clippy with warnings denied before the upstream merge
isolated DocumentDB playground deployment using transaction_read_concern=majority
CRUD, TransactWriteItems, and TransactGetItems all succeeded
no snapshot/CommandNotSupported code 115 errors

The playground documentation and smoke test were also updated in the sibling operator working tree; those changes are not part of this PR.

LeeroyHannigan · 2026-09-03T12:52:54Z

Should-fix (and the final ask, promise): a loud startup warning when a weaker read concern is configured.

Thanks for the quick iterations on the earlier rounds, the fixes all landed clean. One last thing before this merges: when transaction_read_concern is set to majority or local, the server should say so at startup. A one-line TOML change that weakens transaction isolation shouldn't be silent. It needs to land in both places: the boot banner path, and a warn! through tracing so daemon, syslog, and npm-launcher users (who never see the banner) get it too.

Heads up that the startup banner is about to change (#329 adds a wordmark), so here's roughly what I have in mind once that lands:

    ______     __                 ______  ____
   / ____/  __/ /____  ____  ____/ / __ \/ __ )
  / __/ | |/_/ __/ _ \/ __ \/ __  / / / / __  |
 / /____>  </ /_/  __/ / / / /_/ / /_/ / /_/ /
/_____/_/|_|\__/\___/_/ /_/\__,_/_____/_____/

  DynamoDB-compatible: any SDK, CLI, or tool, unchanged.

extenddb 0.1.11 (catalog 12) starting on 0.0.0.0:18080
  storage: mongodb (mongodb://127.0.0.1:27017/extenddb)

  WARNING: transaction_read_concern = "majority" (default: "snapshot").
  Transactions run below DynamoDB isolation: reads inside a transaction
  are not point-in-time consistent, and TransactGetItems may not return
  a consistent snapshot. Writes remain atomic and durable. Remove the
  setting to restore full DynamoDB transaction semantics.

Exact wording is yours to shape; the points that matter are naming the setting, naming what's weakened, being clear that writes stay atomic and durable, and saying how to get back to full semantics.

…le-transaction-read-concern

Show backend startup warnings in the interactive banner and tracing logs when MongoDB transactions use majority or local read concern. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c9b74922-c628-425e-bf1b-484b223c52b5 Signed-off-by: German <geeichbe@microsoft.com>

xgerman · 2026-09-03T15:39:12Z

Addressed the final startup-warning request in 9d7d729.

merged the startup wordmark from current main
added a backend-agnostic StartupWarning hook with a default no-warning implementation
MongoDB emits an advisory for majority and local, but not snapshot
the warning appears in the pre-daemon startup banner and as a one-line tracing::warn! event after logging initializes
the message names the configured/default setting, explains lost point-in-time read semantics, confirms writes remain atomic and durable, and explains how to restore full semantics
local additionally calls out failover rollback risk

Affected crate tests and Clippy pass.

German and others added 2 commits August 25, 2026 17:11

xgerman requested review from LeeroyHannigan, amrith, c33howard, jcshepherd, pdf-amzn, robinnsc and yesyayen as code owners August 26, 2026 04:10

xgerman added 2 commits August 26, 2026 08:22

Merge branch 'main' into geeichbe/configurable-transaction-read-concern

8df81c0

Merge branch 'main' into geeichbe/configurable-transaction-read-concern

059ffea

LeeroyHannigan reviewed Sep 2, 2026

View reviewed changes

German and others added 2 commits September 2, 2026 14:33

Merge remote-tracking branch 'origin/geeichbe/configurable-transactio…

a6bd975

…n-read-concern' into geeichbe/configurable-transaction-read-concern

xgerman mentioned this pull request Sep 2, 2026

feat(playground): run ExtendDB on DocumentDB documentdb/documentdb-kubernetes-operator#454

Draft

German and others added 2 commits September 3, 2026 08:36

Merge remote-tracking branch 'upstream/main' into geeichbe/configurab…

3fb2559

…le-transaction-read-concern

Navigation Menu

Uh oh!

feat(mongodb): make transaction read concern configurable - #314

feat(mongodb): make transaction read concern configurable#314
xgerman wants to merge 8 commits into
ExtendDB:mainfrom
xgerman:geeichbe/configurable-transaction-read-concern

xgerman commented Aug 26, 2026

Uh oh!

LeeroyHannigan left a comment

Uh oh!

LeeroyHannigan Sep 2, 2026

Uh oh!

xgerman Sep 2, 2026

Uh oh!

LeeroyHannigan Sep 2, 2026

Uh oh!

xgerman Sep 2, 2026

Uh oh!

LeeroyHannigan Sep 2, 2026

Uh oh!

xgerman Sep 2, 2026

Uh oh!

LeeroyHannigan Sep 2, 2026

Uh oh!

xgerman Sep 2, 2026

Uh oh!

xgerman commented Sep 2, 2026

Uh oh!

LeeroyHannigan commented Sep 3, 2026

Uh oh!

xgerman commented Sep 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

xgerman commented Aug 26, 2026

Problem

Change

Validation

Trade-offs

Uh oh!

LeeroyHannigan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xgerman commented Sep 2, 2026

Uh oh!

LeeroyHannigan commented Sep 3, 2026

Uh oh!

xgerman commented Sep 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants