diff --git a/specifications/retryable-reads/tests/README.rst b/specifications/retryable-reads/tests/README.rst index f0f805b0b9..bbb374de62 100644 --- a/specifications/retryable-reads/tests/README.rst +++ b/specifications/retryable-reads/tests/README.rst @@ -9,8 +9,10 @@ Retryable Reads Tests Introduction ============ -The YAML and JSON files in this directory tree are platform-independent tests -that drivers can use to prove their conformance to the Retryable Reads spec. +The YAML and JSON files in the ``legacy`` and ``unified`` sub-directories are platform-independent tests +that drivers can use to prove their conformance to the Retryable Reads spec. Tests in the +``unified`` directory are written using the `Unified Test Format <../../unified-test-format/unified-test-format.rst>`_. +Tests in the ``legacy`` directory are written using the format described below. Prose tests, which are not easily expressed in YAML, are also presented in this file. Those tests will need to be manually implemented by each driver. @@ -79,33 +81,25 @@ Each YAML file has the following keys: the default is all topologies (i.e. ``["single", "replicaset", "sharded", "load-balanced"]``). - - ``serverless``: Optional string. Whether or not the test should be run on - serverless instances imitating sharded clusters. Valid values are "require", - "forbid", and "allow". If "require", the test MUST only be run on serverless - instances. If "forbid", the test MUST NOT be run on serverless instances. If - omitted or "allow", this option has no effect. - - The test runner MUST be informed whether or not serverless is being used in - order to determine if this requirement is met (e.g. through an environment - variable or configuration option). Since the serverless proxy imitates a - mongos, the runner is not capable of determining this by issuing a server - command such as ``buildInfo`` or ``hello``. - - - ``serverless``: Optional string. Whether or not the test should be run on - serverless instances imitating sharded clusters. Valid values are "require", - "forbid", and "allow". If "require", the test MUST only be run on serverless - instances. If "forbid", the test MUST NOT be run on serverless instances. If - omitted or "allow", this option has no effect. - - The test runner MUST be informed whether or not serverless is being used in - order to determine if this requirement is met (e.g. through an environment - variable or configuration option). Since the serverless proxy imitates a - mongos, the runner is not capable of determining this by issuing a server - command such as ``buildInfo`` or ``hello``. + - ``serverless``: (optional): Whether or not the test should be run on Atlas + Serverless instances. Valid values are "require", "forbid", and "allow". If + "require", the test MUST only be run on Atlas Serverless instances. If + "forbid", the test MUST NOT be run on Atlas Serverless instances. If omitted + or "allow", this option has no effect. + + The test runner MUST be informed whether or not Atlas Serverless is being + used in order to determine if this requirement is met (e.g. through an + environment variable or configuration option). + + Note: the Atlas Serverless proxy imitates mongos, so the test runner is not + capable of determining if Atlas Serverless is in use by issuing commands + such as ``buildInfo`` or ``hello``. Furthermore, connections to Atlas + Serverless use a load balancer, so the topology will appear as + "load-balanced". - ``database_name`` and ``collection_name``: Optional. The database and collection to use for testing. - + - ``bucket_name``: Optional. The GridFS bucket name to use for testing. - ``data``: The data that should exist in the collection(s) under test before @@ -113,19 +107,28 @@ Each YAML file has the following keys: into the collection under test (i.e. ``collection_name``); however, this field may also be an object mapping collection names to arrays of documents to be inserted into the specified collection. - + - ``tests``: An array of tests that are to be run independently of each other. Each test will have some or all of the following fields: - ``description``: The name of the test. - + - ``clientOptions``: Optional, parameters to pass to MongoClient(). - - ``useMultipleMongoses`` (optional): If ``true``, the MongoClient for this - test should be initialized with multiple mongos seed addresses. If ``false`` - or omitted, only a single mongos address should be specified. This field has - no effect for non-sharded topologies. - + - ``useMultipleMongoses`` (optional): If ``true``, and the topology type is + ``Sharded``, the MongoClient for this test should be initialized with multiple + mongos seed addresses. If ``false`` or omitted, only a single mongos address + should be specified. + + If ``true``, the topology type is ``LoadBalanced``, and Atlas Serverless is + not being used, the MongoClient for this test should be initialized with the + URI of the load balancer fronting multiple servers. If ``false`` or omitted, + the MongoClient for this test should be initialized with the URI of the load + balancer fronting a single server. + + ``useMultipleMongoses`` only affects ``Sharded`` and ``LoadBalanced`` + topologies (excluding Atlas Serverless). + - ``skipReason``: Optional, string describing why this test should be skipped. - ``failPoint``: Optional, a server fail point to enable, expressed as the @@ -144,10 +147,10 @@ Each YAML file has the following keys: - ``result``: Optional. The return value from the operation, if any. This field may be a scalar (e.g. in the case of a count), a single document, or an array of documents in the case of a multi-document read. - + - ``error``: Optional. If ``true``, the test should expect an error or exception. - + - ``expectations``: Optional list of command-started events. GridFS Tests @@ -171,7 +174,7 @@ data. .. _GridFSBucket spec: https://github.com/mongodb/specifications/blob/master/source/gridfs/gridfs-spec.rst#configurable-gridfsbucket-class - + Speeding Up Tests ----------------- @@ -229,10 +232,86 @@ This test requires MongoDB 4.2.9+ for ``blockConnection`` support in the failpoi 9. Disable the failpoint. +Retrying Reads in a Sharded Cluster +=================================== + +These tests will be used to ensure drivers properly retry reads on a different +mongos. + +Retryable Reads Are Retried on a Different mongos if One is Available +--------------------------------------------------------------------- + +This test MUST be executed against a sharded cluster that has at least two +mongos instances. + +1. Ensure that a test is run against a sharded cluster that has at least two + mongoses. If there are more than two mongoses in the cluster, pick two to + test against. + +2. Create a client per mongos using the direct connection, and configure the + following fail points on each mongos:: + + { + configureFailPoint: "failCommand", + mode: { times: 1 }, + data: { + failCommands: ["find"], + errorCode: 6, + closeConnection: true + } + } + +3. Create a client with ``retryReads=true`` that connects to the cluster, + providing the two selected mongoses as seeds. + +4. Enable command monitoring, and execute a ``find`` command that is + supposed to fail on both mongoses. + +5. Asserts that there were failed command events from each mongos. + +6. Disable the fail points. + + +Retryable Reads Are Retried on the Same mongos if No Others are Available +------------------------------------------------------------------------- + +1. Ensure that a test is run against a sharded cluster. If there are multiple + mongoses in the cluster, pick one to test against. + +2. Create a client that connects to the mongos using the direct connection, + and configure the following fail point on the mongos:: + + { + configureFailPoint: "failCommand", + mode: { times: 1 }, + data: { + failCommands: ["find"], + errorCode: 6, + closeConnection: true + } + } + +3. Create a client with ``retryReads=true`` that connects to the cluster, + providing the selected mongos as the seed. + +4. Enable command monitoring, and execute a ``find`` command. + +5. Asserts that there was a failed command and a successful command event. + +6. Disable the fail point. + Changelog ========= +:2023-08-26 Add prose tests for retrying in a sharded cluster. + +:2022-04-22: Clarifications to ``serverless`` and ``useMultipleMongoses``. + +:2022-01-10: Create legacy and unified subdirectories for new unified tests + +:2021-08-27: Clarify behavior of ``useMultipleMongoses`` for ``LoadBalanced`` topologies. + :2019-03-19: Add top-level ``runOn`` field to denote server version and/or topology requirements requirements for the test file. Removes the ``minServerVersion`` and ``topology`` top-level fields, which are diff --git a/specifications/retryable-reads/tests/unified/readConcernMajorityNotAvailableYet.json b/specifications/retryable-reads/tests/unified/readConcernMajorityNotAvailableYet.json new file mode 100644 index 0000000000..8aa6a6b5e5 --- /dev/null +++ b/specifications/retryable-reads/tests/unified/readConcernMajorityNotAvailableYet.json @@ -0,0 +1,147 @@ +{ + "description": "ReadConcernMajorityNotAvailableYet is a retryable read", + "schemaVersion": "1.3", + "runOnRequirements": [ + { + "minServerVersion": "4.0", + "topologies": [ + "single", + "replicaset" + ] + }, + { + "minServerVersion": "4.1.7", + "topologies": [ + "sharded", + "load-balanced" + ] + } + ], + "createEntities": [ + { + "client": { + "id": "client0", + "useMultipleMongoses": false, + "observeEvents": [ + "commandStartedEvent" + ] + } + }, + { + "database": { + "id": "database0", + "client": "client0", + "databaseName": "retryable-reads-tests" + } + }, + { + "collection": { + "id": "collection0", + "database": "database0", + "collectionName": "readconcernmajoritynotavailableyet_test" + } + } + ], + "initialData": [ + { + "collectionName": "readconcernmajoritynotavailableyet_test", + "databaseName": "retryable-reads-tests", + "documents": [ + { + "_id": 1, + "x": 11 + }, + { + "_id": 2, + "x": 22 + }, + { + "_id": 3, + "x": 33 + } + ] + } + ], + "tests": [ + { + "description": "Find succeeds on second attempt after ReadConcernMajorityNotAvailableYet", + "operations": [ + { + "name": "failPoint", + "object": "testRunner", + "arguments": { + "client": "client0", + "failPoint": { + "configureFailPoint": "failCommand", + "mode": { + "times": 1 + }, + "data": { + "failCommands": [ + "find" + ], + "errorCode": 134 + } + } + } + }, + { + "name": "find", + "arguments": { + "filter": { + "_id": { + "$gt": 1 + } + } + }, + "object": "collection0", + "expectResult": [ + { + "_id": 2, + "x": 22 + }, + { + "_id": 3, + "x": 33 + } + ] + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "command": { + "find": "readconcernmajoritynotavailableyet_test", + "filter": { + "_id": { + "$gt": 1 + } + } + }, + "commandName": "find", + "databaseName": "retryable-reads-tests" + } + }, + { + "commandStartedEvent": { + "command": { + "find": "readconcernmajoritynotavailableyet_test", + "filter": { + "_id": { + "$gt": 1 + } + } + }, + "commandName": "find", + "databaseName": "retryable-reads-tests" + } + } + ] + } + ] + } + ] +} diff --git a/specifications/retryable-reads/tests/unified/readConcernMajorityNotAvailableYet.yml b/specifications/retryable-reads/tests/unified/readConcernMajorityNotAvailableYet.yml new file mode 100644 index 0000000000..707a62acd7 --- /dev/null +++ b/specifications/retryable-reads/tests/unified/readConcernMajorityNotAvailableYet.yml @@ -0,0 +1,68 @@ +description: "ReadConcernMajorityNotAvailableYet is a retryable read" + +schemaVersion: "1.3" + +runOnRequirements: + - minServerVersion: "4.0" + topologies: [single, replicaset] + - minServerVersion: "4.1.7" + topologies: [sharded, load-balanced] + +createEntities: + - client: + id: &client0 client0 + # Ensure the `configureFailpoint` and `find` commands are run on the same mongos + useMultipleMongoses: false + observeEvents: [ commandStartedEvent ] + - database: + id: &database0 database0 + client: *client0 + databaseName: &database0Name "retryable-reads-tests" + - collection: + id: &collection0 collection0 + database: *database0 + collectionName: &collection0Name "readconcernmajoritynotavailableyet_test" + +initialData: + - collectionName: *collection0Name + databaseName: *database0Name + documents: + - { _id: 1, x: 11 } + - { _id: 2, x: 22 } + - { _id: 3, x: 33 } + +tests: + - description: "Find succeeds on second attempt after ReadConcernMajorityNotAvailableYet" + operations: + - name: failPoint + object: testRunner + arguments: + client: *client0 + failPoint: + configureFailPoint: failCommand + mode: { times: 1 } + data: + failCommands: [ "find" ] + errorCode: 134 # ReadConcernMajorityNotAvailableYet + - name: find + arguments: + filter: { _id: { $gt: 1 } } + object: *collection0 + expectResult: + - { _id: 2, x: 22 } + - { _id: 3, x: 33 } + expectEvents: + - client: *client0 + events: + - commandStartedEvent: + command: + find: *collection0Name + filter: { _id: { $gt: 1 } } + commandName: find + databaseName: *database0Name + - commandStartedEvent: + command: + find: *collection0Name + filter: { _id: { $gt: 1 } } + commandName: find + databaseName: *database0Name diff --git a/src/MongoDB.Driver.Core/Core/Operations/RetryabilityHelper.cs b/src/MongoDB.Driver.Core/Core/Operations/RetryabilityHelper.cs index eff9c1c620..bc5a683acc 100644 --- a/src/MongoDB.Driver.Core/Core/Operations/RetryabilityHelper.cs +++ b/src/MongoDB.Driver.Core/Core/Operations/RetryabilityHelper.cs @@ -50,7 +50,7 @@ static RetryabilityHelper() __retryableWriteExceptions = new HashSet(resumableAndRetryableExceptions); - var retryableErrorCodes = new HashSet + var retryableReadAndWriteErrorCodes = new HashSet { ServerErrorCode.ExceededTimeLimit, ServerErrorCode.HostNotFound, @@ -59,9 +59,12 @@ static RetryabilityHelper() ServerErrorCode.SocketException }; - __retryableReadErrorCodes = new HashSet(retryableErrorCodes); + __retryableReadErrorCodes = new HashSet(retryableReadAndWriteErrorCodes) + { + ServerErrorCode.ReadConcernMajorityNotAvailableYet + }; - __retryableWriteErrorCodes = new HashSet(retryableErrorCodes); + __retryableWriteErrorCodes = new HashSet(retryableReadAndWriteErrorCodes); __resumableChangeStreamErrorCodes = new HashSet() { diff --git a/src/MongoDB.Driver.Core/ServerErrorCode.cs b/src/MongoDB.Driver.Core/ServerErrorCode.cs index 84e2aa3d80..99e32958b3 100644 --- a/src/MongoDB.Driver.Core/ServerErrorCode.cs +++ b/src/MongoDB.Driver.Core/ServerErrorCode.cs @@ -40,6 +40,7 @@ internal enum ServerErrorCode NotPrimaryNoSecondaryOk = 13435, NotPrimaryOrSecondary = 13436, PrimarySteppedDown = 189, + ReadConcernMajorityNotAvailableYet = 134, RetryChangeStream = 234, ShutdownInProgress = 91, SocketException = 9001,