v0.17.3 - Updated spec runner for replica-set CI

Spec runner fix for a replica-set CI flake after v0.17.2. Driver code is unchanged, just CI code.

v0.17.2 turned leftover failCommand off with directConnection. The third replica-set GitHub run then failed once: logging-replicaset.json Failing heartbeat, waitForEvent for serverHeartbeatFailedEvent got 0 in 10s. The client uses the default 10s heartbeat. Awaitable hello does not see failCommand until that wait ends, and waitForEvent is also 10s.

This release:

• After a hello handshake-error failPoint, aborts in-progress monitor hello on the matching appName client when that client observes serverHeartbeatFailedEvent
• Leaves backpressure tests alone (they freeze monitors on purpose)
• Does not copy that failPoint to every replica-set member

On this run, Failing heartbeat was 101ms (was ~10s).

GitHub Docker (CRYSTAL_WORKERS=2), 0 failures. Same example / pending counts as v0.17.2:

Topology	Time	Examples	Pending
standalone	1:54	754	14
replica set	4:42	879	15
sharded	6:51	868	15
load-balanced	4:23	822	15

Same time band as v0.17.1 / v0.17.2 (2:01 / 4:32 / 6:55 / 4:03).

Upgrade: applications on 0.17.2 do not need this release unless they run this repo's spec suite. Next work is Phase 3.14 (performance).

v0.17.2 - Updated spec runner for replica-set CI

Spec runner fix for a replica-set CI flake after v0.17.1. The production driver is unchanged.

v0.17.1 locked one UTF file at a time. That lock held but new runs still failed with extra insert events: leftover failCommand on an Unknown member, and a hello failPoint on the URI seed that also matched CSOT blockConnection.

This release:

• Arms the URI-seed hello failPoint only for handshake errors (errorCode or closeConnection)
• Turns off failCommand on every mongod with directConnection, including Unknown / paused pools
• Waits for a w:1 insert before waitForPrimaryChange returns

GitHub Docker (CRYSTAL_WORKERS=2), 0 failures. Same example / pending counts and time band as v0.17.1:

Topology	Time	Examples	Pending
standalone	2:01	754	14
replica set	4:32	879	15
sharded	6:55	868	15
load-balanced	4:03	822	15

Upgrade: applications on 0.17.1 do not need this release unless they run this repo's spec suite. Next work is Phase 3.14 (performance).

v0.17.1 - Updated spec runner for replica-set CI

Spec runner fix for GitHub replica-set CI. The production driver is unchanged from v0.17.0.

After v0.17.0, crystal spec with CRYSTAL_WORKERS=2 failed on replica set: extra insert events, extra checkouts, and a missing Unknown on monitor handshake.

Two UTF files (and prose failCommand tests) were overlapping failCommand, killAllSessions, and replSetStepDown on one mongod.

Event lists were not pinned, so a closed client could append into the next test. waitForPrimaryChange returned before the new primary accepted writes. A hello failPoint on the primary does not hit a secondary URI seed.

This release:

• Locks one UTF JSON file at a time (CMAP files and prose failCommand tests use the same lock)
• Pins UTF event lists per client under a mutex
• Waits until the new primary is writable before waitForPrimaryChange returns
• Arms a hello failPoint on the URI seed only when that appName client does not exist yet

GitHub Docker (CRYSTAL_WORKERS=2), 0 failures:

Topology	Time	Examples	Pending
standalone	2:01	754	14
replica set	4:32	879	15
sharded	6:55	868	15
load-balanced	4:03	822	15

Same time band as v0.17.0 / 3.13.3.

Upgrade: applications on 0.17.0 do not need this release unless they run this repo's spec suite. Next work is Phase 3.14 (performance).

v0.17.0 - Phase 3

Phase 3 for MongoDB 8.0 and Crystal 1.21. Core CRUD, sessions, transactions, CSOT, and load balancer are in place.

• CMAP and SDAM: pool at discovery, monitor hello, heartbeats, backpressure, interrupt in-use
• UTF specifications are only counted in topologies in which they are valid, and omitted otherwise
• CRUD 8.0: client bulkWrite, let, count, mapReduce, GridFS names, drop
• Auth, TLS key password
• Full compression options landed: zlib / snappy / zstd
• Spec logs, CLAM, CMAP JSON
• Fiber-local session, overload retargeting, handshake retry on a pinned mongos (DRIVERS-2032)

GitHub specs, 0 failures: standalone 2:03 / 14 pending / 754; replica set 4:33 / 15 / 879; sharded 7:10 / 15 / 868; load-balanced 4:23 / 15 / 822.

1.0 waits on client-side encryption. AWS / OIDC come after that. Next work is Phase 3.14 (performance).

See CHANGELOG.md for the full list.

v0.16.0 - Phase 2 (cloud, CSOT, load balancers)

Phase 2 of the roadmap is done.

CI tests covering standalone, replica set, sharded, and load-balanced topologies.

Test matrix (GitHub, 766 examples, 0 failures)

Topology	Time	Pending
standalone	23.86s	219
replica set	3:36	104
sharded	7:05	112
load-balanced	2:58	137

Added

• auth: SASLprep (RFC 4013) for SCRAM-SHA-256 passwords. Printable ASCII is unchanged (no extra allocation). Usernames are not prepared.
• uri: timeoutMS (CSOT deadline), srvMaxHosts, srvServiceName. mongodb+srv URI validation for loadBalanced, replicaSet, and directConnection. timeoutMS=0 means infinite. Negative timeoutMS is rejected.
• sdam: SRV polling fiber for mongodb+srv:// on Sharded or Unknown. Adds and removes mongos hosts. Does not run in load-balanced mode. Monitor hellos keep the last 10 RTT samples for CSOT min RTT.
• csot: Remaining timeoutMS minus min RTT is sent as maxTimeMS. Server code 50 (MaxTimeMSExpired) becomes Error::Timeout. timeoutMS on the client, database, collection, CRUD, indexes, GridFS, commit_transaction, and abort_transaction. timeoutMode (cursorLifetime / iteration) on find, aggregate, listCollections, and listIndexes. Tailable awaitData sends maxTimeMS on find and caps getMore with maxAwaitTimeMS. Change streams use iteration timeouts and resume in the same next call. GridFS uses one deadline for the whole upload/download (including listIndexes before the first write). Retryable reads/writes retry until timeoutMS (or forever if timeoutMS=0). Wait-queue timeout is Error::Timeout when timeoutMS is set. wTimeoutMS is omitted when timeoutMS is set. Official CSOT UTF is 28 files.
• run command: Database#run_command and #run_cursor_command. The caller's document is copied. Not retryable. Database read/write concern is not applied. timeoutMode and cursorType follow CSOT. batch_size, comment, and max_time_ms go on getMore only. Official run-command UTF is copied.
• load balancer: Do not pre-create minPoolSize sockets. Require serviceId on hello. Pin the TCP socket for a transaction and for an open cursor. Unpin returns the socket to the pool. Pool generation is per serviceId. Wait-queue timeout lists cursor / transaction / other in-use counts. poolClearedEvent includes serviceId. Command events include serviceId. CMAP connectionReadyEvent and connectionClosedEvent with a reason. UTF assertNumberConnectionsCheckedOut. Official load-balancer UTF runs (except the JSON skipReason file and one per-serviceId test that needs two mongos from HAProxy).
• testing: UTF ops iterateUntilDocumentOrError, iterateOnce, createFindCursor, runCursorCommand, createCommandCursor, GridFS upload / delete / rename, dropIndex / dropIndexes, assertNumberConnectionsCheckedOut. Official GridFS, collection-management, index-management, run-command, and CSOT JSON. CLAM redacted-commands.json and CRUD create-null-ids.json now run. GitHub uploads tmp/utf-timing.log. GitHub load-balanced runs full crystal spec (HAProxy via spec/support/run-load-balancer.sh).
• change streams: watch sends comment on aggregate and getMore (string or document). showExpandedEvents and fullDocumentBeforeChange go on $changeStream. A labeled getMore error resumes with a new aggregate (getMore is not a retryable read). maxAwaitTimeMS must be less than timeoutMS when both are set.

Changed

• Cursor finalize no longer sends killCursors or touches the pool (GC thread). Call #close, #each, or a block find.
• Load-balanced topology does not start monitor sockets. Sessions are always supported in that mode. LoadBalancer always allows retryable reads and writes (hello fields stay unset without monitors).
• getMore is not retried as a retryable read. Overload retry still runs.
• timeoutMode iteration starts a fresh timeoutMS on each next / try_next (change streams still own the deadline for resume).
• HAProxy multi-mongos uses roundrobin and health-checks the normal mongos ports (not the PROXY v2 ports).

Fixed

• Upsert reply with _id: null deserializes. Duplicate-key write errors expose code.
• GridFS rename errors when the file id is missing. UTF downloadByName honors revision.
• UTF matcher: $date canonical vs relaxed, $$type int/long with $numberInt.
• Find omits tailable and awaitData unless they are true. Sending tailable: false broke Versioned API strict (crud-api-version-1-strict.json).
• A runCommand reply with a cursor id pins the load-balanced socket (the raw BSON is scanned for cursor.id).
• UTF createCollection stores the collection entity and sends capped / size / max.
• runCommand does not send $readPreference to a standalone, even when the caller asked for a non-primary mode.
• getMore and killCursors do not get readConcern / afterClusterTime (explicit causal sessions).

Test info

Sharded takes more time than replica set because change-stream iterate on mongos often waits about 2 seconds. Load-balanced 2:58 is a full crystal spec (not the old official-LB-UTF-only job). It has more pending files, so it is faster than replica set and sharded.

v0.15.0 - Roadmap Phase 1

Phase 1 of the roadmap concluded.

This fork of elbywan/cryomongo is now safe enough for baseline work on standalone, replica set, and sharded MongoDB 8.0. GitHub Actions runs all three topologies.

Topology	Spec time	Result
standalone	12s	pass
replica set	2:16	pass
sharded	3:58	pass

pending in the spec log means the test was not run. It is not a green pass of that file.

Added

• Redact authenticate, saslStart, saslContinue, createUser, updateUser, getnonce, copydb*, and hello with speculativeAuthenticate in APM events and Log.trace.
• Handshake: backpressure: "2", OS name / architecture / version, platform, optional env metadata. First handshake uses legacy hello when Server API and load-balanced are unset. Client#append_metadata for wrapping libraries.
• Cursors: #each and block Collection#find close the cursor. Call #close if you iterate with #next. Do not rely on finalize.
• Backpressure: retry SystemOverloadedError + RetryableError with backoff. URI option maxAdaptiveRetries (default 2).
• CMAP: pool events and retry of PoolClearedError on retryable reads and writes.
• Shared server-selection / max-staleness / RTT JSON tests (no mongod).
• Prose tests: transaction write concern, SDAM RTT, PoolCleared, backpressure, find/getMore.
• GitHub Actions matrix. Local scripts/mongo-topology.sh.

Fixed

• URI options after the host with no / (mongodb://localhost:27017?k=v).
• Standalone retryable reads after closeConnection (Unknown server, then handshake).
• Mongos pin / unpin. Retry commit after a network error. killAllSessions after each unified test so a leftover sharded transaction does not block the next drop.
• Session pool close no longer holds the mutex while it sends endSessions (that crashed one sharded CI run with signal 11).

Not in this release

• Unified pool-cleared-error.json is still skipped.
• SASLprep for SCRAM-SHA-256.
• SRV polling, load-balancer socket pin, CSOT, OP_COMPRESSED.
• Official Change Stream / GridFS / index UTF suites.
• MONGODB-AWS, MONGODB-OIDC, CSFLE.

Requires Crystal 1.21+ (shard.yml allows >= 1.20.0) and MongoDB 8.0. Wire version 25. BSON: alumna/bson.cr 0.8.1.

See CHANGELOG.md and ROADMAP.md.

v0.14.0 - Performance with updated BSON shard

This release brings updates to the driver implementation with the recent and more efficient BSON v0.8.0/v0.8.1

What changed

• bson: Command bodies use one BSON.append for options and session fields. Receive reads documents with BSON.view over the OP_MSG buffer. copy_with is one builder pass.
• hello: lastWrite is a typed document. lastWriteDate is BSON::DateTime on the wire and becomes Time for max-staleness.
• apm: safe_payload builds a new document. It does not mutate the live reply (BSON.new(BSON) is a no-op).
• utf: Date match accepts both canonical $numberLong and relaxed ISO $date.
• insert: inserted_ids is ignored on BSON decode. The server does not send this field.
• bson.cr: Serializable / Array / Hash can deserialize BSON::Value after 0.8.0 (no duplicate when BSON::DateTime). This reflected in an update to alumna/bson.cr, released as v0.8.1.

What was fixed

• Replica-set hello no longer raises TypeCastError on lastWriteDate.
• Removed the unused UNACKNOWLEDGED_WRITE_PROHIBITED_OPTIONS constant. Unack writes still omit lsid and still send hint / collation / arrayFilters (UTF wants that).

v0.13.0 - Correctness for MongoDB 8.0 and Crystal 1.21

This release improves the rough edges in the codebase before continuing with the roadmap.

Added

• insert: Client-generated _id and insertedIds on insert results. insertMany is one retryable command.
• gridfs: session: on all methods. Stream #close waits for the background fiber.
• testing: Honest UTF runner (results, errors, events, outcomes). Local scripts/mongo-rs.sh and LOCAL_TESTING.md.

Changed

• cursors / change streams: Pin session and server. Tailable streams stay open on an empty getMore. find honors limit. getMore can retry.
• sessions / transactions: One implicit session for a whole bulk. Empty commit is reset. Cluster time and txn numbers are locked.
• sdam / selection: Time.instant for selection, monitor cooldown, and session idle. serverSelectionTryOnce works (default false). Stale topologyVersion errors do not mark the server Unknown.
• uri / tls / pool: Case-insensitive URI bools, typed loadBalanced, maxIdleTimeMS, hostname TLS flags.
• crystal: Sync::Mutex, Time.instant, no spawn(same_thread:)
• testing: Live UTF is 362 examples, 0 failures, about 5.5 minutes (was ~25). Unknown work is pending, not a fake pass.

Fixed

• GridFS chunk math, index names, and optional metadata. Counts return Int64. Timeout units. Tag-set match. list* can use a secondary.
• Monitor close, APM request ids, APM callbacks without the list lock, client close ends sessions first.

v0.12.0 - Versioned API & Basic Authentication

This release brings full support for Versioned API. It also adds basic authentication and makes the URI parser more robust.

Added

• Versioned API: You can now configure the ServerApi version on the client to guarantee backward compatibility with the database.
• Authentication: Added MONGODB-X509 and PLAIN (LDAP) support. Basic authentication is now ready.
• Testing: Added parallel test runners for the legacy authentication specification and the Versioned API specification.

Changed

• URI Parser: If an option appears multiple times in a connection string, the driver now uses the last value. This behavior correctly matches the MongoDB specification.
• Unix Sockets: The driver no longer changes Unix socket paths to lowercase letters.

Fixed

• URI Parser: Fixed bounds-checking issues when the parser splits connection strings.

v0.11.0 - Faster CI & Code organization

This release improves the internal code structure and reduces the time continuous integration (CI) pipeline.

There are no breaking changes to the public API.

Code Organization

Divided several large files to keep the code clean and maintainable. We separated the data models from the business logic in the following modules:

• Mongo::Bulk
• Mongo::GridFS
• Mongo::Unified::Dispatcher (Test Runner)

CI Performance

Decreased the total CI execution time from 19 minutes to 5.5 minutes. To do this, we created a new test sharding tool (Mongo::SpecSharding).

This tool reads the file size of each JSON test file. It then distributes the tests equally across 5 parallel CI runners. We also changed the test log level to :info to prevent slow text output operations during tests.

Full Changelog: v0.10.0...v0.11.0