Skip to content

v0.5.4b235

Pre-release
Pre-release

Choose a tag to compare

@jdrumgoole jdrumgoole released this 16 Jul 11:49

Point-in-time recovery, a SQL server with its own gauges, and operator parity across both servers

This is the largest SecantusDB release to date — a month of parallel work, 125 changelog entries. The headline capability is point-in-time recovery: every write already flowed through the oplog, and secantusAdmin.restoreToTimestamp now replays it to reconstruct the database exactly as it stood at any moment inside the retention window — with hot backup archives, base snapshots, and archives portable between the Python and Rust servers. The other headline is housekeeping with teeth: the daemons got distinguishable names (secantusd-py, secantusd-rs, secantusd-py-pg — the old secantusdb console script is gone), and Python 3.10 is now genuinely supported and genuinely tested in CI.

The PostgreSQL-wire SQL server graduated from experiment to measured surface. It now has two external conformance gauges of its own — psycopg 3's unmodified test suite and the SQLite-originated sqllogictest corpus — and the long tail they surfaced landed alongside them: server-side cursors over the wire, COPY inside transaction blocks, CREATE SCHEMA, LANGUAGE plpgsql function bodies, the full binary codec surface with real Postgres type OIDs, per-statement RBAC reusing the Mongo role model, and SQL's three-valued NULL semantics carried all the way down the filter-pushdown path.

On the MongoDB side, both servers picked up a wide operator-fidelity batch — the $setWindowFields operator set completed ($derivative / $integral with time units, $locf, $linearFill, $expMovingAvg, range windows), the N-ary accumulators, trigonometric and set expressions, a much larger date toolbox, and dozens of exact-error-code alignments — and the Rust server reached pymongo-suite parity with the Python server (99.5% each). Measurement grew to match: sixteen driver-conformance gauges now run weekly (C, C++, C#, Kotlin, pymongo async, psycopg, and the Rust-server gates joined this cycle), feeding a regenerated cross-driver summary and a new three-way feature-comparison page. One genuine bug that machinery caught — an awaitData wake race that could delay change-stream delivery by a full maxTimeMS — is fixed, alongside security hardening (two admin-UI CVEs, SCRAM-credential leak paths closed, constant-time token comparison).

Highlights

  • Point-in-time recovery: secantusAdmin.backupArchive / archiveBaseSnapshot / restoreArchive / restoreToTimestamp on the Python server; the same archives restore on the Rust server via secantusd-rs restore. See Recovery.
  • Daemon renames: secantusd-py (MongoDB wire, Python), secantusd-rs (MongoDB wire, Rust), secantusd-py-pg (PostgreSQL wire). The legacy secantusdb / secantus console scripts are removed.
  • SQL server conformance gauges: invoke validate-psycopg (psycopg 3's own suite) and invoke validate-slt (sqllogictest corpus), with the wire-protocol, codec, cursor, COPY, schema, and plpgsql work they drove.
  • Both servers: completed $setWindowFields operators, $topN / $bottomN / $firstN / $lastN / $maxN / $minN accumulators, $mergeObjects as an accumulator, trigonometric / set / bitwise expression operators, $dateFromParts, $toDate, timezone-aware date extraction, and mongod-exact error codes for unknown operators.
  • Rust server catch-up to parity: index-driven $lookup, views, getLog, killOp, role grants, oplog maintenance commands, IANA timezones in date formatting, and the pymongo suite at 99.5% — level with the Python server.
  • Change streams: the awaitData wake race is fixed (a write landing between the producer drain and the wait could stall delivery a full maxTimeMS); resume tokens advance per event even at batchSize 1.
  • Security: two admin-UI CVEs fixed plus a stored-XSS; admin.system.users no longer leaks SCRAM credentials; constant-time secret comparison in the PostgreSQL SCRAM and admin-token checks.
  • Conformance measurement: sixteen gauges (C, C++, C#, Kotlin, pymongo async, psycopg, sqllogictest, and the two Rust-server gates joined), weekly report + cross-driver-summary refresh, and the Feature comparison page.
  • Python 3.10 support, tested per-version in CI.
  • Process: changelog fragments (changelog.d/) and release-time version assignment ended cross-PR conflicts on docs/changelog.md and pyproject.toml.

This is the curated headline of a 125-entry release — the full engineering changelog for 0.5.4b235 (every entry, with its own prose and Added/Changed/Fixed detail) is at https://secantusdb.readthedocs.io/en/latest/changelog.html — the release body above is excerpted verbatim from it (GitHub's body-size limit rules out the full 230 KB section).