Skip to content

v1.26.0

Choose a tag to compare

@github-actions github-actions released this 04 Sep 10:36
· 53 commits to main since this release
3f28717

Fixes and stability improvements

  • Configurable snapshot socket timeout by @henriquekraemer in #739
  • Fix connection stall after data fetch errors in #773
  • Fix equality for table-valued functions in #779
  • Fix provenance for lookups with multiple inputs in #780
  • Fix MySQL replication stalls caused by silently dropped idle connections by @bean1352 in #760

Performance improvements

  • Improve sync throughput by persisting subkey in MongoDB storage in #753
  • Improve Postgres storage performance by avoid ts-codec unions in #775
  • Incremental parameter index compacting for MongoDB storage in #759

Diagnostic improvements

  • Log checkpoint_interrupted for errors while fetching data in #774
  • Add optional version_label to sync configs in #756 and #781

MongoDB storage version 4

MongoDB storage version 3 is now promoted to version 4, marking the format as stable. This is still opt-in, and we encourage users to try it out.

Over storage version 2 (the current default), this has the following improvements:

  1. Incremental reprocessing of sync streams, only re-snapshotting data for new and updated sync streams in many cases. Right now this supports MongoDB source databases, with others to follow.
  2. Optionally store bulk data on S3-compatible object storage, significantly reducing overhead on the storage cluster in some cases.
  3. Faster clearing of old replication streams by dropping entire collections at a time, instead of deleting individual documents.
  4. Track storage stats attributed to individual sync streams, in addition to the overall aggregate.
  5. Restructure bucket compacting to be incremental, instead of a single large "run to completion job": Can run many times a day without significant overhead, can be interrupted and resumed at any time, and can run multiple compaction jobs concurrently.
  6. Various further performance improvements.

Opt in by specifying config.storage_version: 4 in the sync config. This triggers reprocessing, with the new data then using storage version 4.

Over storage version 3 in v1.25.0, this release contains the following changes:

  • Add storage version 4 in #776
  • Restructure compacting in #752
  • Fix S3 stalls in #766 and #769
  • Restructure sync event / custom checkpoint request support in #764
  • Report S3 storage stats in #770
  • Add S3 timing info for sync checkpoints in #768
  • Force create new replication stream if old one is invalidated in #755

Full Changelog: v1.25.0...v1.26.0