Repository navigation
v1.26.0
Fixes and stability improvements
- Configurable snapshot socket timeout by @henriquekraemer in #739
- Fix connection stall after data fetch errors in #773
- Fix equality for table-valued functions in #779
- Fix provenance for lookups with multiple inputs in #780
- Fix MySQL replication stalls caused by silently dropped idle connections by @bean1352 in #760
Performance improvements
- Improve sync throughput by persisting subkey in MongoDB storage in #753
- Improve Postgres storage performance by avoid ts-codec unions in #775
- Incremental parameter index compacting for MongoDB storage in #759
Diagnostic improvements
- Log checkpoint_interrupted for errors while fetching data in #774
- Add optional version_label to sync configs in #756 and #781
MongoDB storage version 4
MongoDB storage version 3 is now promoted to version 4, marking the format as stable. This is still opt-in, and we encourage users to try it out.
Over storage version 2 (the current default), this has the following improvements:
- Incremental reprocessing of sync streams, only re-snapshotting data for new and updated sync streams in many cases. Right now this supports MongoDB source databases, with others to follow.
- Optionally store bulk data on S3-compatible object storage, significantly reducing overhead on the storage cluster in some cases.
- Faster clearing of old replication streams by dropping entire collections at a time, instead of deleting individual documents.
- Track storage stats attributed to individual sync streams, in addition to the overall aggregate.
- Restructure bucket compacting to be incremental, instead of a single large "run to completion job": Can run many times a day without significant overhead, can be interrupted and resumed at any time, and can run multiple compaction jobs concurrently.
- Various further performance improvements.
Opt in by specifying config.storage_version: 4 in the sync config. This triggers reprocessing, with the new data then using storage version 4.
Over storage version 3 in v1.25.0, this release contains the following changes:
- Add storage version 4 in #776
- Restructure compacting in #752
- Fix S3 stalls in #766 and #769
- Restructure sync event / custom checkpoint request support in #764
- Report S3 storage stats in #770
- Add S3 timing info for sync checkpoints in #768
- Force create new replication stream if old one is invalidated in #755
Full Changelog: v1.25.0...v1.26.0