[Postgres] Improve handling of "lost" replication slots

[rkistner]

This adds a check for replication slots that are "lost" due to max_slot_wal_keep_size exceeded, to automatically re-create the slot if needed. There is an explicit wal_status field since Postgres 13+ that we now check, as well as some additional error message checks.

The slot health check is now also modified to only wait 2 minutes for the slot to become healthy, rather than 120 tries. This is relevant because each individual try can take 2 minutes in some scenarios, which can cause the overall check to only fail after 4 hours.

This does not yet solve the issue of this health check potentially causing high load on the source database. For that we should probably use an exponential back-off mechanism for the overall replication retries (separate PR).

[changeset-bot]

🦋 Changeset detected

Latest commit: 72b12b1

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages

Name	Type
@powersync/service-module-postgres	Patch
@powersync/lib-service-postgres	Patch
@powersync/service-schema	Patch
@powersync/service-image	Patch
@powersync/service-module-postgres-storage	Patch
@powersync/service-module-mongodb	Patch
@powersync/service-module-mysql	Patch
@powersync/service-core	Patch
@powersync/service-core-tests	Patch
@powersync/service-module-core	Patch
@powersync/service-module-mongodb-storage	Patch
test-client	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[stevensJourney]

Looks good to me :)

The base branch was changed.