v0.156.8

HIGH fixes — three silent-loss paths in released code (GC-41 (i), GC-42). (1) v0.99.29 through v0.156.7: a migrate with table parallelism of 2 or more (the default is 4) into a Postgres or MySQL-family target, without --upfront-indexes, that failed on one table and was then finished with migrate --resume, could record a partly copied table as complete and indexed, so the resume skipped its unread rows at exit 0. (2) v0.103.1 through v0.156.7: on a Postgres target with a DEFERRABLE primary key, a CDC DELETE or UPDATE replayed from a source transaction that moved a key onto an existing value could delete or rewrite two rows where the source touched one. (3) Every release since v0.1.0: on a table without a key, a change to ONE of several identical rows deleted or rewrote EVERY copy on the target, on Postgres and MySQL-family targets. Also fixed: an idle Postgres 15+ stream's replication slot now follows the server's WAL instead of retaining it (MEDIUM, no data loss), and --exactly-once-lanes is now roughly as fast as --apply-concurrency 1. Upgrading does not repair data an older release already lost. Operators of the affected configurations must check their targets; read Who needs this — action required first. Not a silent drop-in: a CDC write that used to lose data now stops with SLUICE-E-CDC-KEY-MATCHED-MULTIPLE-ROWS, and an idle backup stream from a Postgres source now commits windows it used to skip. See Compatibility.

Features

sync start --exactly-once-lanes is now 0.86–0.97× the speed of --apply-concurrency 1 on a secondary-unique-heavy workload, up from about 0.5× (ADR-0190 amendment D). With the flag, before the first change of a source transaction that writes an apply mark reaches a lane, sluice drains every lane and moves the persisted position to that transaction's start. That position used to be its own synchronous target commit, followed by the lane's commit carrying the marks: two commits per such transaction. The position now rides the lane batch that carries the transaction's first marked change, written last in that batch's own transaction (data, then marks, then position), so the marks and the position become durable together. Until that batch commits, no other lane may write the transaction's marks. Measured with the new benchmarks/fencebench harness (medians of three, every cell verified against the source by a full-row hash), on transactions that all touch a secondary-unique table: MySQL → Postgres 176.2 transactions/s (0.86× serial, was 109.1), Postgres → Postgres 189.8 (0.88×, was 112.8), MySQL → MySQL 65.6 (0.86×, was 38.4), and MySQL → Postgres at a 5 ms round trip 26.6 (0.97×, was 15.1). With one transaction in five touching such a table (MySQL → Postgres only), 545.0, was 384.0 (1.42×, short of the 1.5× the design targeted, because there the drain rather than the extra commit is most of the cost). The drain remains, so the flag is still 97–99% slower than the default lanes on such a workload, and it stays off by default. Without the flag nothing changes (measured within noise of v0.156.7 on every pair). Not measured: Postgres → Postgres and MySQL → MySQL at 5 ms. One behaviour change with the flag, under Compatibility. Pinned on every lane write core and on every source × target pair of the crash suite, including the new window where the position-carrying batch has committed and the transaction's other lanes have not.

Fixed

migrate --resume after a failed parallel migrate no longer skips the unread tail of a table that was cut short (HIGH, silent loss; GC-41 (i)). A cancellation closes a copy's row stream the same way the end of a table does, and the target write loops returned success when their "stream closed" branch won the race against the cancellation with nothing left to write. A whole-table copy then reported success for a partly copied table. That became durable in migrate's overlapped copy-and-index phase, which is the default for a Postgres or MySQL-family target unless --upfront-indexes is set, with --table-parallelism of 2 or more. When one table's copy failed, only the copy pool was cancelled. A table still copying could return the false success, was handed to the still-running index builder, and the "indexes built" callback wrote its progress row on the run's live context: complete, indexes_built, and only the rows copied so far. The run then failed loudly on the other table, but a later migrate --resume skipped the cut-short table, so its tail was never copied, at exit 0. It needs the in-flight table to be on a whole-table copy (below --bulk-parallel-min-rows, without a chunkable primary key, or under --bulk-parallelism 1; the PK-range-chunked path already re-checked its context) and its writer to have had nothing buffered at the cancel, or a source reader that ends a cancelled stream without an error, as the mydumper reader does. Reproduced 20/20 by TestOverlapPhase_PeerFailureRecordsNothingForTheInFlightTable against the released code shape, with a writer that resolves the race the bad way every time; not reproduced against a real database. Not durable, so not affected: a Ctrl-C of the whole run, --upfront-indexes runs (every complete write ran on the same cancelled context, which both state stores refuse), and sync start (a restart resumes only when every table is recorded complete, which the failed table never is). The fix: every copy entry point now returns success only when the source ended naturally, the run was still live when the writer returned, and the writer consumed everything it was handed (migcore.SourceEnd), and migcore.ReaderStreamErr no longer forgives a cancelled reader (it had since v0.69.1). Only then is a table's complete row written, and since it can now only record the truth it is written on a stop-proof context. So a stop that lands just after a table finished no longer loses that row, which had sent a restart into a needless full re-copy, or, on a Postgres source at its replication-slot ceiling, into SLUICE-E-CDC-REPLICATION-HEADROOM. backup's per-table completion had the identical race (never durable) and takes the same check. Also pinned by TestStopMidCopyRecordsNoCompleteRow, TestStopMidCopy_AReaderStoppedByItsOwnShutdownIsRefused, TestTableCompleteRowSurvivesAStop and TestCopyEntryPointRoster_EveryWriteConsultsTheSourceEnd, which walks every copy entry point and fails the build if one writes without consulting the source end. The end-to-end TestStreamer_StopMidColdCopy_ThenResume_EveryRowArrives is a behaviour check that does not discriminate this defect (it stays green with the fix reverted). Affected releases: v0.99.29 (which added the cross-table copy pool and overlapped index phase, Postgres targets) through v0.156.7; MySQL-family targets from v0.99.30 (ADR-0080).

Postgres targets: a CDC UPDATE or DELETE that names its row by key no longer changes or deletes more than one row, the shape a DEFERRABLE primary key reaches mid-transaction (HIGH, silent loss; GC-42). Every UPDATE and DELETE the applier issues names its row by the change's before-image. Under a DEFERRABLE primary key, a source transaction may make two rows share a key value until it commits, and when the before-image is narrowed to that key the replayed change addresses both. BEGIN; UPDATE s SET id = 2 WHERE id = 1; DELETE FROM s WHERE u = 'b'; COMMIT; replayed as DELETE … WHERE id = 2, deleted both rows, and the commit-time check passed on the final state: the source ended with (2,a),(3,c) and the target with (3,c), at exit 0, on the default lanes and on --apply-concurrency 1, with or without the replica privilege (measured on Postgres 14 and 16). Any UPDATE or DELETE that matches more than one row is now refused before it commits, with marker KEY-SCOPED-WRITE-MATCHED-MULTIPLE-ROWS and code SLUICE-E-CDC-KEY-MATCHED-MULTIPLE-ROWS, naming the table, the key and the count, and the transaction is rolled back. Zero rows stays tolerated. The same refusal covers two other ways to address several rows by key: a before-image narrowed to columns the target does not hold unique (a source under REPLICA IDENTITY USING INDEX whose index the target lacks), and a key-narrowed before-image against a target table without a key. It is on the MySQL applier too, as a backstop; MySQL has no deferrable constraints. The loss needs a before-image narrowed to the primary key, meaning a Postgres source table under REPLICA IDENTITY FULL (sluice narrows it to the key) or a postgres-trigger source set up with --capture-payload minimal, feeding a Postgres target table with a deferrable primary key, plus a source transaction that made two rows share a key value before it committed (a key shift or swap). A source under REPLICA IDENTITY USING INDEX cannot reach it, and under REPLICA IDENTITY DEFAULT such a table is refused at cold start (SLUICE-E-SOURCE-REPLICA-IDENTITY). The key read that decides refusal versus the keyless case is lazy: it happens only when a write matched more than one row. Affected releases: v0.103.1 (the first release that carried a DEFERRABLE primary key from a Postgres source) through v0.156.7. From v0.103.1 until v0.104.0 the first INSERT to such a table killed the stream (Bug 211); since v0.104.0 the table also needs an immediate NOT NULL UNIQUE index to be applied at all (SLUICE-E-TARGET-DEFERRABLE-KEY). A key the target does not hold unique was exposed in every release.

Tables without a key: a change to ONE of several identical rows no longer deletes or rewrites every copy on the target (HIGH, silent loss; GC-42). For a table with no primary key and no NOT NULL unique index, the applier names a row by its whole before-image, and a whole-row WHERE matches every identical copy. A source that deleted one of two identical rows (1,'a'),(1,'a') (a Postgres table under REPLICA IDENTITY FULL, a trigger-CDC source, a MySQL binlog full image) sent one change, and the target deleted both; an UPDATE of one copy rewrote both, at exit 0, on every apply path. Measured through the applier on Postgres 16 and MySQL 8: per-change, batched, serial fall-back and lanes, including a partitioned Postgres table and rows holding NULLs. Such a write now reaches exactly one of the identical rows: on Postgres by (tableoid, ctid) from a LIMIT 1 sub-select, so a partitioned table's repeated ctids cannot widen it; on MySQL with LIMIT 1. Any UPDATE or DELETE that still matches more than one row is refused as KEY-SCOPED-WRITE-MATCHED-MULTIPLE-ROWS. The one-row addressing is used only when the target table has no key and the before-image covers every non-generated target column; a key-narrowed image keeps the old WHERE and falls to the refusal instead. The CDC appliers are also what the broker and a backup chain's restore replay apply through. Pinned by TestGC42_KeylessIdenticalRows (Postgres: delete, update, NULL and partitioned cases × 4 apply paths × 2 privilege modes, exact final state) and the MySQL keyless cells. Affected releases: every release since v0.1.0, Postgres and MySQL-family targets.

sync and backup stream from a Postgres source: a stream whose own tables are idle now lets its replication slot follow the server's WAL, instead of retaining WAL until the next write to a replicated table (MEDIUM, unbounded source WAL retention, no data loss; GC-41 (j)). The slot is released only to a position the target has persisted, and a position was persisted only at a decoded source commit. Postgres 15 and later skip a transaction that touches no published table, so while the stream's tables saw no writes nothing was decoded and confirmed_flush_lsn stayed put however much WAL the server wrote elsewhere: measured at 5.69 MB retained after 135 s with about 40 KB/s of other WAL, on PG 15 and PG 16, to Postgres and MySQL targets. --source-heartbeat-interval did not help on 15+, because its table is outside the publication and its transactions are skipped too; its help text, which promised the position "advances even against an idle source", had been wrong there since it shipped in v0.82.0, and now says so. The reader now treats the walsender's keepalive position as a transaction boundary whenever none of the stream's source transactions is in flight. It emits an empty boundary there, at most one per 10 s and only when the walsender's position has moved, which every apply path persists like a commit, and the slot is released to it once the target holds it. This is the rule PostgreSQL's own subscriber follows. The acknowledgement rule itself is unchanged: the slot still never passes the target's persisted position. A server that goes fully quiet still keeps the WAL written since its last activity (restart_lsn advances only at a running-transactions record), but no more. If a delivered transaction is ever seen to commit below a boundary already emitted, the reader logs KEEPALIVE-BOUNDARY-PASSED-COMMIT at ERROR and stops emitting boundaries for that connection. That is a tripwire on an uninterrupted connection, not a guard; see Not run for the premise it rests on. Pinned on real servers by the idle-source slot test and by TestStreamer_PostgresKeepaliveBoundaryStraddle_OpenTxnSurvivesResume, which holds a source transaction open across a persisted, acknowledged boundary, ends the stream (in-process stop and a killed binary, small and spilled transaction, serial and 4 lanes) and requires every row on resume: 8 of 8 cells green, and red on the silent-loss assertion when the resume position is mutated. Affected releases: every release, on Postgres 15+ sources.

sluice sync run: a fleet leg that stops on SLUICE-E-CDC-KEY-MATCHED-MULTIPLE-ROWS is not restarted (MEDIUM, loud). The new refusal repeats on every restart: the apply transaction is rolled back, and a restart replays the same source changes onto the same committed rows. Coded refusals are otherwise restarted under max-consecutive-failures, so under the default 0 such a leg would have restarted forever, the shape v0.156.7 fixed for five uncoded refusals. It now joins that list: the leg is marked failed with an ERROR line carrying the marker, and the other legs keep running. DEFERRED-KEY-CHECK-FAILED-AT-COMMIT is deliberately not listed, because one of its causes depends on arrival timing and a restart can clear it. Pinned by TestSupervisor_RefusalsARestartRepeatsAreNotRestarted (built by the real applier constructor) and, in the other direction, TestSupervisor_RefusalsARestartCanClearAreStillRestarted. Affected releases: none released; the refusal is new in this release.

Postgres targets: a DEFERRABLE constraint the apply does not re-check is now named (LOW, no data loss). The Bug 164 foreign-key bypass, session_replication_role = replica, also switches off Postgres's deferred uniqueness and exclusion re-check, which is an ordinary trigger internally. The applied rows still match the source, but a target constraint stricter than the source's can end up holding rows it forbids, and REINDEX on it would fail. sluice keeps replica mode, since leaving it would fire the target's own triggers on replicated rows, and now logs one DEFERRED-KEY-CHECK-OFF-IN-REPLICA-MODE WARN per table, read once per table per apply run, naming the deferrable constraint, the REINDEX check and the remedy (match the target's constraint to the source's). The deferred check still runs for an apply role without the replica privilege; a commit it refuses is now marked DEFERRED-KEY-CHECK-FAILED-AT-COMMIT. Affected releases: v0.99.150 through v0.156.7.

Compatibility

New loud stop: SLUICE-E-CDC-KEY-MATCHED-MULTIPLE-ROWS (marker KEY-SCOPED-WRITE-MATCHED-MULTIPLE-ROWS, exit 3). A source transaction that used to lose or overwrite target rows at exit 0 now stops the stream, with the apply transaction rolled back. Every shape it refuses was already wrong on the target. One could look harmless: a target table without a key that holds identical rows, fed a before-image that does not cover every target column (the target has a column the source does not send, or the source sends a key-narrowed image). Such a DELETE used to remove every identical copy where the source removed one; it now stops. Recover by giving the target table the source's key, or by matching its columns to the source's and re-copying it. Nothing that converged before is refused now: in replica mode a split key swap still converges, an apply role without the replica privilege still gets Postgres's own refusal for a split (now marked DEFERRED-KEY-CHECK-FAILED-AT-COMMIT), and a source under REPLICA IDENTITY USING INDEX shifting its key behaves as before.

New WARN: DEFERRED-KEY-CHECK-OFF-IN-REPLICA-MODE. Once per Postgres target table with a deferrable constraint per apply run, in replica mode only. It never stops the apply.

Keyless one-row addressing. On a MySQL target, an UPDATE or DELETE of a table without a key now carries LIMIT 1. Under STATEMENT binlog format that statement is flagged unsafe for statement-based replication (warning 1592), and a sharded Vitess keyspace may refuse it; both are loud and limited to tables without a key. On Postgres the write addresses the row by (tableoid, ctid). UNVERIFIED PREMISE: that addressing has not been tested on PlanetScale Neki (sharded Postgres).

Copy-completion semantics. A copy now counts as complete only when its source was read to the end, the run was still live when the writer returned, and the writer consumed everything it was handed. A stop that lands in the instant between a table's last commit and that check now reads as interrupted, so that table is re-copied on restart rather than resumed: a loud re-copy, never loss. A stop after the check no longer loses the complete row, so the opposite case (a needless full re-copy) is gone.

Behaviour change: an idle backup stream / backup incremental from a Postgres source now writes what it used to skip. Both capture a keepalive boundary like any transaction boundary (two change records), so an idle window in which the server wrote WAL now seals one change chunk instead of none, and its end position advances. On backup stream it also turns the window from a skipped empty rollover (the default, which skips windows with no changes) into a committed one: one manifest plus one small change chunk for each rollover window in which the server wrote WAL, where on Postgres 15+ neither was written before. Bounded by the throttle (at most one boundary pair per 10 s, so a window holds at most 2 × ⌈window / 10 s⌉ records, a 15 s window holding 4, plus one more pair when a boundary emitted while the previous window was committing lands at the start of this one), and nothing on a fully quiet server. (Corrected after publish: this said 2 × window / 10 s.) That is the cost of the slot advancing. Code-read, not benchmarked.

Idle Postgres sync streams write to the target. An idle Postgres-source stream now costs one position-only target transaction per 10 s at most; a stream whose own commits keep coming costs none. It refreshes the control row's updated_at, so sluice_seconds_since_last_apply stays low on an idle Postgres stream while the server writes WAL; alerting that keyed on it to detect an idle source should be checked. Sync lag stays unknown on an idle stream rather than reporting zero.

--source-heartbeat-interval on Postgres. The heartbeat writer is unchanged, but on Postgres it is no longer needed to keep the slot advancing while sync is connected: any WAL written anywhere on the server moves it. It still matters for MySQL sources. The help text is corrected.

--exactly-once-lanes retry. With the flag, a lane commit carrying the position whose COMMIT fails with an unknown outcome (a commit that outlived the apply's statement timeout, a connection lost during it) is no longer retried in place, since a retry of a commit that did land would count its rows twice. The apply attempt stops and the retry loop re-enters from what the target holds. FOLD-TICKET-DUPLICATE names an internal-consistency refusal that should never fire.

sluice sync run fleet behaviour. A leg that stops on SLUICE-E-CDC-KEY-MATCHED-MULTIPLE-ROWS is failed and not restarted. Alert on failed legs.

Otherwise a drop-in. No flag was removed or renamed, and no control-table or backup-format schema changed.

Not run for this release: the dispatch-only psverify (live PlanetScale, a billable cluster), d1verify (live Cloudflare D1) and nekiverify (a billable Neki cluster) suites were not dispatched, because each provisions billable or live infrastructure and runs only on the operator's call. UNVERIFIED PREMISES: the keyless (tableoid, ctid) addressing on Neki is untested. The keepalive boundary's safety rests on the walsender never sending a keepalive past the commit of a transaction it delivers later; that is cited from PostgreSQL source, not measured, and KEEPALIVE-BOUNDARY-PASSED-COMMIT only observes an uninterrupted connection. The Neki commit-order premise from v0.156.7 (whether a sharded Postgres target with the control tables on another shard commits atomically) is still unmeasured.

Who needs this — action required

Check after upgrading. Upgrading stops new loss; it does not restore rows an older release already lost.

• migrate into a Postgres target on v0.99.29–v0.156.7, or into a MySQL-family target on v0.99.30–v0.156.7, with table parallelism of 2 or more (the default auto is 4) and without --upfront-indexes, that failed on one table and was then finished with migrate --resume. A table that was mid-copy when the other one failed may be short. Run sluice verify --depth count (or --depth sample) between that source and target; a short table is the signature. Re-copy what is short, for example with a fresh migrate --reset-target-data. A run that succeeded first time, or that was finished by re-running from scratch rather than --resume, is not affected.
• CDC into a Postgres target table with a DEFERRABLE primary key, on v0.103.1–v0.156.7, fed from a Postgres source table under REPLICA IDENTITY FULL or a postgres-trigger source set up with --capture-payload minimal, whose source transactions shift or swap key values. List the candidate tables on the target with SELECT conrelid::regclass, conname FROM pg_constraint WHERE condeferrable AND contype = 'p';, compare each one's key set with the source, and re-copy any that differ. Expect the target to be missing rows the source still has.
• Anyone who has replicated a table without a primary key or NOT NULL unique index, on any release, whose source can hold identical rows (an append-only log or event table, a join table without a key) and whose source deleted or updated rows of it. Compare, per such table, the count of each distinct row between source and target: SELECT <all columns>, count(*) FROM t GROUP BY <all columns> on both sides, or a per-row hash and count. Re-copy any table whose counts differ. The target will be short of copies the source still has, or hold extra updated copies. A target fed by a restored backup chain with incrementals is in scope too.
• Postgres targets with a deferrable constraint stricter than the source's. If the new DEFERRED-KEY-CHECK-OFF-IN-REPLICA-MODE WARN names one, run the REINDEX check it prints and match the target's constraint to the source's.
• Postgres 15+ sources with periods where the replicated tables are idle but the server is not. No data was lost. After upgrading, check that pg_replication_slots.confirmed_flush_lsn follows pg_current_wal_lsn() during those periods. If you ran the heartbeat only for this, it can be turned off on Postgres sources.
• backup stream users with a Postgres source. Expect a committed window, with a manifest and a small chunk, for each idle rollover window in which the server wrote WAL; size retention and backup prune schedules accordingly.
• sluice sync run fleet operators. Alert on failed legs; SLUICE-E-CDC-KEY-MATCHED-MULTIPLE-ROWS no longer restart-loops and needs the remedy it names.
• Everyone else: upgrade; nothing to check.

---

Install: brew install sluicesync/tap/sluice · go install sluicesync.dev/sluice/cmd/sluice@v0.156.8 · Container: ghcr.io/sluicesync/sluice:0.156.8

Full changelog: https://github.com/sluicesync/sluice/blob/main/CHANGELOG.md