Skip to content

Conversation

@samwillis
Copy link
Collaborator

@samwillis samwillis commented Nov 27, 2025

🎯 Changes

CI run failing with the reproduction from before fixing: https://github.com/TanStack/db/actions/runs/19739188291/job/56558506582?pr=924

Issue

In eager sync mode, a snapshot-end message arriving before the first up-to-date message was incorrectly triggering a commit, making data visible prematurely.

Root Cause

Electric adds a snapshot-end marker after the initial snapshot data in the log. However, this snapshot-end can be from a significant period before the stream is actually up to date (i.e., fully synced with the current state of the database). The existing code at line 993 in electric.ts:

if (hasUpToDate || hasSnapshotEnd) {

...would enter the commit logic for either message type. In the else branch, transactions were committed whenever snapshot-end arrived, even before the first up-to-date:

} else {
  // Normal mode or on-demand: commit transaction if one was started
  if (transactionStarted) {
    commit()
    transactionStarted = false
  }
}

This caused eager mode to show potentially stale/incomplete data to users before the stream was truly up to date.

Note: progressive mode was already protected because it buffers messages during isBufferingInitialSync() and only commits on up-to-date. on-demand mode is not affected because the log is in changes_only mode and doesn't have an initial snapshot.

Fix

Modified the commit logic to only commit on snapshot-end if:

  1. It's an up-to-date message, OR
  2. It's on-demand mode (where snapshot-end semantically means "ready"), OR
  3. The first up-to-date has already been received (hasReceivedUpToDate is true)
const shouldCommit =
  hasUpToDate ||
  syncMode === `on-demand` ||
  hasReceivedUpToDate

if (transactionStarted && shouldCommit) {
  commit()
  transactionStarted = false
}

This ensures that in eager mode, snapshot-end before the first up-to-date is ignored (no commit), while subsequent snapshot-end messages (after the first up-to-date) will correctly trigger commits.

Tests

  • Added test: should ignore snapshot-end before first up-to-date in progressive mode
  • Added test: should commit on snapshot-end in eager mode AFTER first up-to-date
  • Updated test: should NOT commit multiple snapshot-end messages before up-to-date in eager mode

✅ Checklist

  • I have followed the steps in the Contributing guide.
  • I have tested this code locally with pnpm test:pr.

🚀 Release Impact

  • This change affects published code, and I have generated a changeset.
  • This change is docs/CI/dev-only (no release).

@changeset-bot
Copy link

changeset-bot bot commented Nov 27, 2025

🦋 Changeset detected

Latest commit: 1e178db

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@tanstack/electric-db-collection Patch
@tanstack/db-collection-e2e Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new
Copy link

pkg-pr-new bot commented Nov 27, 2025

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@924

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@924

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@924

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@924

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@924

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@924

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@924

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@924

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@924

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@924

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@924

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@924

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@924

commit: 1e178db

@github-actions
Copy link
Contributor

github-actions bot commented Nov 27, 2025

Size Change: 0 B

Total Size: 87 kB

ℹ️ View Unchanged
Filename Size
./packages/db/dist/esm/collection/change-events.js 1.38 kB
./packages/db/dist/esm/collection/changes.js 977 B
./packages/db/dist/esm/collection/events.js 388 B
./packages/db/dist/esm/collection/index.js 3.24 kB
./packages/db/dist/esm/collection/indexes.js 1.1 kB
./packages/db/dist/esm/collection/lifecycle.js 1.67 kB
./packages/db/dist/esm/collection/mutations.js 2.26 kB
./packages/db/dist/esm/collection/state.js 3.43 kB
./packages/db/dist/esm/collection/subscription.js 2.55 kB
./packages/db/dist/esm/collection/sync.js 2.37 kB
./packages/db/dist/esm/deferred.js 207 B
./packages/db/dist/esm/errors.js 4.19 kB
./packages/db/dist/esm/event-emitter.js 748 B
./packages/db/dist/esm/index.js 2.64 kB
./packages/db/dist/esm/indexes/auto-index.js 742 B
./packages/db/dist/esm/indexes/base-index.js 766 B
./packages/db/dist/esm/indexes/btree-index.js 1.87 kB
./packages/db/dist/esm/indexes/lazy-index.js 1.1 kB
./packages/db/dist/esm/indexes/reverse-index.js 513 B
./packages/db/dist/esm/local-only.js 837 B
./packages/db/dist/esm/local-storage.js 2.1 kB
./packages/db/dist/esm/optimistic-action.js 359 B
./packages/db/dist/esm/paced-mutations.js 496 B
./packages/db/dist/esm/proxy.js 3.69 kB
./packages/db/dist/esm/query/builder/functions.js 733 B
./packages/db/dist/esm/query/builder/index.js 3.96 kB
./packages/db/dist/esm/query/builder/ref-proxy.js 917 B
./packages/db/dist/esm/query/compiler/evaluators.js 1.35 kB
./packages/db/dist/esm/query/compiler/expressions.js 430 B
./packages/db/dist/esm/query/compiler/group-by.js 1.8 kB
./packages/db/dist/esm/query/compiler/index.js 1.96 kB
./packages/db/dist/esm/query/compiler/joins.js 2 kB
./packages/db/dist/esm/query/compiler/order-by.js 1.25 kB
./packages/db/dist/esm/query/compiler/select.js 1.07 kB
./packages/db/dist/esm/query/expression-helpers.js 1.43 kB
./packages/db/dist/esm/query/ir.js 673 B
./packages/db/dist/esm/query/live-query-collection.js 360 B
./packages/db/dist/esm/query/live/collection-config-builder.js 5.33 kB
./packages/db/dist/esm/query/live/collection-registry.js 264 B
./packages/db/dist/esm/query/live/collection-subscriber.js 1.74 kB
./packages/db/dist/esm/query/live/internal.js 130 B
./packages/db/dist/esm/query/optimizer.js 2.56 kB
./packages/db/dist/esm/query/predicate-utils.js 2.91 kB
./packages/db/dist/esm/query/subset-dedupe.js 921 B
./packages/db/dist/esm/scheduler.js 1.3 kB
./packages/db/dist/esm/SortedMap.js 1.18 kB
./packages/db/dist/esm/strategies/debounceStrategy.js 247 B
./packages/db/dist/esm/strategies/queueStrategy.js 428 B
./packages/db/dist/esm/strategies/throttleStrategy.js 246 B
./packages/db/dist/esm/transactions.js 2.9 kB
./packages/db/dist/esm/utils.js 881 B
./packages/db/dist/esm/utils/browser-polyfills.js 304 B
./packages/db/dist/esm/utils/btree.js 5.61 kB
./packages/db/dist/esm/utils/comparison.js 852 B
./packages/db/dist/esm/utils/index-optimization.js 1.51 kB
./packages/db/dist/esm/utils/type-guards.js 157 B

compressed-size-action::db-package-size

@github-actions
Copy link
Contributor

github-actions bot commented Nov 27, 2025

Size Change: 0 B

Total Size: 3.34 kB

ℹ️ View Unchanged
Filename Size
./packages/react-db/dist/esm/index.js 225 B
./packages/react-db/dist/esm/useLiveInfiniteQuery.js 1.17 kB
./packages/react-db/dist/esm/useLiveQuery.js 1.11 kB
./packages/react-db/dist/esm/useLiveSuspenseQuery.js 431 B
./packages/react-db/dist/esm/usePacedMutations.js 401 B

compressed-size-action::react-db-package-size

@samwillis samwillis force-pushed the samwillis/ignore-snapshot-end-before-up-to-date branch from 8b977c5 to 10e4db1 Compare November 27, 2025 14:16
@samwillis samwillis changed the title WIP fix(electric-db-colleciton): ignore snapshot-end before up-to-date fix(electric-db-colleciton): ignore snapshot-end before up-to-date Nov 27, 2025
@samwillis samwillis marked this pull request as ready for review November 27, 2025 14:22
@samwillis samwillis requested a review from kevin-dp November 27, 2025 14:30
Copy link
Contributor

@kevin-dp kevin-dp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

@samwillis samwillis merged commit 03842c6 into main Nov 27, 2025
7 checks passed
@samwillis samwillis deleted the samwillis/ignore-snapshot-end-before-up-to-date branch November 27, 2025 15:28
@github-actions github-actions bot mentioned this pull request Nov 27, 2025
@github-actions
Copy link
Contributor

🎉 This PR has been released!

Thank you for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants