Skip to content

Conversation

@samwillis
Copy link
Collaborator

@samwillis samwillis commented Nov 24, 2025

Fixes #813

Failed test run showing reproduction: https://github.com/TanStack/db/actions/runs/19632437525/job/56215175357?pr=898


Summary

Fix scheduler deadlocks from lazy left-join sources (e.g., the nested left join in the “should handle optimistic mutations with nested left joins without scheduler errors” test) by only blocking on dependencies that are actually enqueued or have pending work in the same transaction. Non-enqueued lazy deps are treated as satisfied, preventing unresolved dependency errors.

Background: How the scheduler and live queries interact

  • The scheduler is a simple per-transaction queue. Each job is enqueued with optional dependencies. A job runs only after all its listed dependencies have run in that same context. If a dependency is listed but no such job is enqueued, the scheduler cannot make progress and throws Scheduler detected unresolved dependencies for context ....
  • Live query builders schedule their own jobs per transaction. Dependencies are used so parent queries run before dependent queries when both are scheduled.
  • Lazy join sources (e.g., right side of a left join) may not enqueue work in a transaction; they often load data on-demand inside another builder’s run. If a consumer adds such a lazy source as a hard dependency but the lazy source never enqueues a job, the scheduler waits forever.

Original issue

  • Deadlock with lazy left joins (modeled by “should handle optimistic mutations with nested left joins without scheduler errors”): a consumer live query declared the lazy right side of a left join as a dependency, but that lazy source didn’t enqueue work in the transaction. The scheduler blocked waiting for a non-existent job and threw the unresolved-dependency error.
  • Stale-read scenario (second new test): when a lazy source does have pending work but runs after its consumer, the consumer can read stale data if it isn’t ordered behind the lazy source. The test exercises this by keeping the transaction open.

What changed (code)

  • packages/db/src/scheduler.ts
    • Added a PendingAwareJob type and isPendingAwareJob guard to remove any casting when checking for pending work.
    • Dependency resolution now treats a dependency as blocking only if:
      • The dep job is enqueued and not yet completed, or
      • The dep reports a pending run in the same context via hasPendingGraphRun.
      • Otherwise, the dependency is considered satisfied. This avoids deadlocks on lazy sources that never schedule work while still ordering against deps that actually have work.
  • packages/db/src/query/live/collection-config-builder.ts
    • Provides hasPendingGraphRun so dependents can detect when a builder has work pending in a transaction (used by the scheduler dependency logic).
  • Test adjustment (no API change)
    • The stale-read test uses autoCommit: false to keep the transaction open during the race, ensuring the scheduler ordering is exercised without dropping optimistic state prematurely.

Why this fixes the problem

  • Deadlocks avoided: lazy-only dependencies that aren’t enqueued no longer block the scheduler; the dependency edge is treated as satisfied if there is no job or pending work.
  • Ordering preserved when needed: if a lazy source is actually scheduled or has pending work, the dependency still blocks, so the consumer won’t outrun it.
  • Type safety: dependency checks no longer use duck typing; the scheduler knows how to ask a job if it has pending work.

Testing

  • pnpm vitest packages/db/tests/query/scheduler.test.ts -t "live query scheduler" (all 10 tests now pass, covering the deadlock scenario and the stale-read race).

Notes

  • No changes to public API.
  • Changeset: .changeset/soft-lazy-deps.md (patch for @tanstack/db).

@changeset-bot
Copy link

changeset-bot bot commented Nov 24, 2025

🦋 Changeset detected

Latest commit: 0861ff9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 11 packages
Name Type
@tanstack/db Patch
@tanstack/angular-db Patch
@tanstack/db-collection-e2e Patch
@tanstack/electric-db-collection Patch
@tanstack/powersync-db-collection Patch
@tanstack/react-db Patch
@tanstack/rxdb-db-collection Patch
@tanstack/solid-db Patch
@tanstack/svelte-db Patch
@tanstack/trailbase-db-collection Patch
@tanstack/vue-db Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@samwillis samwillis force-pushed the samwillis/scheduler-bug branch from 2123063 to d06f565 Compare November 24, 2025 11:17
@pkg-pr-new
Copy link

pkg-pr-new bot commented Nov 24, 2025

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@898

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@898

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@898

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@898

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@898

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@898

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@898

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@898

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@898

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@898

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@898

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@898

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@898

commit: 0861ff9

@github-actions
Copy link
Contributor

github-actions bot commented Nov 24, 2025

Size Change: +150 B (+0.17%)

Total Size: 86.5 kB

Filename Size Change
./packages/db/dist/esm/query/live/collection-config-builder.js 5.33 kB +67 B (+1.27%)
./packages/db/dist/esm/scheduler.js 1.3 kB +83 B (+6.84%) 🔍
ℹ️ View Unchanged
Filename Size
./packages/db/dist/esm/collection/change-events.js 1.38 kB
./packages/db/dist/esm/collection/changes.js 977 B
./packages/db/dist/esm/collection/events.js 388 B
./packages/db/dist/esm/collection/index.js 3.24 kB
./packages/db/dist/esm/collection/indexes.js 1.1 kB
./packages/db/dist/esm/collection/lifecycle.js 1.67 kB
./packages/db/dist/esm/collection/mutations.js 2.26 kB
./packages/db/dist/esm/collection/state.js 3.43 kB
./packages/db/dist/esm/collection/subscription.js 2.48 kB
./packages/db/dist/esm/collection/sync.js 2.37 kB
./packages/db/dist/esm/deferred.js 207 B
./packages/db/dist/esm/errors.js 4.19 kB
./packages/db/dist/esm/event-emitter.js 748 B
./packages/db/dist/esm/index.js 2.64 kB
./packages/db/dist/esm/indexes/auto-index.js 742 B
./packages/db/dist/esm/indexes/base-index.js 766 B
./packages/db/dist/esm/indexes/btree-index.js 1.87 kB
./packages/db/dist/esm/indexes/lazy-index.js 1.1 kB
./packages/db/dist/esm/indexes/reverse-index.js 513 B
./packages/db/dist/esm/local-only.js 837 B
./packages/db/dist/esm/local-storage.js 2.1 kB
./packages/db/dist/esm/optimistic-action.js 359 B
./packages/db/dist/esm/paced-mutations.js 496 B
./packages/db/dist/esm/proxy.js 3.22 kB
./packages/db/dist/esm/query/builder/functions.js 733 B
./packages/db/dist/esm/query/builder/index.js 3.96 kB
./packages/db/dist/esm/query/builder/ref-proxy.js 917 B
./packages/db/dist/esm/query/compiler/evaluators.js 1.35 kB
./packages/db/dist/esm/query/compiler/expressions.js 430 B
./packages/db/dist/esm/query/compiler/group-by.js 1.8 kB
./packages/db/dist/esm/query/compiler/index.js 1.96 kB
./packages/db/dist/esm/query/compiler/joins.js 2 kB
./packages/db/dist/esm/query/compiler/order-by.js 1.25 kB
./packages/db/dist/esm/query/compiler/select.js 1.07 kB
./packages/db/dist/esm/query/expression-helpers.js 1.43 kB
./packages/db/dist/esm/query/ir.js 673 B
./packages/db/dist/esm/query/live-query-collection.js 360 B
./packages/db/dist/esm/query/live/collection-registry.js 264 B
./packages/db/dist/esm/query/live/collection-subscriber.js 1.74 kB
./packages/db/dist/esm/query/live/internal.js 130 B
./packages/db/dist/esm/query/optimizer.js 2.56 kB
./packages/db/dist/esm/query/predicate-utils.js 2.88 kB
./packages/db/dist/esm/query/subset-dedupe.js 921 B
./packages/db/dist/esm/SortedMap.js 1.18 kB
./packages/db/dist/esm/strategies/debounceStrategy.js 247 B
./packages/db/dist/esm/strategies/queueStrategy.js 428 B
./packages/db/dist/esm/strategies/throttleStrategy.js 246 B
./packages/db/dist/esm/transactions.js 2.9 kB
./packages/db/dist/esm/utils.js 881 B
./packages/db/dist/esm/utils/browser-polyfills.js 304 B
./packages/db/dist/esm/utils/btree.js 5.61 kB
./packages/db/dist/esm/utils/comparison.js 852 B
./packages/db/dist/esm/utils/index-optimization.js 1.51 kB
./packages/db/dist/esm/utils/type-guards.js 157 B

compressed-size-action::db-package-size

@github-actions
Copy link
Contributor

github-actions bot commented Nov 24, 2025

Size Change: 0 B

Total Size: 3.34 kB

ℹ️ View Unchanged
Filename Size
./packages/react-db/dist/esm/index.js 225 B
./packages/react-db/dist/esm/useLiveInfiniteQuery.js 1.17 kB
./packages/react-db/dist/esm/useLiveQuery.js 1.11 kB
./packages/react-db/dist/esm/useLiveSuspenseQuery.js 431 B
./packages/react-db/dist/esm/usePacedMutations.js 401 B

compressed-size-action::react-db-package-size

@samwillis samwillis marked this pull request as ready for review November 24, 2025 15:25
@samwillis samwillis changed the title WIP fix for schedular bug fix for schedular bug Nov 24, 2025
@samwillis samwillis requested a review from kevin-dp November 24, 2025 15:31
@samwillis samwillis changed the title fix for schedular bug fix for schedular bug with lazy left-join sources Nov 24, 2025
return (
typeof dep === `object` &&
dep !== null &&
typeof (dep as { hasPendingGraphRun?: unknown }).hasPendingGraphRun ===
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes more sense to type this function's dep argument as any such that you don't need this typecast here.

@samwillis samwillis merged commit c8a2c16 into main Nov 25, 2025
7 checks passed
@samwillis samwillis deleted the samwillis/scheduler-bug branch November 25, 2025 12:44
@github-actions github-actions bot mentioned this pull request Nov 25, 2025
@github-actions
Copy link
Contributor

🎉 This PR has been released!

Thank you for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduler detected unresolved dependencies v0.5.0

3 participants