Add optional compareOptions to the Collection Config which act as col… #762

kevin-dp · 2025-11-05T10:22:54Z

This PR adds support for collection-wide string ordering/locale configuration. It is possible to override these in a specific query by providing specific compareOptions in the query's orderBy clause. The priority is as follows (high to low):

query provided compareOptions
collection's compareOptions if none specified in query
default compare options (machine's locale) if the user didn't pass compareOptions when creating the collection

The idea behind providing collection-wide locale configuration is that most applications should use the same locale configuration as their backend. This is needed because tanstack DB does a mix of loading ordered data from the backend and loading ordered data that is available locally. Hence, the local ordering must be the same as the one on the backend otherwise we get inconsistent behavior.

Implementation

High-level

We store the collection-wide StringSortOpts in the collection itself. If none are provided, we store the default locale configuration.
Modified the builder such that it no longer hydrates the orderBy clause with defaults for the locale, if no locale is specified, we keep them undefined in the orderBy clause
At runtime, when processing an ordered query, we build the locale configuration from the orderBy clause's config and the collection's config. If the orderBy clause specifies locale configuration, we use that one. Otherwise, we use the locale configuration from the collection.

Details

Indexes are sensitive to locale options (i.e. different locale options require different indexes).
We had to change all index creations and index lookups such that they keep into account the locale configuration.
- When optimising ordered queries, we need to build a locale configuration (take the locale config from the orderBy clause and if not provided then use the locale config from the collection) and pass that config when we create and/or lookup indexes
- Other places in the code were passing the default locale configuration when creating/looking up indexes. We had to change this to pass the collection's locale configuration instead

changeset-bot · 2025-11-05T10:22:58Z

🦋 Changeset detected

Latest commit: 464c74c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages

Name	Type
@tanstack/db	Patch
@tanstack/angular-db	Patch
@tanstack/electric-db-collection	Patch
@tanstack/offline-transactions	Patch
@tanstack/powersync-db-collection	Patch
@tanstack/query-db-collection	Patch
@tanstack/react-db	Patch
@tanstack/rxdb-db-collection	Patch
@tanstack/solid-db	Patch
@tanstack/svelte-db	Patch
@tanstack/trailbase-db-collection	Patch
@tanstack/vue-db	Patch
todos	Patch
@tanstack/db-example-paced-mutations-demo	Patch
@tanstack/db-example-react-todo	Patch
@tanstack/db-example-solid-todo	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

pkg-pr-new · 2025-11-05T10:24:32Z

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@762

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@762

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@762

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@762

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@762

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@762

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@762

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@762

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@762

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@762

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@762

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@762

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@762

commit: 464c74c

github-actions · 2025-11-05T10:26:04Z

Size Change: +346 B (+0.42%)

Total Size: 83.3 kB

Filename	Size	Change
`./packages/db/dist/esm/collection/change-events.js`	1.36 kB	+22 B (+1.64%)
`./packages/db/dist/esm/collection/index.js`	3.25 kB	+121 B (+3.87%)
`./packages/db/dist/esm/indexes/auto-index.js`	742 B	+11 B (+1.5%)
`./packages/db/dist/esm/query/builder/index.js`	3.84 kB	-15 B (-0.39%)
`./packages/db/dist/esm/query/compiler/order-by.js`	1.25 kB	+79 B (+6.75%)	🔍
`./packages/db/dist/esm/query/live/collection-config-builder.js`	5.11 kB	+108 B (+2.16%)
`./packages/db/dist/esm/utils/index-optimization.js`	1.51 kB	+20 B (+1.34%)

ℹ️ View Unchanged

Filename	Size
`./packages/db/dist/esm/collection/changes.js`	977 B
`./packages/db/dist/esm/collection/events.js`	388 B
`./packages/db/dist/esm/collection/indexes.js`	1.1 kB
`./packages/db/dist/esm/collection/lifecycle.js`	1.67 kB
`./packages/db/dist/esm/collection/mutations.js`	2.26 kB
`./packages/db/dist/esm/collection/state.js`	3.43 kB
`./packages/db/dist/esm/collection/subscription.js`	2.42 kB
`./packages/db/dist/esm/collection/sync.js`	2.02 kB
`./packages/db/dist/esm/deferred.js`	207 B
`./packages/db/dist/esm/errors.js`	3.9 kB
`./packages/db/dist/esm/event-emitter.js`	748 B
`./packages/db/dist/esm/index.js`	2.49 kB
`./packages/db/dist/esm/indexes/base-index.js`	766 B
`./packages/db/dist/esm/indexes/btree-index.js`	1.87 kB
`./packages/db/dist/esm/indexes/lazy-index.js`	1.1 kB
`./packages/db/dist/esm/indexes/reverse-index.js`	513 B
`./packages/db/dist/esm/local-only.js`	837 B
`./packages/db/dist/esm/local-storage.js`	2.04 kB
`./packages/db/dist/esm/optimistic-action.js`	359 B
`./packages/db/dist/esm/paced-mutations.js`	496 B
`./packages/db/dist/esm/proxy.js`	3.22 kB
`./packages/db/dist/esm/query/builder/functions.js`	606 B
`./packages/db/dist/esm/query/builder/ref-proxy.js`	917 B
`./packages/db/dist/esm/query/compiler/evaluators.js`	1.26 kB
`./packages/db/dist/esm/query/compiler/expressions.js`	674 B
`./packages/db/dist/esm/query/compiler/group-by.js`	1.78 kB
`./packages/db/dist/esm/query/compiler/index.js`	1.95 kB
`./packages/db/dist/esm/query/compiler/joins.js`	2 kB
`./packages/db/dist/esm/query/compiler/select.js`	1.07 kB
`./packages/db/dist/esm/query/ir.js`	673 B
`./packages/db/dist/esm/query/live-query-collection.js`	360 B
`./packages/db/dist/esm/query/live/collection-registry.js`	214 B
`./packages/db/dist/esm/query/live/collection-subscriber.js`	1.77 kB
`./packages/db/dist/esm/query/optimizer.js`	2.6 kB
`./packages/db/dist/esm/query/predicate-utils.js`	2.88 kB
`./packages/db/dist/esm/query/subset-dedupe.js`	921 B
`./packages/db/dist/esm/scheduler.js`	1.21 kB
`./packages/db/dist/esm/SortedMap.js`	1.18 kB
`./packages/db/dist/esm/strategies/debounceStrategy.js`	237 B
`./packages/db/dist/esm/strategies/queueStrategy.js`	418 B
`./packages/db/dist/esm/strategies/throttleStrategy.js`	236 B
`./packages/db/dist/esm/transactions.js`	2.9 kB
`./packages/db/dist/esm/utils.js`	881 B
`./packages/db/dist/esm/utils/browser-polyfills.js`	304 B
`./packages/db/dist/esm/utils/btree.js`	5.61 kB
`./packages/db/dist/esm/utils/comparison.js`	660 B
`./packages/db/dist/esm/utils/type-guards.js`	157 B

_{compressed-size-action::db-package-size}

github-actions · 2025-11-05T10:27:36Z

Size Change: 0 B

Total Size: 3.34 kB

ℹ️ View Unchanged

Filename	Size
`./packages/react-db/dist/esm/index.js`	225 B
`./packages/react-db/dist/esm/useLiveInfiniteQuery.js`	1.17 kB
`./packages/react-db/dist/esm/useLiveQuery.js`	1.11 kB
`./packages/react-db/dist/esm/useLiveSuspenseQuery.js`	431 B
`./packages/react-db/dist/esm/usePacedMutations.js`	401 B

_{compressed-size-action::react-db-package-size}

…lection-wide config and allow queries to override those.

samwillis · 2025-11-05T16:34:26Z

packages/db/src/types.ts

+  /**
+   * Specifies how to compare data in the collection.
+   * This should be configured to match data ordering on the backend.
+   * E.g., when using the Electric DB collection these options
+   *       should match the database's collation settings.
+   */
+  compareOptions?: StringSortOpts


@KyleAMathews this is a new collection config option, you may have some to say on naming.

It's a bit ambiguous at the collection level what is compared — defaultCollation or defaultStringCollation is more clear.

Ok, went for defaultStringCollation and renamed StringSortOpts type to StringCollationConfig (cc @KyleAMathews @samwillis)

samwillis · 2025-11-05T16:41:31Z

@kevin-dp as discussed we need to traverse the from in live queries to determine the compare options for the live query.
Also remember to pull as I rebased!

…fig and live query collection config

samwillis

#762) * Add optional compareOptions to the Collection Config which act as collection-wide config and allow queries to override those. * Fix import * changeset * Ran prettier * update docs * Live queries inherit compare options from collection in the FROM clause * Explicit compare options for live queries * Rename compareOptions to defaultStringCollation in the collection config and live query collection config * Docs update --------- Co-authored-by: Sam Willis <sam.willis@gmail.com>

kevin-dp requested a review from samwillis November 5, 2025 10:23

samwillis mentioned this pull request Nov 5, 2025

Query Driven Sync with On-Demand Collection Loading #763

Open

6 tasks

samwillis force-pushed the kevin/pred-pushdown-to-sync-electric-coll branch from e4a6188 to ffe08e9 Compare November 5, 2025 14:45

Base automatically changed from kevin/pred-pushdown-to-sync-electric-coll to query-driven-sync November 5, 2025 15:30

kevin-dp added 4 commits November 5, 2025 15:52

Add optional compareOptions to the Collection Config which act as col…

2d26219

…lection-wide config and allow queries to override those.

Fix import

4f517f4

changeset

e867665

Ran prettier

57bfe3d

samwillis force-pushed the kevin/locale-config branch from fa1e85e to 57bfe3d Compare November 5, 2025 15:53

update docs

133ca28

samwillis reviewed Nov 5, 2025

View reviewed changes

kevin-dp added 4 commits November 6, 2025 10:36

Live queries inherit compare options from collection in the FROM clause

151d917

Explicit compare options for live queries

bd44ca7

Rename compareOptions to defaultStringCollation in the collection con…

fe1623a

…fig and live query collection config

Docs update

464c74c

samwillis approved these changes Nov 6, 2025

View reviewed changes

kevin-dp merged commit 4e77602 into query-driven-sync Nov 6, 2025
7 checks passed

kevin-dp deleted the kevin/locale-config branch November 6, 2025 15:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add optional compareOptions to the Collection Config which act as col… #762

Add optional compareOptions to the Collection Config which act as col… #762

Uh oh!

kevin-dp commented Nov 5, 2025 •

edited

Loading

Uh oh!

changeset-bot bot commented Nov 5, 2025 •

edited

Loading

Uh oh!

pkg-pr-new bot commented Nov 5, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 5, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 5, 2025 •

edited

Loading

Uh oh!

samwillis Nov 5, 2025

Uh oh!

KyleAMathews Nov 5, 2025

Uh oh!

kevin-dp Nov 6, 2025

Uh oh!

samwillis commented Nov 5, 2025

Uh oh!

samwillis left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Add optional compareOptions to the Collection Config which act as col… #762

Add optional compareOptions to the Collection Config which act as col… #762

Uh oh!

Conversation

kevin-dp commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Implementation

High-level

Details

Uh oh!

changeset-bot bot commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

pkg-pr-new bot commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

samwillis Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

KyleAMathews Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

kevin-dp Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

samwillis commented Nov 5, 2025

Uh oh!

samwillis left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kevin-dp commented Nov 5, 2025 •

edited

Loading

changeset-bot bot commented Nov 5, 2025 •

edited

Loading

pkg-pr-new bot commented Nov 5, 2025 •

edited

Loading

github-actions bot commented Nov 5, 2025 •

edited

Loading

github-actions bot commented Nov 5, 2025 •

edited

Loading