Skip to content

Conversation

@samwillis
Copy link
Collaborator

Summary

  • Replace hash-based object ID tie-breaking with direct key comparison for deterministic ordering when ORDER BY values are equal
  • Simplify internal data structures by removing the TaggedValue abstraction
  • Remove globalObjectIdGenerator dependency from the topK operators

Problem

When multiple rows have equal ORDER BY values, the previous implementation used globalObjectIdGenerator.getId(key) to generate a numeric tag for tie-breaking. This had several issues:

  1. Hash collisions: The simple hash function could produce the same value for different keys
  2. Unstable for objects: Object references get auto-incrementing IDs, which aren't stable across page reloads
  3. Unnecessary complexity: The TaggedValue<K, V> = [K, V, Tag] 3-tuple added overhead

Solution

Use the row key directly for tie-breaking:

// Before (hash-based, potentially unstable):
const tieBreakerA = getTag(a)  // globalObjectIdGenerator.getId(key)
const tieBreakerB = getTag(b)
return tieBreakerA - tieBreakerB

// After (stable, deterministic):
if (aKey === bKey) return 0
if (aKey < bKey) return -1
return 1

This works because:

  • Row keys are always string | number (from collection's getKey)
  • Each key appears at most once in the topK (enforced by multiplicity tracking)
  • Direct comparison is deterministic and stable across sessions

Changes

  • topKWithFractionalIndex.ts:
    • Replace TaggedValue<K, V> with [K, T] tuple
    • Use direct key comparison for tie-breaking
    • Remove globalObjectIdGenerator import
    • Remove tagValue, getKey, getVal, getTag helpers
  • topKWithFractionalIndexBTree.ts: Update type signatures to use [K, T]

Test plan

  • All existing db-ivm tests pass (267 tests)
  • All existing db package tests pass (1718 tests)
  • ORDER BY tests specifically verified

@changeset-bot
Copy link

changeset-bot bot commented Dec 3, 2025

🦋 Changeset detected

Latest commit: 97bd6c9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 14 packages
Name Type
@tanstack/db-ivm Patch
@tanstack/db Patch
@tanstack/angular-db Patch
@tanstack/db-collection-e2e Patch
@tanstack/electric-db-collection Patch
@tanstack/offline-transactions Patch
@tanstack/powersync-db-collection Patch
@tanstack/query-db-collection Patch
@tanstack/react-db Patch
@tanstack/rxdb-db-collection Patch
@tanstack/solid-db Patch
@tanstack/svelte-db Patch
@tanstack/trailbase-db-collection Patch
@tanstack/vue-db Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new
Copy link

pkg-pr-new bot commented Dec 3, 2025

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@957

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@957

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@957

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@957

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@957

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@957

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@957

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@957

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@957

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@957

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@957

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@957

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@957

commit: 97bd6c9

@github-actions
Copy link
Contributor

github-actions bot commented Dec 3, 2025

Size Change: 0 B

Total Size: 87.2 kB

ℹ️ View Unchanged
Filename Size
./packages/db/dist/esm/collection/change-events.js 1.39 kB
./packages/db/dist/esm/collection/changes.js 977 B
./packages/db/dist/esm/collection/events.js 388 B
./packages/db/dist/esm/collection/index.js 3.24 kB
./packages/db/dist/esm/collection/indexes.js 1.1 kB
./packages/db/dist/esm/collection/lifecycle.js 1.67 kB
./packages/db/dist/esm/collection/mutations.js 2.31 kB
./packages/db/dist/esm/collection/state.js 3.43 kB
./packages/db/dist/esm/collection/subscription.js 2.55 kB
./packages/db/dist/esm/collection/sync.js 2.37 kB
./packages/db/dist/esm/deferred.js 207 B
./packages/db/dist/esm/errors.js 4.19 kB
./packages/db/dist/esm/event-emitter.js 748 B
./packages/db/dist/esm/index.js 2.66 kB
./packages/db/dist/esm/indexes/auto-index.js 742 B
./packages/db/dist/esm/indexes/base-index.js 766 B
./packages/db/dist/esm/indexes/btree-index.js 1.87 kB
./packages/db/dist/esm/indexes/lazy-index.js 1.1 kB
./packages/db/dist/esm/indexes/reverse-index.js 513 B
./packages/db/dist/esm/local-only.js 837 B
./packages/db/dist/esm/local-storage.js 2.1 kB
./packages/db/dist/esm/optimistic-action.js 359 B
./packages/db/dist/esm/paced-mutations.js 496 B
./packages/db/dist/esm/proxy.js 3.75 kB
./packages/db/dist/esm/query/builder/functions.js 733 B
./packages/db/dist/esm/query/builder/index.js 3.96 kB
./packages/db/dist/esm/query/builder/ref-proxy.js 917 B
./packages/db/dist/esm/query/compiler/evaluators.js 1.35 kB
./packages/db/dist/esm/query/compiler/expressions.js 430 B
./packages/db/dist/esm/query/compiler/group-by.js 1.8 kB
./packages/db/dist/esm/query/compiler/index.js 1.96 kB
./packages/db/dist/esm/query/compiler/joins.js 2 kB
./packages/db/dist/esm/query/compiler/order-by.js 1.25 kB
./packages/db/dist/esm/query/compiler/select.js 1.07 kB
./packages/db/dist/esm/query/expression-helpers.js 1.43 kB
./packages/db/dist/esm/query/ir.js 673 B
./packages/db/dist/esm/query/live-query-collection.js 360 B
./packages/db/dist/esm/query/live/collection-config-builder.js 5.33 kB
./packages/db/dist/esm/query/live/collection-registry.js 264 B
./packages/db/dist/esm/query/live/collection-subscriber.js 1.74 kB
./packages/db/dist/esm/query/live/internal.js 130 B
./packages/db/dist/esm/query/optimizer.js 2.56 kB
./packages/db/dist/esm/query/predicate-utils.js 2.91 kB
./packages/db/dist/esm/query/subset-dedupe.js 921 B
./packages/db/dist/esm/scheduler.js 1.3 kB
./packages/db/dist/esm/SortedMap.js 1.18 kB
./packages/db/dist/esm/strategies/debounceStrategy.js 247 B
./packages/db/dist/esm/strategies/queueStrategy.js 428 B
./packages/db/dist/esm/strategies/throttleStrategy.js 246 B
./packages/db/dist/esm/transactions.js 2.9 kB
./packages/db/dist/esm/utils.js 881 B
./packages/db/dist/esm/utils/browser-polyfills.js 304 B
./packages/db/dist/esm/utils/btree.js 5.61 kB
./packages/db/dist/esm/utils/comparison.js 852 B
./packages/db/dist/esm/utils/index-optimization.js 1.51 kB
./packages/db/dist/esm/utils/type-guards.js 157 B

compressed-size-action::db-package-size

@github-actions
Copy link
Contributor

github-actions bot commented Dec 3, 2025

Size Change: 0 B

Total Size: 3.35 kB

ℹ️ View Unchanged
Filename Size
./packages/react-db/dist/esm/index.js 225 B
./packages/react-db/dist/esm/useLiveInfiniteQuery.js 1.17 kB
./packages/react-db/dist/esm/useLiveQuery.js 1.12 kB
./packages/react-db/dist/esm/useLiveSuspenseQuery.js 431 B
./packages/react-db/dist/esm/usePacedMutations.js 401 B

compressed-size-action::react-db-package-size

Copy link
Contributor

@kevin-dp kevin-dp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Way better to tie break on the key rather than a hash.
Make sure to fix the problem with the current tie-breaker though such that it correctly compares keys of different types (strings vs numbers).

@samwillis samwillis force-pushed the fix/stable-order-by-tiebreaker branch from 48853db to f745b26 Compare December 5, 2025 14:05
Replace hash-based object ID tie-breaking with direct key comparison
for deterministic ordering when ORDER BY values are equal.

- Use row key directly as tie-breaker (always string | number, unique per row)
- Remove globalObjectIdGenerator dependency
- Simplify TaggedValue from [K, V, Tag] to [K, T] tuple
- Clean up helper functions (tagValue, getKey, getVal, getTag)

This ensures stable, deterministic ordering across page reloads and
eliminates potential hash collisions.
@samwillis samwillis force-pushed the fix/stable-order-by-tiebreaker branch from 1047564 to 4c5cda6 Compare December 5, 2025 14:13
@samwillis samwillis merged commit 52c29fa into main Dec 5, 2025
8 checks passed
@samwillis samwillis deleted the fix/stable-order-by-tiebreaker branch December 5, 2025 14:24
@github-actions github-actions bot mentioned this pull request Dec 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants