Deduplicate queries by `id`

[evanob]

One thing I love about TanStack DB is that it has such parallels to TanStack Query that it's simple to swap from Query to DB, making it easy to adopt incrementally. One significant challenge though is that while Query deduplicates queries by queryKey, DB does not. That means we can't just swap our useVendorsQuery for a useLiveVendorsQuery; we need to be careful that useLiveVendorsQuery isn't called 20 times on the same page (e.g. in a table view), or else the query will run 20 times.

TanStack Query solves this problem with a QueryCache and a QueryObserver for every call to the same key. It uses an array for the key because they need to think about query invalidation, but a simple string would probably be fine here because we don't need to worry about invalidation.

We can work around this by hoisting the query higher up the component tree and share the results explicitly, but it makes adoption a little less simple.

[KyleAMathews]

The Query Collection takes all the same options — are you not passing a query key to your query collections?

What does your useLiveVendorsQuery look like?

[evanob]

We're not using QueryCollection – for the query stuff, we're just using plain useQuery, abstracted into useVendorsQuery. The cache works as expected with Query. The problem for us is that there is no shared cache for DB queries. (The other problem is that we are overly reliant on Query's deduplication, but I think it's a very common pattern too, which is why I thought it would be worthwhile to create an issue even though we plan to change our own data fetching patterns.)

---

Our useVendorsQuery is super simple, as shown below. useCollection in this case pulls the collection from a React Context to ensure stability within a given workspace.

export const useLiveVendors = () => {
  const vendorsCollection = useCollection('vendors')
  return useLiveQuery((q) =>
    q.from({ vendor: vendorsCollection }).orderBy(({ vendor }) => lower(vendor.name)),
  )
}

export const useCollection = <K extends keyof SyncContextValue>(key: K): SyncContextValue[K] => {
  const collection = useContext(SyncContext)?.[key]
  assert(collection, `useCollection must be used within a WorkspaceSyncProvider: ${key}`)
  return collection
}

type SyncContextValue = {
  vendors: ReturnType<typeof createVendorsCollection>
}

const SyncContext = createContext<SyncContextValue | null>(null)

// we place this as high as we can in the component tree
const WorkspaceSyncProvider: FC<PropsWithChildren<{ workspaceId: string }>> = ({
  children,
  workspaceId,
}) => {
  const vendors = createVendorsCollection(workspaceId)

  useEffect(() => {
    vendors.preload()

    return () => {
      vendors.cleanup()
    }
  }, [vendors])

  return (
    <SyncContext.Provider
      value={{
        vendors,
      }}
    >
      {children}
    </SyncContext.Provider>
  )
}

export default WorkspaceSyncProvider

export const createVendorsCollection = (workspaceId: string) =>
  createElectricCollection<Vendor>('vendors', `/sync/${workspaceId}/vendors`, {
    getKey: (item) => item.id,
  })

export const createElectricCollection = <T extends Row>(
  id: string,
  path: `/${string}`,
  options: {
    getKey: (item: T) => string | number
    shapeStream?: Omit<ShapeStreamOptions<GetExtensions<T>>, 'url'>
    collection?: Omit<ElectricCollectionConfig<T>, 'id' | 'shapeOptions' | 'getKey'>
  },
) => {
  return createCollection<T>(
    electricCollectionOptions({
      id,
      shapeOptions: getShapeStreamOptions(path, options.shapeStream),
      getKey: options.getKey,
      startSync: false,
      ...options.collection,
    }),
  )
}

const getShapeStreamOptions = <T extends Row>(
  path: `/${string}`,
  options?: Omit<ShapeStreamOptions<GetExtensions<T>>, 'url'>,
): ShapeStreamOptions<GetExtensions<T>> => {
  return {
    url: `${API_BASE_URL}${path}`,
    headers: {
      Authorization: async () => {
        const authToken = await getAccessToken()
        return `Bearer ${authToken}`
      },
    },
    ...options,
  }
}

[KyleAMathews]

Live queries are going to be mutable soon so they'll be less instant sharable in the future.

Also live queries are much less costly than a useQuery as there's no network calls or (much) state that's created.

What we recommend for sharing queries is either export named live queries from e.g. a queries.ts file or create factory functions e.g. #652 (comment)