-
Notifications
You must be signed in to change notification settings - Fork 121
Description
- I've validated the bug against the latest version of DB packages
Describe the bug
When the result of a live query collection is stale, it is still being used to fulfill other live queries with the stale data.
In my testing, I query for users using their id property (which is also the item key), then run multiple live queries with various id conditions. I noticed that if I have overlapping id predicates in them, the later live queries return the stale user entries from previous stale queries and prune the affected user entity id predicates from the currently running queryFn.
To Reproduce
- run query 1 asking for a specific user by
id
const { data, isLoading } = useLiveQuery((q) =>
q
.from({ user: userCollection })
.where(({ user }) => eq(user.id, `123`))
.findOne()
);Result: the REST API is successfully queried for the user id 123
-
wait for the query stale time to hit (e.g. for testing, specify
3_000as the default staleTime for the collection -
mount query 2 asking for other users by
id
const { data, isLoading } = useLiveQuery((q) =>
q
.from({ user: userCollection })
.where(({ user }) => inArray(user.id, [`123`, `456`, `789`]))
);Result: the REST API is only queried for user ids 456 and 789, the respective queryFn of the userCollection only ever gets a filter for these 2 ids (the id 123 is not included in the parsed result set). The live query, however, returns the 3 respective users (including id 123), with the 2 new ids being fresh from the REST API and the previously queried id being from the cache.
This is the collection at hand:
export const userCollection = createCollection(
queryCollectionOptions<DbSyncMetaRecord<UserSchema>>({
id: 'users',
queryClient,
getKey: (record) => record.id,
syncMode: 'on-demand',
startSync: false,
queryKey: (opts) => ['users', JSON.stringify(opts || {})],
queryFn: async (ctx) => {
// get existing data or empty array
const existingData = (queryClient.getQueryData(ctx.queryKey) || []) as DbSyncMetaRecord<UserSchema>[];
if (!ctx.meta?.loadSubsetOptions) return existingData; // return existing data or empty array
const parsed = parseLoadSubsetOptions(ctx.meta?.loadSubsetOptions);
const ids: string[] = [];
// the following portion is me trying to debug where the missing id went
for (let i = 0; i < parsed?.filters?.length; i++) {
const { field, value } = parsed.filters[i];
const fieldName = field.join('.');
if (fieldName === 'id') {
const __ids = Array.isArray(value) ? value : [value]; // quick gotcha, filtering for arrays returns a value of type `string[]` and not just a single `string` with multiple `id` filter entries
for (let x = 0; x < __ids.length; x++) {
if (ids.indexOf(__ids[x]) === -1) ids.push(__ids[x]); //quick and dirty dedupe
}
}
}
if (ids.length > 0) { // only query API if we have at least 1 id to look up
const r = await fetch(`/api/users`, { method: 'POST', body: JSON.stringify({ ids }) });
if (!r.ok) return existingData;
const res = (await r.json()) || { users: [] };
if (res?.users?.length > 0) {
const existingMap = new Map(existingData.map((item) => [item.id, item]));
for (let x = 0; x < res.users.length; x++) { // fill in new data in the existing data map
existingMap.set(res.users[x].id, { id: res.users[x].id, record: res.users[x] });
}
return Array.from(existingMap.values()) || []; // return all entries with newly added ones
}
return existingData; // fallback to existingData
}
return existingData;
}
})
);I have to merge the API result into existingData and return that in the queryFn, cause returning only the API result will prune the id 123 (due to it not being part of the predicate in query 2 of step 3).
Expected behavior
The second query (step 3) should also include id 123 in the parsed queryFn filter setup, seeing as the user with id 123 is only included in a stale live query result / cache. Expected behavior is that the collection detects id 123 requiring fresh data from the queryFn / REST API.
Ideally the collection would keep track of the last "update time" an individual item had. That way it can determine whether an item being queried for needs their data to be refreshed via the queryFn / in the background (return stale data from cache and pull fresh data from the API).
Screenshots
Desktop (please complete the following information):
- OS: macOS 26.1
- Browser: Edge, Safari
- Version: latest
Additional info
As seen on discord: https://canary.discord.com/channels/719702312431386674/1369767025723052134/1439026581623275630