Queries persisted by experimental_createQueryPersister should be invalidated/removed as well

[odelmarcelle]

I've implemented the experimental persister in my app with the aim or re-use loading data when opening new tabs. Because there's sometimes a lot of data in the cache, the QueryClient persister turned out to be quite slow at startup. With the experimental persister, the performance is much better and thus appears to be the way to go.

However, I noticed that the persisted queries are not affected by invalidateQueries (if the user reloads the page) and removeQueries (in all cases). If nothing has subscribed to the query, it is then easily possible to enter a situation where an outdated query is returned from the persisted storage despite invalidation/removal.

I believe that the invalidation behavior (setting isInvalidated) and removal behavior (removing the query from the cache) should also apply to the persisted storage.

I prepared the following CodeSandbox to highlight the issue: https://codesandbox.io/p/devbox/compassionate-james-yvg5j9

The fetched data is immediately persisted to IndexedDB. The "invalidate/removal" attempts to erase the query from the cache. However, the next click on "fetch" will return the data from the persisted cache. This is easily observed by looking at the value of fetchedAt.

[TkDodo]

This is largely on purpose because we want to decouple what is in the in-memory cache from what is in the external storage. This + “lazy loading” was the main motivation behind this feature.

Especially for environments like react-native, to limit the amount of memory used, the persister can be used to set a low gcTime for memory efficiency, so that data will be deleted from memory when it becomes unused, but keep it in the external storage for longer (e.g. 24 hours), so that it loads from there the next time.

We have exposed utils to clean-up the external storage for yourself lately:

• https://tanstack.com/query/latest/docs/framework/react/plugins/createPersister#persistergc-promisevoid

We might want to add periodic cleanup for clearing expired, busted or malformed entries, but this must be done by a framework-specific provider because it likely needs to tie in to the component lifecycle, similar to how we do it with the <PersistQueryClientProvider>. In React, this would likely just be a useEffect with an interval (@DamianOsipiuk FYI, what do you think about this?).

This however is not the same as clearing something immediately when it becomes unused in memory, which is what removeQueries does.