Is there a way to persist the cache in the browser?

[frederikhors]

I just found out this amazing project! Congratulations!

In my dreams I would like to replace @tanstack/query and urql entirely with nanostores/query. It would be a crazy joy! 😄

The only thing that blocks me now is that I need to persist all the query data (the cache) in the browser (like I do with those libraries).

I was wondering if there is a way to persist the cahce using IndexedDB or something else (I prefer not to use localStorage which besides its already well known limitations also has the problem that it is sync and blocks the main thread).

@ai suggested me here to use https://developer.mozilla.org/en-US/docs/Web/API/Cache.

I need this to be able to restore data when there is no or intermittent internet (I'm already using PWA).

What do you think?

Once again: thank you for these magical projects!

[dkzlv]

@frederikhors Hi there!

Thanks for your kind words here 😄

In my dreams I would like to replace @tanstack/query and urql entirely with nanostores/query.

We do operate on the same level of abstraction as @tanstack/query and urql. Just to be safe I'll mention that you'll still need something to work out the specifics of GraphQL queries/mutations if you plan to use them. To my knowledge, the slimmest feature-full implementation of GQL is graphql-request (4x the size of nanoquery, 6.1Kb, jeez!).

I was wondering if there is a way to persist the cahce using IndexedDB or something else (I prefer not to use localStorage which besides its already well known limitations also has the problem that it is sync and blocks the main thread).

Currently—no. But that's something I long wanted myself, actually. Besides, it's somehow related to SSR (that's inner implementation details), which we'll need to add once Contexts are merged, so why not now?

Give me a few days (maybe a week?) to poke at this bear.

[frederikhors]

Your answer fills me with shivers of joy! 😄

I can help you test even the smallest quibble, and I can help you eventually with any concerns because I've used pretty much every library in existence today to do this.

I will be able to test with either Svelte (and SvelteKit) or SolidJS (and SolidStart).

Regarding GraphQL: we are planning to remove graphql entirely from our stack and start using grpc-web with tools like connect-es and others.

Therefore we are looking for an "agnostic" store that stores and persists data in the browser asynchronously and fastest!

tanstack/query is perhaps the closest to what we are looking for but it doesn't have a persistent in-browser store plugin for Svelte and SolidJS: only for React and I should write it myself, but I feel myself not very good for such a task.

Offline management is also premature for now.

Regarding SSR: I think investing so much time in SSR NOW is premature.

I mean that most of our applications are behind login and not publicly accessible because SAAS, CRM and similar WITHOUT SSR.

And so far for the few public sites we have we haven't had the need for persistence in SSR (since they then could load from the browser once hydrated).

Therefore for us SSR is not a priority.

I repeat, I'm here to help. Anything I can do.

Long live the bear!

And thank you in advance!

[dkzlv]

As an update, this takes a lot more time than I anticipated, because after a brief internal discussion we decided that it would be better to implement it in a good way and thorough instead of a poor ad hoc kind of way.

FYI, here's a very simple demo of how you can achieve some level of persistence, but it's bad.

The problem of persisting changes is essentially the same as SSR and tests: you need to be able to instantiate a store with some pre-defined value and treat it as a fresh piece of data. So I could theoretically come up with an ad hoc solution to this, but instead we decided to implement a nanostores-wide solution. Below is a quote from this message from #17:

I'll share our vision for testing in near future.

We have a WIP for a new entity in the core of nanostores called Context. Context is essentially a place where all state for all atoms is stored. Once we land it in the main nanostores repo (I suspect it'll happen in August), I'll adapt nanoquery to this concept, and it'll simplify things a lot.

Using this new concept you'll be able to explicitly set a value for any atom by its reference for a certain isolated context (e.g., a single test unit!). I'll probably move fetcher functions inside atoms, which will give you a granular tool to change a fetcher function for a single fetcher/mutator store for a single test. That should probably be the better way that won't be nanoquery specific at all and will be the new way to test everything nanostores-related.

So, to summarize, this issue will need to wait for the PR to land in the core (I expect it to happen in August), and then it'll be a breeze to implement persisting in query.

(I don't expect you'll be interested in it by then, @frederikhors, but hopefully some other folk will)

[frederikhors]

Yeah. I'll patiently wait your amazing work. Thanks!

[frederikhors]

@dkzlv thanks for working on this project!

After #36 I do not understand how to persist in IndexedDB. Is there any docs/tutorial/guide?

[dkzlv]

@frederikhors Sure! I updated the snippet I shared here before.

To persist something, we basically need to do 3 things:

1. inherit Map and overwrite .set method so that it caches data somewhere. In my example I call it Cache use idb-keyval that simplifies overly complex interface of IndexedDB down to a simple KV-storage.
2. provide this custom Cache to nanoquery function. Under the hood it defaults to new Map(), but it's allowed to change this.
3. retrieve initial cache values and set them (we can call it hydration just for the laugh of it). Since IDB is async in nature, I also added a separate $appReady store, but if you plan to use localStorage, this can be dropped and process can be executed synchronously.

Here's a demo of the result:

Screen.Recording.2024-04-16.at.16.13.49.mov

Notice how it works:

• value is cached along with its created and expires timestamps;
• upon "hydration" nanoquery recognises cached values as not stale, and uses them without calling fetcher function;
• but if created + dedupeTime has passed, cache is in fact stale. Stale cache is shown but invalidated instantly. New data silently swaps the stale cache.

That all is a bit barebones, I understand. My next priority is to introduce some helpers for both offline caching and SSR, so things will be a bit simpler in future.

[frederikhors]

Thank you very much, we are considering a switch from tanstack/query to truly offline-first projects perhaps with sqlite in wasm.

We need to figure out if it's worth moving from the current tanstack/query to nanostores/query.

We will evaluate and if necessary you will hear from me.

Great project nanostores by the way. ❤️