Persistent state management

[esimkowitz]

Specific Demand

Dioxus should support saving state to either local or browser storage, depending on target, so it can be preserved both when an element falls out of scope and when an app is closed or refreshed.

Implement Suggestion

This could be built on top of the new dioxus-signals library. A new function use_signal_persisted could subscribe to changes in the Signal contents and write that back to the storage. To save on unnecessary writes, the window.beforeUnload event and equivalents in Tauri could be used to persist the data only before the page changes.

As for how to store the data, there's a package bevy_pkv that seems like a good start. It uses redb as the default database on local filesystems and LocalStorage on browser. Not sure how that would translate to mobile.

[ealmloff]

This would be a great addition to dioxus-std! Before the dioxus-std crate existed, I created the dioxus-storage library that allows you to create persistent values. We can move that over to dioxus std and make a signal and non-signal version of the hook

[esimkowitz]

Good idea! Looks like dioxus-storage currently just works for web targets, is that right? Happy to take that and add support for other targets. I'm also thinking we should distinguish between user settings (i.e. something like Swift UserDefaults) and persistent states.

I think for use_persistent, the developer should be able to specify whether the lifetime should be tied to the current session (i.e. until refresh/close) or in perpetuity (i.e. until browser cache is emptied). For web targets, this should be very easy as there are two buckets we can use in the Storage API (LocalStorage and SessionStorage). For desktop/terminal/mobile targets, this would be a bit trickier. Tauri seems to have its own storage plugin tauri-plugin-store that allows for simple KV store in memory, which can then be persisted to disk by running a save() function on the store instance. We could create two stores, one for persistent and one for session data, and only save the persistent one on close. Not sure how to handle things for Terminal though.

As for user settings, I think it may be best to expose those as a separate mechanism, whereby each component can register its own settings which can then be built into a static map. This would give apps a straightforward way to then generate a common settings page to manipulate those user settings. Those can then be persisted via the platform-specific interface or we could reuse the mechanisms from above.

[ealmloff]

Good idea! Looks like dioxus-storage currently just works for web targets, is that right? Happy to take that and add support for other targets. I'm also thinking we should distinguish between user settings (i.e. something like Swift UserDefaults) and persistent states.

It should work for desktop targets as well. It uses the directories crate to save the state as a file in the users data directory.

I think for use_persistent, the developer should be able to specify whether the lifetime should be tied to the current session (i.e. until refresh/close) or in perpetuity (i.e. until browser cache is emptied). For web targets, this should be very easy as there are two buckets we can use in the Storage API (LocalStorage and SessionStorage). For desktop/terminal/mobile targets, this would be a bit trickier. Tauri seems to have its own storage plugin tauri-plugin-store that allows for simple KV store in memory, which can then be persisted to disk by running a save() function on the store instance. We could create two stores, one for persistent and one for session data, and only save the persistent one on close. Not sure how to handle things for Terminal though.

Session storage (as opposed to a normal use state) will be persevered when you duplicate a tab reload a page or restore a page. I'm not sure what that would look like for a desktop application as there isn't any concept of a page reload or restore. Does a session storage state wrapper make sense in desktop renderers at all?

[esimkowitz]

Actually there may be a regression here between 0.3 and 0.4. I noticed that state info used to be preserved when my route would change but with the shift to 0.4, it no longer does. Any idea why that might be?

[ealmloff]

Actually there may be a regression here between 0.3 and 0.4. I noticed that state info used to be preserved when my route would change but with the shift to 0.4, it no longer does. Any idea why that might be?

If you change the route with a link from the router, it should preserve state in any layouts or outside of the router component in both versions.

However, if you manually type in a new URL in the browser, I would expect both versions to loose all state from the previous route.

Do you have a specific example of state that was preserved in 0.3 that is not preserved in 0.4?

[esimkowitz]

Do you have a specific example of state that was preserved in 0.3 that is not preserved in 0.4?

I use the use_shared_state here and when I navigate away from this page to one of the other ones in my app and then return, I notice that the state is reset. I haven't updated the code of this page since before 0.4.0, except to use the render! macro, which shouldn't affect the state management AFAICT.

You can repro here.

This isn't just limited to this page, though, they're all affected, regardless of if I use use_shared_state, use_ref, or use_state.

[ealmloff]

Do you have a specific example of state that was preserved in 0.3 that is not preserved in 0.4?

I use the use_shared_state here and when I navigate away from this page to one of the other ones in my app and then return, I notice that the state is reset. I haven't updated the code of this page since before 0.4.0, except to use the render! macro, which shouldn't affect the state management AFAICT.

You can repro here.

This isn't just limited to this page, though, they're all affected, regardless of if I use use_shared_state, use_ref, or use_state.

I cloned the repo, reverted to dioxus 0.3 and I can reproduce this. Going back and forth inside of the app loses state but going to another page and going pack keeps the state. Very odd... But it also seems to work on the published version of the site which seems like it is 0.4

[ealmloff]

A hello world dioxus 0.4 application also seems to save state when you go pack to a page you left. I wonder if chrome clears previous page state if it uses too much memory making this randomly not work in 0.4 for you but work in 0.3

[esimkowitz]

It's also not working in my desktop app or Safari. Could it be because I use nested Route enums? Does that somehow trigger the states to get pruned when I move around?

[ealmloff]

It's also not working in my desktop app or Safari. Could it be because I use nested Route enums? Does that somehow trigger the states to get pruned when I move around?

It depends where you create the state. If you create the state in a route and that route is switched, the state will be cleared. If you create the state above the router, the state should stay when you switch routes inside of your application. This is true for both the old and new router

[esimkowitz]

I think I figured out what is happening: before adopting the new router, I was trying to keep the route definitions isolated to one module, so I used a static PHF to store pointers to the functions for each page. I had a for loop to iterate through the PHF and define the routes and I passed the function for rendering each of the pages as a parameter inside the route. In the for loop that declared the routes, I then called these functions, passing them the root context. That meant that when the route went out of scope, the page was still registered to the root context, so its state wasn't getting dropped.

With the new router, I just use the Router enum itself to define the structure instead of a PHF and since it automatically links to the functions for rendering each page, I got rid of the for loop and the complicated system of calling the functions in the root App. That meant that now I am properly using the Router but it also means that the states that were previously tied to the root App context are now tied to each route, so they get dropped when the route changes.

[esimkowitz]

Now that we can rule out regression, let's get back to the design for persistent state management.

Session storage (as opposed to a normal use state) will be persevered when you duplicate a tab reload a page or restore a page. I'm not sure what that would look like for a desktop application as there isn't any concept of a page reload or restore. Does a session storage state wrapper make sense in desktop renderers at all?

I think it still makes sense to be able to persist state if routes change on desktop/mobile but I agree it doesn't need a persistent backing so an in-memory store should be sufficient. I think all targets would benefit from a storage-backed User Settings container though. And it would be even cooler if changing those common user settings in one tab/window could trigger a re-render in all other windows. I am going to break that out as a separate issue though.

[esimkowitz]

I'm going to start on this using an in-memory HashMap to establish use_cached_state and use_cached_shared_state_provider. I am thinking that cache persistence to localStorage can be hidden behind a feature as not all devs may want this.

[esimkowitz]

I've done some digging and I think this can be accomplished as a drop-in replacement for the existing functions. One limitation I'm running into is that code in the dioxus-std crate does not have access to the UseState or ProvidedStateInner structs from the dioxus-hooks crate. @ealmloff what do you think about exposing these structs as public?

[ealmloff]

I've done some digging and I think this can be accomplished as a drop-in replacement for the existing functions. One limitation I'm running into is that code in the dioxus-std crate does not have access to the UseState or ProvidedStateInner structs from the dioxus-hooks crate. @ealmloff what do you think about exposing these structs as public?

UseState is public: https://docs.rs/dioxus-hooks/latest/dioxus_hooks/struct.UseState.html

We can make some version of it public. It might be better to decouple it from use_shared_state. For use_shared_state, you shouldn't really need the struct as a user, so I imagine it will be confusing to tie it to the hook. Making it public would just make it easier to implement your own version of use_shared_state. We could expose it under some other module with a different name (SubscriberState or something less related to use_shared_state). Pulling out the subscription system out of use_shared_state, use_state, and use_signal would make implementing custom wrappers much easier

[esimkowitz]

Actually after playing with it some more, I don't think I need to update anything about the underlying structs. Since all I really care about is capturing when the state is initialized and when it updates, I can just wrap the hook for the initialization and then outside of use_hook, I can read the value and store it in the HashMap. That should mark it as a subscriber so next time the state changes, the component will reload and the new value will be read and stored. Am I understanding this correctly?

Also what is the future for use_state and use_shared_state? They seem superseded by use_signal, since you can specify the owning scope directly for the signal.

I have a draft here, though it's currently failing with a confusing "in a virtual dom" error from Signal::new(), what is the meaning of this error?

[ealmloff]

Actually after playing with it some more, I don't think I need to update anything about the underlying structs. Since all I really care about is capturing when the state is initialized and when it updates, I can just wrap the hook for the initialization and then outside of use_hook, I can read the value and store it in the HashMap. That should mark it as a subscriber so next time the state changes, the component will reload and the new value will be read and stored. Am I understanding this correctly?

That sounds like it should work

Also what is the future for use_state and use_shared_state? They seem superseded by use_signal, since you can specify the owning scope directly for the signal.

Depending on community feedback, use_state and use_shared_state may be depreciated in the future.

I have a draft here, though it's currently failing with a confusing "in a virtual dom" error from Signal::new(), what is the meaning of this error?

You need to create signals within a virtual dom (and inside a scope) so that dioxus knows when to destroy the signal and has somewhere to store the signal in. If you try to do something like this:

Signal::new(0)

There is no virtual dom to store the signal in, so it panics

[esimkowitz]

There is no virtual dom to store the signal in, so it panics

How can I ensure it is running in a virtual dom? This is failing when running the small example I wrote to test this. Do you see anything weird in this example that could be causing it to not run in a virtual DOM?

[esimkowitz]

Never mind, it was occurring because I was using a git rev for the dioxus-signals crate while using the 0.4.0 version of the main dioxus crate for everything else. Matching them all to the git rev worked.

[esimkowitz]

I didn't realize that ScopeId is non-deterministic and therefore won't work as a good key for indexing the cached states. I see there's a name field in the Scoped struct, but that seems to just be the name of the function hosting the current scope and is not globally unique. Are there any deterministic, globally unique fields associated with a scope that could be used for a key? I suppose I could traverse the VirtualDOM tree to get the full path, but that seems expensive.

[ealmloff]

I didn't realize that ScopeId is non-deterministic and therefore won't work as a good key for indexing the cached states. I see there's a name field in the Scoped struct, but that seems to just be the name of the function hosting the current scope and is not globally unique. Are there any deterministic, globally unique fields associated with a scope that could be used for a key? I suppose I could traverse the VirtualDOM tree to get the full path, but that seems expensive.

If you run the exact same components with the same state every time, ScopeId should be deterministic.

If you want to store state only in components that always exist in the first render called in a determinist order, you can use a stack based approach similar to hooks. This is the approach we use for server to client storage. If the value does not exist, you can push it to a global array (this will happen during the first render). If values do exist, take one value and move the cursor forward.

[esimkowitz]

Closing as completed with v0.5 :)