[discussion] how can we improve state management? #252
Comments
|
I don't know. Maybe this is an interesting link, too: https://circleci.com/blog/why-we-use-om-and-why-were-excited-for-om-next/ It is about Om Next and why it uses graphs for data management. |
|
Really hard questions. Slightly feel a tendency to reselect over an orm-y solution. Maintaining nested data often becomes error prone and managing it other than deriving from a store feels contra redux and I'd fear losing out of advantages. I feel sceptical in wrapping data in objects in general. I'd rather but it in data structure if need be e.g. a graph. Is there maybe any way to create views towards the single data structure in a way like lenses (http://ramdajs.com/docs/#lens)? |
|
I'm no longer using choo for commonform.org directly, but I've retained a good bit of its structure, including namespaces, the reducer-action dichotomy, and Namespacing actions feels very worthwhile. Then again, I suppose that could be accomplished with a naming convention alone, without any built-in, object-based dispatch mechanism. An EventEmitter can handle prefixes fine. As for namespaces as a data segregation method, it's been tough. My application has a few "view modes" corresponding to entirely different ways the page can look at any given time: There's "browsing" views for listing content, "editor" views for viewing and changing content, and so on. I've ended up doing terrible things, like firing off reductions from the router to clear out state for views that are now hidden, or getting around the data barriers by encoding information in URLs that hit the router. Then there's the inevitable problem of wanting to write an action handler that needs state from multiple namespaces. The effect of that structural limitation is to glom lots of state together under ever-larger namespaces. That's in direct tension with modularizing (meaning, in choo, namespacing) actions and reducers. It's the old which-code-goes-with-which-data OOP problem again. At first blush, the idea of "importing" relations or graph structures, wholesale, kind of turns my stomach. Either would bring with it a whole mound of concepts dwarfing all the other fundamental primitives in choo combined. "If you wish to make a choo app from scratch, you must first understand relational theory?" Then again, it's clear the point of choo is to offer something more than just a more modular view concept, like React, with room to plug and play your choice of data management solution. I'll put some more though on it. My gut tells me it would still be awesome if choo took up even a little bit more of the state-management problem, even if it still left substantial room to plug in roll one's own approach, at a more-than-TodoMVC level of complexity. |
|
I recommend to pay attention to Datalog systems - Datomic/DataScript. D Datalog is more flexible, dynamic and powerful than SQL(RDBMS), but more complicated. |
|
I like the ideas of namespaces in Choo. I always try to modularize each "feature" in the app so it does not know anything at all about other "features" in the app. And preferably not about any library either. Kind of what is mentioned in this old talk. However I also always end up with the problem that some data is needed by multiple features. In react-redux-saga apps I solve this in the sagas as they have access to the full state. But that couples the features together through state which is not good. I mainly have two kinds of data, UI-state data (transient), and database data (data that came from the server where it is persisted in a database). Sometimes UI-state data need to shared, like when a currently selected product has information shown in two features. I can have the currently selected product id at one place in the state and both features render from that so they are in sync. There is also need to share database data, for example data such as a product name that need to be shown in two different features. One idea would be to get looser coupling by having data-duplication but not code duplication by way of sharing reducer logic. For example, if a return from a server call causes an action like PRODUCT_DATA_LOADED, we could put this data in one place in the state. But each feature has it's own sub-part of state which it is only allowed to read so it will not be able to get the product data. So instead we can have every feature store this same data in it's sub-state. So this means every feature must have reducer logic to do this. But that reducer logic code can be shared. So the code would be shared but the data would be duplicated in the state (as it appears in multiple sub-states). Same can be done if an action such as SELECTED_PRODUCT_ID_CHAGE comes. Every feature could store the new id in it's own sub-state. I think the advantage of this would be that a single feature is not dependent on state from any other feature and therefore can easily be re-used. But I have not tried this and do not know if it would work out in a larger project. Maybe someone else have tried it? |
|
It should be easy enough to allow choo apps to be nested. What I mean is that multiple parts of an application can usually function entirely by themselves, only requiring few "external" objects (like a WebSocket) that could be passed down in a function call. In one of the biggest apps I wrote, we came up with a component model convention that worked like this: exports.component = function (options) {
// Initialize stuff here
return {
html: some_template_thingy,
build: ($target) => {
// Bind component to $target
}
};
});What I like about this approach is that it's a convention, it's not bound to any framework or a library. Also, one can launch and test each component in an application separately. We usually had an I imagine this can be reduced down for choo to something like this: exports.component = function (options) {
const comp = choo();
// ...
return comp;
};The question is, how would I "mount" components together? I would like to be able to somehow achieve this: const other_thingy = require('other-thingy');
exports.component = function (options) {
const other = other_thingy.component();
return choo({
thingy: other // allow ${component.thingy} to be used in the component html
});
};With a technique like this, I would prefer keeping choo's current flexibility to let a developer choose whether they want namespaces in their models or not. Some small components might not need namespaces, while larger ones can still have the separation. What do you think? |
|
@mantoni I'm not sure I follow, but wouldn't requiring one thingy within another thingy couple them together? I think it would be nicer if thingys/components/modules/features/parts (need to find a good name for a stand-alone application part) only were coupled by actions, so that one thingy only would expect some actions to be passed to it, and produce some other actions. So the only way for a thingy to see the outside world would be through actions. |
|
Yes, good points. Let me elaborate. The example I made shows a simple composition where Does that answer your question? |
|
I've encountered the same problem and have had the same doubts about Choo's state management at scale. Once you have hundreds of components and hundreds of models, it will just not work to have everything global and have string namespaces for everything. All I can say is Mercury, despite its flaws, has the best model for state management I've seen in JS. In my opinion, the best three things about its state story are:
Mercury has a lot of flaws in terms of understandability, but we should learn from its good parts (and Elm/Om). I'd caution against copying Redux too much... It falls apart at scale in similar ways to Choo's current state system. Generally, I agree with the idea behind @mantoni's comments -- one of the biggest problems with the current system is that "everything is global." If we moved to smaller apps that can require() and compose each other, we can approach a much better system. I will try to elaborate with more specifics later. Let's keep this discussion going. |
|
It would be possible for the result of calling This would allow us to maintain isolated components and the ability to define a well-known interface for communication between them.
|
👍 to Mercury as an inspiration. i'd say though, i'm happy Redux popularized a global state atom over many isolated component states, because with a single state atom it is much easier to reason about time travel, logging, etc. maybe we can have the best of both worlds.
yes! with for a wildly different approach that i'm very interested in: thanks for opening this discussion @yoshuawuyts, keen to hear more about the experience of people building apps in the real-world, curious about the trade-offs between being manual and explicit (for example, userland namespacing by wrapping a reducer with a single function) or automatic and sugary. |
|
The current isolation namespaces provide is inadequate, but IMO it's not a problem of needing access to data through a more sophisticated query mechanism. I agree with some of the other commenters that the problem is being able to offer a well defined interface between "models". Part of the problem is that "models" are not "things" that can be referred to (since there really is only one model). "namespaces" are really "subsets of the state", although not for dispatch. My current solution is to strictly enforce the "subset" idea by wrapping The only thing this means is that every now and then I need to "cross send" between namespaces like As far as views are concerned, I like that views are handed the total state from the router and they just go down from there. I don't see a need to complicate that; I can choose to How can we isolate stores/apps/models better without complicating things? Or are we trying to solve different problems? |
|
Before sharing my opinion, I must say that before choo I used vue a lot, so I'm quite influenced by the that framework. I think that there are two good points on what have been said, 1) Add more strict rules and 2) isolate choo components as a bunch of micro-apps. In my understandindg, this would allow to write apps like this const choo = require('choo')
const html = require('choo/html')
const app = choo()
const loginComponent = choo.extend({
model: require('./models/login'),
view: require('./views/login')
})
const dashboardComponent = choo.extend({
model: require('./models/dashboard'),
view: require('./views/dashboard')
})
app.router((route) => [
route('/', loginComponent ),
route('/dashboard', dashboardComponent ),
])
const tree = app.start()
document.body.appendChild(tree)The benefit of the before code is that every component act as an individual app (with the I hope I made myself clear. |
|
Let me add my 2 cents, thanks @yoshuawuyts for involving me in the conversation. The problems with
IMHO we should get rid of the concept of models. There is state, and a user can add reducers or effects on a choo instance. This would enable people to experiment more on their own way of doing things. We should provide a shim layer with the current API. I'm not sure if it is possible, but it might be amazing if we could run two instances of choo together for two different part of the page. Regarding ORM vs graphs: I do not think it's Choo responsibility. The state holds what is displayed. If there is an ORM/graph behind, the user should feed that into the state. Another interesting data model can be modelling it via Event Sourcing (http://martinfowler.com/eaaDev/EventSourcing.html). |
|
I think event sourcing is an interesting concept which is what I tried to describe in my previous comment. Also the components/modules/thingys could work together without requiring each other or having to declare interfaces if the collaboration between them is modeled as event collaboration (http://martinfowler.com/eaaDev/EventCollaboration.html). The interface would then just be the actions and the coupling would be very low. Specifically this part of the event collaboration link is relevant when it comes to state management:
|
|
Agreeing with @mcollina, you don't want to be thinking about ORM, relationships, or graphs. That should be delegated to your database or other library, which then would provide an API for you to hook into. You should not have "models" in the traditional/ORM sense, that's what the database is for. What you want is a big observable black box of state that is faceted so that you can hook into it at various levels. Using a contrived example we can think of an app with books and movies. I would like to see an API using functional operators (map, filter, etc), but I haven't had time to try it yet. var booksEvents = db.books.getChangesStream()
var movieEvents = db.movies.getChangesStream()
var booksStore = BooksStore(db.booksInitialState())
var moviesStore = MoviesStore(db.moviesInitialState())
var sendDbEvents = booksStore({
create: (state, event) => newState
delete: (state, event) => newState
})
// we have closed around the internal state for "books", and we are
// able to use the same event names without collisions.
var sendOtherEvents = booksStore({
create: (state, event) => newState
delete: (state, event) => newState
})
booksEvents
.map(toOurEventNames())
.subscribe((ev) => sendDbEvents(ev))
// each `store.state` is an instance of observ-struct because it is
// the best observable API. We can observe state at any level of granularity.
// But it would be good to not expose the `.set` method, so that only the
// reducers are able to update state.
var appState = observStruct({
books: booksStore.state,
movies: moviesStore.state
})
// The views only care about when the state has updated, but they are
// also observable, because they need to emit actions.
var bookEvents = BooksView(appState.books)
booksEvents.on('submit', (ev) => db.newBook(ev))
// We have created a feedback loop of `view.pipe(database).pipe(view)`.
// What I want to know is how granular the feedback loop structure can be,
// and what can we say about it if we restrict the operations involved? This
// is what people call "fractal", where the application structure is the same
// as the structure of sub-components. The database is basically the same also —
// a change log of operations over time.Looking at the original comment, it seems like maintaining view state is the issue really, not persisted state. Maybe it would be good to look at it from the perspective of what is not to like about the mercury pattern, using a tree of |
|
Let me go on the "mad science" path. Let's consider:
I consider an event as a js object, and the handlers can subscribe for changes using pattern matching, see https://github.com/mcollina/bloomrun for that (the size of this might be an issue, but we can work on that, there is plenty of room for improvements). We can shortcut the pattern with https://github.com/mcollina/tinysonic. I'm throwing in @mcdonnelldean because he worked on https://github.com/mcdonnelldean/nanite |
|
@yoshuawuyts Is there any "conclusion" about this discussion? Is the state management of choo going to change in the mean time? |
|
So here's the two things I like most about
So basically I would happily adapt to any state management changes that respect these advantages. One caveat I'd like to add though, is to be careful using large-scale apps as a guideline. Apps at scale will by definition be complex, often in very different ways, and if you start to import all that complexity in a framework like |
|
Heya, thanks for the replies everyone! - Even though I've been a bit silent, I've read it all, and y'all put out some great stuff. What I'm thinking right now is:
How does that sound? ps. Also docs everywhere; we're using selectors on a real world project rn and it seems to work alright enough - just a few tweaks could probably make it better C: |
|
Just thought I'd chime in on this. Nothing significant to add solution-wise, but want to voice some friction I'm experiencing with namespaced stores. This could either be resolved with a rejig in my app, or add weight to the argument that namespaced stores aren't helpful. Let's see! So in my app the logged in user's auth credentials (a jwt) live in the top level app model with no namespace. This is all well and good for views, because they recieve the entire state tree (for all those occasions when I want to render the jwt to the screen 😁). However, when I want to trigger an effect on a namespaced model (e.g. Any ideas on how I can rework this, or is this exactly the type of reason why namespaces have come in to question? Random unthought through proposition: what if effects had access to the entire state tree? |
|
@bengourley I also experienced some friction with namespaced models not having access outside of the namespace. I was able to work around it, but I can see it becoming a problem. My impression so far is that namespacing of models is nice, in the way that modularizing your app is, but not so sure that restricting access between them is going to work. |
|
@bengourley and @aknuds1 I have dealt with the sharing problem by copying the data across namespaces during update. For the credentials problem I've made an effect I don't have a way to do this in a declarative way (this property of this state should always match this other property of this state) and I have to manually make sure that I never update the cloned state in any way other than through the |
|
@myobie, @bengourley , @aknuds1 - I've encountered the same problem. I've worked around this by having a global vuex handles effects nicely IMO. Effects have access to the entire global state, whereas mutations only have access to state from the namespace. I draw a lot of similarities with vuex simply because it's so easy to use, but here's my humble two cents regarding state in choo. When effects and mutations in vuex are called, they return the result of the effect / mutation. This makes it real easy to use promises instead of callbacks if that's your jam, or immediately access the updated state of a reducer during an effect. I haven't had any trouble with views and state, but being able to access state outside of choo's views and models would be really handy for use cases such as authentication. |
|
@josephluck I really like the idea that reducers are only for updating local state while effects have access to the global state. @yoshuawuyts have you considered this? About using global state outside of choo: I have found subscriptions to be more than good enough. They always force me to have good boundaries and so far I have always been able to do what I want with a simple EventEmitter as a message bug into a subscription. |
|
Yeah I've thought about this - it might make sense, and if prior art in Vue
shows this is a good idea then it might definitely be something we could
adopt
…On Thu, 29 Dec 2016, 15:43 Nathan, ***@***.***> wrote:
@josephluck <https://github.com/josephluck> I really like the idea that
reducers are only for updating local state while effects have access to the
global state. @yoshuawuyts <https://github.com/yoshuawuyts> have you
considered this?
About using global state outside of choo: I have found subscriptions to be
more than good enough. They always force me to have good boundaries and so
far I have always been able to do what I want with a simple EventEmitter as
a message bug into a subscription.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#252 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ACWlepxFUHPxx5faMDrAtjZ2vT3DKh0Eks5rM8cMgaJpZM4J5FAq>
.
|
|
So the proposition, if I understand correctly:
Sounds good to me ✅ |
|
Great rabbithole of a thread here, thanks for the links. I was looking for a simple way to cache some expensive state, and found reselect to be functional / legible enough to work quite easily with choo. Here's my PR pulling some expensive stuff to cached selectors: softfab/tshirt#2 |
|
We figured it out - we're now using an event emitter and all state woes are solved. Check out master for the latest version; run Closing because |
|
I think a way to import emit and state would have been nice addition. |
and others
As I'm working on a real-world application I'm noticing there's friction using
stores, and I'm becoming unsure if namspacing is the best construct available.
Nested data structures are hard to model and use. Say we're building a giant
form website with multiple forms on it, we might have the fields of "form",
"questions" and "fields". We might perhaps want to track the state of these
separately; as having a single blob make up 90% of our application doesn't
quite feel right. "form", "questions" and "fields" all have their own little
bits of logic and what not.
rdbms
An approach we could do here is have each of these be expressed in a relation
of "one-to-one", "many-to-many", "many-to-one" (
oneToOne,many,fkinRDBMS-speak respectively). This would allow us to define the fields between the
respective items. And flatten them in our representation, making it easier to
view and reason about. There's prior art in redux-orm and querying a
redux store
The redux docs state the following:
There's also reselect which creates higher level views on top of
normalized data, and updates to match. I like this idea because it matches the
idea of how I think about data.
namespaces
But all of this raises the question: how useful are namespaces at this point?
If our data is completely intertwined, and method calls on values rely on all
data being available (e.g. "how many fields are completed on this form?"), then
what's the point of having namespaces? In a sense we'd be implementing
namespaces as a way to interact with our flat data.
Some considerations we might need to make regarding namespaces:
wrappers API from
3.3.0?I suspect it's definitely possible, and would shrink the codebase significantly
namespace packages on top of existing choo if we just don't namespace our
models. Probably it would need to use a different character internally to
namespace when sending actions, but that's about it (e.g. don't use
:as it'sspecial right now).
graph
I feel the endgame of RDBMS and selectors is to implement a graph structure
where relationships can be used to query data (e.g. "give me all questions of
form foo", "give me all fields of question bar").
Now what I wonder is if we could leapfrog over the implementation details, of
RDBMS and selectors and implement a graph structure directly on top of a single
object. This would be similar in spirit to @mcollina's levelgraph.
Now my knowledge on databases isn't great, and I don't have any prior
experience implementing databases (relational, graph or otherwise) so any input
here would be grand.
edit: for those unfamiliar with graph databases, I've found a great article
explaining the differences between graph and RDBMS databases:
https://neo4j.com/developer/graph-db-vs-rdbms/. Especially the chart showing
the same data expressed in the different paradigms is interesting I found:
relational
graph
wrapping up
I'd be keen to hear people's thoughts on this. Like I said this post is coming
from seeing choo's existing abstractions break down at a larger scale, and the
desire to do better. Thanks for making it this far down the post; let me know
what you think ✌️
Related issues
See Also
The text was updated successfully, but these errors were encountered: