Octopus State Graph

State and Orchestration for javascript apps

This turborepo contains:

• octopus-state-graph, the source of the npm package, ready for use in your apps.
• two sample applications in React and Vue
• octopus devtools, as a locally hosted popup, which can also be packaged and installed as a Chrome extension. Visualize the emergent structure and live state of your running application.

Get started

git clone https://github.com/bbsimonbb/octopus-turbo.git
cd octopus-turbo
npm install -g pnpm
pnpm install
pnpm run dev

What am I looking at here?

The problem

The state and behaviour of our applications have a structure distinct from the structure of the DOM. When we try and shoehorn state-and-behaviour into the DOM tree, shared state ends up arbitrarily in a common ancestor, behaviour ends up all over the place, and props are used excessively to pass state and behaviour wherever it's needed.

State management solutions help, but they may misframe the problem by prioritising state over behaviour. The functional paradigm that dominates the React space may not be best adapted to modelling UI's, where user input, and network responses, can come from any direction. The various implementations of computed may help, but they fall short if they're only availabe for use in UI components. The distinction between state and computed may be unecessesary. Computed should be able to generate more state, and unleash more computed.

The Octopus solution

Octopus invites you to put your shared state and the logic that acts on it in nodes. A node has a val, methods and a recalculate function, reup(), that can reference other nodes in its argument list. val can only be modified by the node's own methods and reup().

When a method returns, reup(), will be called, and supplied with all the values named in its arg list, then the chain of reup()s referencing our node will be called in sequence, and supplied with the vals they request.

Under the hood

The structure formed by the reup() functions is a Directed Acyclic Graph. (Octopus enforces this, and prevents you from creating cycles.) DAGs are much discussed in comp-sci (and implemented internally all over the place), but Octopus appears to be the first reusable, turnkey, ready-to-wear, off-the-shelf implementation of a DAG for application development, in any language, that I'm aware of. ¹

This is remarkable because DAGs hit a sweet spot in the middle of the three common programming paradigms (OO, event-driven, functional). Let's have a DAG as the top-level structure of our applications. Data-fetching and onChange handlers live in DAG nodes, next to the data they act on. Your app logic and data are cleanly separated from UI (not just for the sake of it, but because they have their own distinct structure). The emergent logical structure of your app can be visualized and reasoned about. Needless recalculation is eliminated and UI components become much simpler, just dumbly reflecting bound values in the graph.

Sample Apps

In the apps folder, there are two versions of the same sample UI, shown above, one with React, one with Vue. Click around. Simple on the surface, there are a lot of rules to implement...

• Total price sums pizza, delivery and tip. The value of tip may depend on the value of pizza
• Any of the five user controls can invalidate the order.
• The state of the (orange) pizza control depends on user input, the size chosen and the base chosen.
• Warnings only show if you've interacted with a control, or if you've tried to submit.
• Warnings only show if preconditions are met. Why bully you to choose a pizza when you haven't chosen a base yet?

Like many (all?) UI's, this UI is a directed acyclic graph. It could be modelled as an object-oriented system, or an event-driven one, or with a functional paradigm, but it intrinsically is a directed acyclic graph, because this is the loosest possible structure that doesn't admit infinite loops. The order of controls on screen is usually a good approximation of the structure of the graph, but of course in a UI the user can interact with controls in any order. How are we going to ensure that the state is always consistent? Let's see what happens when we make the graph structure explicit...

This graph, generated by octopus-state-graph, visualized with octopus devtools, shows you the structure and state of your running application. Essentially, each user control on screen is underpinned by a node in the graph. A node encapsulates a meaningful concept in our subject domain.

As we started to see above, Octopus has essentially two functions: Firstly, on build(), it builds the graph based on the call signatures of all the reup() functions, checking that the requested dependencies exist, and that no cycles are created.

Secondly, octopus ensures that when a node's val changes (when a method returns), the graph will be traversed, and all the reup() functions starting with that of the changed node, will be called sequentially. (The sequence is determined by the topological sort of the graph.) As such, for any given external change, a node only recalculates if it is downstream of the change, and it only reacalculates once, after all it's predecessors have updated.

(There is a ton of scope for optimising traversals. Methods could report if they made a change, or we could detect this. Nodes would then be reupped only if necessary. Going further, a tricky implementation with promises could let different branches of a traversal proceed independently, such that a slow node only delayed downstream branches that depend on it. Stay tuned!)

Nodes can fetch data. In many situations, it will make much more sense to fetch your data into this persistent, reactive structure, rather than fetching from your UI components that come and go as the user navigates. Unlike a reducer, a graph traversal will happily wait while a network call completes, and downstream nodes will then recalculate taking account of the fresh context.

Alternatively, a node can launch a network request and return immediately. Antecedent nodes might go into a waiting state, which you can use to control spinners etc. Then when the fetch returns, a node method handles the return and a new traversal is initiated, clearing the spinners and displaying the fresh data and any cascading effects.

So now the picture emerges. Your nodes lie at the intersection of your system and the outside world. Like in OO, they encapsulate a bit of state, and the methods that modify it. Like event-driven systems, they can "subscribe" to relevant changes and their responsibility ends when they publish their value. There's a hat-tip to functional programming and one-way data flow in the notion of a traversal, but this approach is intentionally much less dogmatic. State, both private and published, can accumulate in nodes, and reup() functions can be async and impure. And we get to mutualise, in the reup() function, the magic sauce that combines some user input with the current state of the system to produce the node's current value, a value to which downstream nodes can then react. All this is done with no funny business. Your val is not proxied, there's no expensive change detection and very limited passing around of functions. There are no new concepts. Node code is biblically simple.

Reporting nodes

A super possibility of octopus is reporting nodes. A reporting node chooses its predecessors not by name, but with a filter function. Look at the totalPrice and allValid nodes in the sample applications to see how this is done. Any new node whose published val contains a price will automatically contribute to the total.

Serialization

The last thing to cover is serialisation. In complex applications, you may want to be able to save and reload your graphs. To do this, we should distinguish between hot and cold state. Hot state is the full picture, presented to the UI layer. In this pizza example, the full list of option values (present in the hot state) comes from the source code, so there's no need to duplicate it into the cold state. The cold state will just need to include the user's choices among the options. The total price is trivially calculated so again there's no need to save it. So to save our graph, we just need to implement saveState() and loadState() on our options. When we call graph.saveState(), we get a little object containing just the saved user input, together with two properties that allow us to efficiently rehydrate the graph without needing to rebuild.

Simplest possible example

See the tests for working simple examples. Essentially you need to...

const graph = createGraph()
graph.addNode("nodeName", aNode)
...
if(/* you're starting afresh */)
    graph.build()
else
    graph.loadState( JSON.parse(savedState) )

// your UI is live, then when you're finished...

const savedState = JSON.stringify(graph.saveState())

Integration with front-end frameworks

To integrate with a front-end framework, we just need the framework to observe val. In Vue, this is easily accomplished by wrapping val in reactive(). For react we need to grab mobx, and wrap val in observable(), and our react components with observer(). Additionally, mobx likes you to tag state-modifying functions as actions, so that all actions can complete before the DOM is modified. You should do this at the outermost level, wrapping your handlers with action() in your react components. You should also wrap any callbacks you create inside reup() and methods. All this is demonstrated in the React sample.

devtools

DO NOT MISS the devtools extension (screenshot above). You can visualise the graph of your UI, see in real time what nodes you're interacting with, what value they publish, and navigate directly to the source. Click the octopus to bring up devtools in a popup. It's not in the Chrome store yet, so to install it you'll need to download and build the project, then Extensions => Pack extension

Footnotes

1. In the React space, Jotai is perhaps closest in spirit (atoms can take dependencies on other atoms). Following this discussion on hackernews, I'm now aware of Dagger and Guice, DI frameworks for java, and XState state management and orchestration for js/ts. I'm developing this to scratch my own itch, and the conviction that I'm onto something is only strengthening, so development continues! ↩