Avoid Stale Observable Values

[twavv]

Continuation of #249.

The current values of the observables are embedded in the WebIO payload when the relevant thing is render'd; so, for example, a refresh will not show the latest values of the observables.

The solution to that part would be to have observables start as empty and have the client side request the latest values upon being displayed in the frontend. The downside to that is that if a notebook is opened and the kernel is gone, there would be no stored values.

Which leads to the idea of storing things in a cache.

The solution to that would be to have data saved in some kind of local storage (but specifically not localStorage, the browser thing). For example, in Jupyter, we can store stuff in notebook metadata (I believe this is what IPyWidgets does); for Blink and Mux, it doesn't really matter (when the kernel is gone, so is the Blink window), so we could set up the save to just be ignored.

The actual way this works might be to expose a setSaveCallback method on the WebIO instance; so, for example, in the Jupyter notebook provider, we could do

WebIO.setSendCallback((data) => comm.send(data));
WebIO.setSaveCallback((data) => Jupyter.notebook.metadata["@webio"] = data]);

Then we'd have to figure out how to query it.

Do we want to always default to cached data?

If yes, we'd still need to request the latest data and potentially update/overwrite the existing data. This might leady to "janky" UI's where old data flashes before new data loads. One alternative might be to request the data, and if WebIO/Julia says no such data exists, use the cached.

Do we want to cache at all?

Things aren't well defined when the kernel isn't available and we're using cached data. How do things work when most WebIO related things don't work (when the kernel is gone)?

What happens when code tries to update observables? Do we update them locally and not server-side? Probably not since that would result in inconsistent UI's (imagine a button that switches the plot rendered from A to B; the UI says plot B but the actual plot, which is Julia-generated, says plot A).

How do we manage the accumulation of data?

If we save data for every scope, a notebook will grow in size a lot (more data stored for every new observable and every new scope, which in a notebook, is likely to be a lot due to the processes of trial and error).

[JeffreySarnoff]

(obviously) when the content of an observed state ([any of] the current value[s] of an observable) is revised [akin to when an html widget is made to change what it widgets about] the medium through which an observable is observable must refresh or repaint or recreate to keep the sense a user ought have when acting through a provided modality [the user interface, or whatever is to event-manage state and change and control].

So it is insufficient to reach, place, grasp, release by reaching into and grasping from caches. One must provide a means of determining whether that which is sought/obtained represents a modification to the experience of the user who interfaces or to the information conveying constructs or to their manners of working.

conceptual sketch

mutable struct StatefulCache{K,V}
    cache::Dict{K,V}
    currkey::K
    currval::V
end

mutable struct StatefulObservables{K, V}
    #   IdDict keys are Symbol(uniquename) e.g. Symbol(html_id)
    #        <button id="buttonbutton">peanutbutter</button>
    observables::IdDict{Symbol, StatefulCache{K,V}}
end 

[twavv]

Hi, I really wasn't able to parse your comment and have no idea what you're trying to say. :^)

(again, I'm not really sure what you were saying, but...) I think what you're saying might be to a misunderstanding of what I meant by cache (and truthfully, cache is not the right name for what I was talking about). By cache, I really just mean local storage that is used on first render or if we're unable to access the latest data (e.g., if the kernel is dead and gone).

[JeffreySarnoff]

yes .. that is always a possibility .. no worries
your work is very helpful 🔦