Speculative Sessions as Data

Sessions as data

Speculative design. Open, not settled — thinking and proposals, not shipped behavior. See Roadmap for what's in flight.

Status: speculative (2026-10-01), replay spiked; notebook records built (#630). How a kernel on the web carries calls, and how it keeps a session so that any engine can serve it. It replaces the pool in Kernels and Front Ends §4, and follows the job-store spike, which showed IndexedDB can carry calls but not that it should.

1. Three mechanisms, one job each

	carries	why this one
Direct messages (a MessagePort to the kernel's SharedWorker)	calls and answers	Point to point, request and response, no shared lock: 0.1–1 ms a round trip.
BroadcastChannel	events: heartbeats, "session s changed", "library loaded"	One to many. A missed event loses nothing that matters.
IndexedDB	the session journal	The only one of the three that outlives a reload or a dead worker.

The job store put calls in IndexedDB. Even batched, a call cost 1–5 ms against 0.1–1 ms for a message, and it took batching at every step to bring a burst within reach (the spike's §6). What the store bought (no coordinator, a call that survives its worker, one evaluation for a call two tabs ask for) messaging gets more simply:

• No coordinator: pages route a call themselves, to worker hash(input) mod N. Identical inputs reach the same worker, which answers the second from its memo.
• Recovery: the page still holds every call it sent. When a worker's heartbeat (on the channel) goes quiet, it sends the call elsewhere and drops the late answer by request id.
• One worker is the default. A pool waits until one kernel saturates.

What the store was good for is state, not transport, and that is the journal.

2. What a session keeps

Two kinds of session keep different things.

A notebook keeps its document. <notatio-notebook> is reactive (TrackedSymbols -> All): it schedules cells by dependency, like Pluto or Observable, so its state follows from its cells as they stand, whatever order they ran in. A run history adds nothing. Its record holds:

• the cells (stable id, source) and each one's last answer (TeX, typeset HTML, InputForm);
• the seed the reader started from.

It is kept in IndexedDB, one record per notebook, keyed by page path and the notebook's id (else its place on the page). On open the notebook draws the reader's cells with their last answers, which stand in until it works them out again (a draw or a newer library can change them). A BroadcastChannel tells the page's other tabs when the record changes, and they redraw from it; the last write wins. Reset returns a notebook to its seed. When the author changes the seed, the reader's edits are kept and Reset brings the new one. Transient -> True (the transient attribute) keeps nothing.

A notebook evaluates on the page, not in the kernel, so each tab works its answers out itself: the tabs show the same cells, but a random draw can differ between them.

A transcript keeps a journal. A Wolfram-style session (a <notatio-dynamic-module> without TrackedSymbols, the CLI) has Out[n] and depends on the order cells ran in. Its journal holds one entry per evaluation, in order: the source as written, the parsed input, its effects, the answer and its line number. Any engine rebuilds the session by replaying it. None is persisted yet. A journal covers:

• a reload, or a second tab;
• a worker that died or stalled: another one replays and carries on;
• a pool, should one come: a session is no longer pinned to the worker that holds it.

Either way, a custom kernel shrinks to an engine built from a configuration (the libraries enabled, precision, notation). One compute-engine constraint keeps some of them separate: the LaTeX dictionary is fixed when an engine is built, so documents with different notation still need different engines. Such an engine is built from data, though, not a different kind of worker.

Replay

A transcript's journal replays like this: only what changes the session is evaluated again. compute-engine labels those cells already, through expr.effects:

the cell	replayed by
changes the session (scope, state, or effects unknown: "any")	evaluating it again
also draws or reads the outside world (random, time, entropy, environment, network, fs_read), and binds a name	binding the journalled value, name := Out
changes nothing	writing its line into the history, unevaluated

The middle row matters because skipping a pure Random() shifts the stream, so drawing again gives another value. A seeded draw (WithRandomSeed) reports no effects and is evaluated again, which reproduces it, once our Random honours the seed (§3).

The journalled Out doubles as a display cache: a session can show every answer at once on reload, before replay finishes, as the build's prerender does for a page. A notebook record could keep its answers the same way. That is the only answer cache this design needs. A general one isn't worth it: most evaluations cost less than a lookup and a version check.

The same journal is what an .ipynb export serialises (its inputs, outputs and metadata), and what a GitHub-friendly render reads.

3. The replay spike

web/.scratch/replay.ts (not committed). It runs the site's own notebook kernel in Node (createKernel with NOTEBOOK_KERNEL, as the build does) through a 16-cell session. The session covers:

• bindings, including a rebinding (a := a + 1);
• a function definition;
• Out, In and InString;
• an assumption;
• a library declared mid-session (Binomial);
• an unseeded and a seeded draw;
• a 1.4 s exact sum.

It journals the session, replays the journal into fresh engines, then probes all three with the same twelve cells.

• Re-run every cell: every probe agrees. 16 of 16 cells evaluated, 1.6 s, about what the session took.
• Replay by the rules above: every probe agrees except the seeded draw, which is our Random ignoring the seed (below). 11 of 16 evaluated, 170–245 ms, against 1.6 s for the session. The sum isn't evaluated again; its line is restored from the journal.

The rules held for everything a session keeps:

• bindings and functions;
• Out/In/InString at the right line numbers;
• assumptions;
• libraries, which are declared again because the cells that name them are evaluated.

What it turned up, outside replay:

• The kernel wipes a function a cell has just defined. It records each cell inside the session's run with the cell's input, so the bound name is declared again (unknown), replacing the definition. Number bindings survive only because re-declaring one throws, and the error is caught. The spike patches it; the fix is to record with no input.
• The Epsil reader doesn't define functions: f(x) := … reads as ["Assign", ["f", "x"], …], which compute-engine leaves alone. LaTeX's f(x)\coloneq … works.
• Our Random ignores WithRandomSeed once combinatorics is declared: ensureRandom replaces compute-engine's stream with ours, and the seed never reaches it, though the expression still reports no effects.
• A fresh engine's stream starts at the same seed, so two tabs draw the same "random" numbers. A host may want to seed each session.
• Out(n) reports effects "any", because the transcript declares it without effects. It is evaluated again on replay, which is cheap but not needed: reading history changes nothing.

4. What carries over from the job store

• The effect rules (what is kept, what replays), "any" included.
• Canonical keys, should a memo or cache key anything.
• Liveness as a heartbeat per worker, on the channel and never in IndexedDB. A write issued just before a blocking evaluation holds its store's lock until the worker comes back.
• A stalled call re-run once in a safer mode, then reported.
• IndexedDB in batches: one transaction for several records.
• compute-engine's async gap (cortex-js/compute-engine#392): until lazy heads have async handlers, a long evaluation blocks its worker, and progress can't be reported.

Steps

1. Fix the kernel's record call, with a test that a := function survives the next cell (#577).
2. Notebook records: kept, reopened, shared between tabs, Reset, Transient (#630).
3. Keep a notebook's answers in its record, typeset, and show them at once on open (#630).
4. The transcript journal: the kernel appends entries; a session opens by replaying one. Behind KernelSession, so the transport doesn't change.
5. Export a notebook as .ipynb, for Jupyter or for GitHub (#630); a journal later.
6. A worker pool, routed by input hash, only once one kernel saturates.

Open

• Where replay starts: always from the top, or from a snapshot of the bindings every n lines. The spike's sessions are short; a long notebook may need snapshots.
• Notebooks on the kernel: a notebook evaluates on the page, so it loads compute-engine and two tabs work out their answers apart. Scheduling its cells through the kernel would give tabs one session, and a page with a notebook no engine.
• Effects compute-engine doesn't see: a cell whose kernel call declared a library has no effect label for it. Replay works because the cell is evaluated again; a skipped cell that needed a library would bring it back only once a later cell names it.
• Seeding per session: whether a session's stream starts from a seed kept in its configuration, so a replay draws the same values, and two tabs don't.
• Journal size and eviction, per notebook and per origin.