v0.3.0

The new package events adds event sourcing to the writer, with inline and catch-up projectors, rebuilds, retention and reads as of any position. A page that is too slow for its stream now stays close to the present with newest-frame.js, a Datastar plugin that lith serves. It applies only the newest frame of each view, and asks for a slower stream when it skips most of them. No API was removed. What to check in an app:

• To bound a slow page's lag, serve live.Scripts' newest-frame.js under /static/ (static.Add("lith/"+live.NewestFrame, js) on the app's edge.Static), load it after Datastar through the import map, and put data-newest-frame on <body>. Open the stream with headers: $_slow ? {'Save-Data': 'on'} : {} at the same URL as before, and set live.Spec.Slow from edge.SlowLink(r). The template does all of it.

• The Grafana dashboard and alert rules for lith apps are in deploy/observability at the repository's root and no longer filter on a scrape job: install them again on a host that has the old ones. They match the metric names and the label app, and the dashboard has a row for events.

• Apps opt in to minified JavaScript and CSS by importing the new module github.com/zweiundeins/lith/minify and passing edge.Minify(minify.Bytes) to edge.NewStatic, outside development; an app that does not import it serves its files as before and pulls in nothing. The template does.

• A stream whose client falls behind no longer costs renders: a dirty view waits while every stream that watches it has a frame of it waiting, and then sends the frame that waited and the newest a tick later. Nothing to change, unless a test counts a slow stream's frames.

• events: a new package for event sourcing on the writer. A command calls Execute in its Apply: it reads the stream's state row, which a Decider folds from the stream's own events, checks the expected version (Any, NoStream, Exactly, or ErrConflict), decides, appends, saves the row and runs the inline projectors (Handle), all in the command's savepoint. Streams are typed by their key, int64 or string, and their events' interface; each event type is defined once with Define, through a helper of one line per stream that makes the compiler check that the type belongs to it. Events record their time in Unix milliseconds, an actor, a correlation and a causation. Append and One append without a decision; Load and Since read as iterators over any sqlite.Querier; Upcast reads an old event type as a new one. New refuses a projector that neither handles nor ignores a type of its stream, and duplicate names. A Runner rebuilds an inline projector whose version changed into copies of the tables it names (Tables), a command per batch, and swaps them in with the last batch, so pages never read a table half built; a decider whose Version changed folds each stream again at its next command. Catch-up projectors (After) run after the commit from a checkpoint, woken by the writer, with the event's position as the idempotency key; an event that keeps failing parks the projector with its error in Runner.Stats. Forget and Prune delete events, and positions, from AUTOINCREMENT, are never handed out again. The tables lith_events, lith_event_state and lith_projections are created by the first command that needs them. events/eventstest tests a decider as given, when, then, and checks a JSON sample of every event type. While a projector rebuilds, new events still reach its live table through its new handlers, in a savepoint whose failure is logged and leaves the command alone; the batches drop the topics they touch, and only the swap touches the projector's topic. A projector whose table is missing fails Runner.Run, and in commands only those on its own stream kind. LoadUntil, SinceUntil and StateAt read a stream, the log and a decider's state as of a position. Through the writer with one inline projector, it appended 18,342 events a second with 200 producers and NORMAL, and rebuilt a projection of 100,000 events in about a second (bench/RESULTS.md).

• events and prom: Runner.Stats has the last position committed (Head) and each projector's Lag behind the last event of its stream kind, kept by the commits without a query, and Store.Stats counts the events committed by stream kind, the conflicts and the refolds. prom.Sources takes the runner (Events) and the store (Store) and exports <namespace>_events_*: appended_total{stream}, conflicts_total, refolds_total, head_position, and by projector projector_position, projector_lag, projector_status (a series per status, 1 for the current one) and projector_attempts. The lith apps dashboard has an Events row, and two alert rules fire for a projector parked for a minute and for a live catch-up projector more than 1,000 positions behind for 10 minutes.

• events: Prune takes an events.Retention: Since keeps the events stored from a time on, Newest the newest so many, and with both an event goes only when both let it go; Limit caps the events one call takes. Compact is Prune that closes the books: each stream that loses events keeps one, its last, rewritten in place with its position, version, time and metadata as a snapshot of the stream's state, an event type that the app defines on the stream and that the decider's Evolve and every projector handle. A rebuild after it makes the tables it made before, as-of reads stay right from the newest snapshot on, and a stream keeps one snapshot at most. It is the one place where a stored event changes; a fourth table, lith_event_kinds, records how far each kind was compacted. Bounds returns a kind's first and last position, 0 and 0 without events, the newest snapshot first after Compact. Load, LoadUntil, Since, SinceUntil, StateAt and Bounds read a database whose tables the first command has not made yet as one without events. StateAt refuses a position before a stream's snapshot.

• writer: Tx.Local and Tx.SetLocal keep a value per writer, for packages built on it that serve many databases, such as what events knows of each.

• writer: Tx.Untouched(fn) runs fn and drops the topics it touched, keeping its changes: for writes that no view reads yet.

• Docs and skill: the guide Event sourcing; the commands guide's event-sourced command and the board example run on events, and the review checklist has a section on events.

• Website: a ninth demo, Town in time: a town of tiles built together and event-sourced on events, a stream per tile with a decider and two inline projectors, and a slider that takes each tab back to any position of the log through a history table, the town as of each position a view that every tab there shares. The site runs an events.Runner and exports lith_site_events_*. The town's sweep keeps its newest 5,000 events and a snapshot of each tile from before them (Compact), so a rebuild makes the same town, and the badge in the stage's corner says past instead of live while a tab looks back.

• Website: a demo can be featured (demo.Demo.Featured): its card comes first and takes four cells, so nine cards fill every row on /demos, and on the landing page, which has five columns when the page is wide enough, it takes two. Town in time is featured.

• Template: the Caddy block is deploy/app.caddy, for /etc/caddy/sites/, and deploy/observability holds the app's blackbox probes and an Alloy file that ships its journal, with slog's level as a label, and its access log to Loki. deploy/setup-host.sh installs them into the folders a shared host's Prometheus and Alloy load, each only where that service is installed, and reloads Alloy, putting the old file back if Alloy refuses the new one. The CI deploy key is now optional. The guide Metrics and monitoring lists the folders and the labels; an app made from an earlier template can copy the folder and the script.

• lith-new: app_ in the template becomes the app's name with - as _, for identifiers such as Alloy's component labels.

• The Grafana dashboard and alert rules for lith apps moved from site/deploy/observability to deploy/observability at the repository's root, with a README on installing them. They no longer filter on a scrape job: they match the metric names and the label app, the dashboard also filters by slot, and each alert fires for one app and slot.

• edge: SlowLink reports whether a request says its client is on a slow link: Save-Data: on, which browsers send when their user asked to save data, and lith's pages send on their stream's request from their own measurement; Chromium's client hints, ECT of slow-2g, 2g or 3g, RTT over 300 ms or Downlink under 0.8 Mbit/s, which Chromium sends once a document has asked for them with Accept-CH: ECT, RTT, Downlink (NetworkHints); the query parameter net=slow, which pages put in their stream's URL before they sent Save-Data, for the tabs still open from then.

• live: Spec.Slow marks a stream to a client on a slow link. It sends each view's newest frame 250 ms or more after the last, about four a second. A view's first frame goes out at once, and so does the frame of a view that sent nothing for 250 ms, which takes the frames that wait along: views that change together still arrive together, and an outcome is not held up. Heartbeats go out as before. On a phone with its CPU slowed four times on DevTools' Slow 4G and 3G presets, a tab of a Game of Life that steps ten times a second kept its main thread 54 to 56% busy instead of 89 to 90%, and its stream sent 2.7 to 2.9 KB/s instead of 5.2 to 6.2. A phone too slow for the page needs it on a fast link too: with the CPU slowed 15 times on an unthrottled link, the tab fell 94 to 98 s behind the board in five minutes without it, while the hub dropped no frame and no write waited, since the browser and the kernel held the frames; with it, the tab stayed within 1.2 s.

• live: a dirty view does not render while every stream that watches it still has a frame of it waiting. The first stream that takes its frame marks the view dirty again, and the next tick renders it as it is then, so a stream that falls behind sends the frame that waited, then the newest a tick later. A stream whose writes block costs no renders until its write deadline ends it, and a view that only slow streams watch renders about four times a second instead of at every tick: for 50 tabs with a view of their own that changes at every tick, 0.4 cores instead of 1.6 to 1.7 on slow streams, and 0.03 to 0.05 instead of 1.4 to 1.7 on streams whose client stopped reading. Stats.RendersDeferred counts the renders put off, and lith/prom exports it as <namespace>_live_renders_deferred_total. A view that one of its streams keeps up with renders at every change, as before.

• The template's page measures its link once as it starts, into $_slow: navigator.connection's saveData or an effective type of 2g or 3g, or where that is missing, a document whose first byte came more than 300 ms after its request or that came at under 100 KB/s. Its stream's request then sends Save-Data: on, at the same URL as on a fast link, its documents send Accept-CH, and its stream handler sets Spec.Slow from edge.SlowLink.

• live: NewestFrame, newest-frame.js in Scripts, is a Datastar plugin that does in the page what the hub does for a stream: the newest frame of a view wins. A page too slow for its stream falls behind without the server noticing, since the browser holds the frames: Chromium then hands the page 2 MiB of the stream per read, which Datastar applies in one task without a paint, and Firefox keeps them in its parent process. With data-newest-frame on <body>, the page holds the stream's patch events, applies only the newest frame of each view among those that arrived together, and after a batch waits half as long as that batch took to apply. A page that keeps up applies every frame. When it drops more than half of its frames for 3 s, it sets $_slow and opens the stream again, which then sends Save-Data: on at the same URL, so Datastar aborts the old request. Serve it under /static/ by its content's hash (edge.Static.Add) and load it after Datastar, through an import map that maps 'datastar'; the template does, and its page now has that import map, with the request's CSP nonce. On lith's site, in headless Chromium 154 with its CPU slowed 15 times on an unthrottled link behind Caddy, a tab of the Game of Life asked for a slow stream within 2 s and stayed 0.68 s behind the board at the median over 2 minutes, 1.08 s at most, where it fell 42.6 s behind without the plugin; with every frame costing the page 300 ms more, Chromium 154 and Firefox 158 both asked after 2.8 s and stayed within 0.8 s.

• edge and the new module minify: edge.Minify(fn) has edge.Static minify its JavaScript and CSS, the files NewStatic reads and those Add gives it, before it names and precompresses them, so the hashed URL, the ETag and the brotli and gzip copies are all of the minified bytes. A file the function fails on, or shrinks by less than a twentieth, goes out as written, and the function never sees *.min.js or *.min.css, so a vendored build keeps the bytes its integrity hash pins: tdewolff takes 0.4% off the Datastar build, which a page that pins it would then refuse. github.com/zweiundeins/lith/minify has the function, minify.Bytes, on tdewolff/minify; an app opts in by importing it, and one that does not pulls in nothing. The template minifies unless it runs with -dev, which go tool task dev passes, so devtools show the files as written; its go.mod requires the module as it requires lith, and lith-new writes lith's version for both, or replaces both with -lith. The site minifies its CSS bundle and its scripts through it instead of its own code, and with -dev serves them as written. The template's stylesheet and newest-frame.js take 1,846 bytes under brotli instead of 3,189, NewStatic takes about 6 ms longer at startup, and the template's stripped binary is 644 KiB larger (bench/RESULTS.md).