Skip to content

Releases: zweiundeins/lith

v0.4.3

Choose a tag to compare

@mbolli mbolli released this 22 Oct 09:23
  • minify: Bytes no longer panics on CSS with a data: URI, which made edge.NewStatic with edge.Minify(minify.Bytes) crash at start. It handed tdewolff's CSS minifier no minify.M, which that minifier needs for a data URI; a data URI now comes out in its shorter encoding.
  • edge: Static hands Minify's function a copy of each file. tdewolff's (*minify.M).Bytes renames JavaScript in the array it is given, so a script it saved less than a twentieth of went out as written but scrambled, and did not parse.
  • writer: tx.Publish(v) hands a value to in-process listeners once the command commits, and w.Subscribe[T](ctx, n) returns a channel of the values of type T, in commit order. It is best effort: a full queue drops the value for its subscriber alone (Stats.Dropped), a crash between commit and delivery loses it, and nothing is replayed, so a listener that must not miss a value reconciles from the data. Stats.Subscribers counts the subscriptions open.
  • prom: writer_published_total, writer_publish_dropped_total and writer_subscribers, and a Published values panel in the dashboard's writer row.

v0.4.2

Choose a tag to compare

@mbolli mbolli released this 21 Oct 09:47
  • events: Compact no longer reads the events a stream keeps, and while Since moves on, Compact and Prune no longer read again on each call the events a Limit left: on a stream that keeps 100,000, a call with Limit: 1 takes 0.06 ms instead of 9 to 14 ms, and with 40,000 waiting, Limit: 10 takes 0.1 ms instead of 3. LoadUntil and StateAt read a stream no further than their position. A Compact whose Since moved back now folds the events after the compacted position that it would have left before.
  • sse: Stream.Upgrade returns nil for a stream that will never move, instead of a channel made outside any testing/synctest bubble, which stopped a bubble's clock once a stream with Reconnect had been open for a minute.

v0.4.1

Choose a tag to compare

@mbolli mbolli released this 20 Oct 15:32

Fixes from an external review of v0.4.0, and a hub that allocates less. What to check in an app:

  • Set Client: edge.ClientKey(r) in each public stream's live.Spec: live.Options.MaxStreamsPerClient (16) then answers a client's further streams with 429.

  • Without GOMEMLIMIT or a cgroup limit, the streams' budget takes a quarter of the machine's memory instead of no limit, and live.New warns.

  • db.R opens the file with mode=ro, and sse.PatchElements panics on a line break in a Patch option.

  • edge.Compress prefers zstd to brotli: Chrome, Firefox and Safari 26.3 now get zstd.

  • A context derived from a Render's ctx that timed out ends with context.Canceled, its cause DeadlineExceeded.

  • live: Spec.Client, Options.MaxStreamsPerClient and ErrClientFull; Hub.HasStream. A tick allocates 3 times a view instead of 13, and 3 instead of 26 when Depends topics move. FrameBuffer(ctx) lets a Render format its frame without a copy.

  • sse: AppendPatchElements and NewElements format into the caller's buffer. Stream.Ping sends a heartbeat without waking a hibernating stream. ReadSignals gives a request body 10 s (SignalsTimeout). BudgetStats.From says where the budget's limit came from.

  • writer: a command alone in its batch skips its savepoint (a 186 MB update: 0.33 s instead of 0.47 s). Options.SlowCommand logs a command that runs over 5 s, and prom exports writer_command_running_seconds. Options.CheckpointAside, off by default, checkpoints beside the batches and halves the stall after a large write.

  • edge: Static.Warmed for readiness, TrustedProxies for a log line at start.

  • Template and site: /pace and /ack limited per address, /static/ outside the sessions, 409 for a tab without a stream, tab state swept after a day, request bodies capped at 1 MB in Caddy, a 0660 socket. The site moves to a Unix socket (site/deploy/README.md).

To update an app made from an earlier template: add Client to the stream's Spec and MaxStreamsPerClient to live.New; put paceAddrs and ackAddrs in front of paces and acks; mount site.Static() outside Routes and add ready.Add("static", site.Static().Warmed); in sign, refuse a tab without s.hub.HasStream; copy migrations/002_tab_state_seen.sql and set TabTTL to a day; chmod the socket 0660, set RuntimeDirectoryMode=0750, run usermod -aG app caddy, restart Caddy, and copy the request_body and format filter parts of deploy/app.caddy.

v0.4.0

Choose a tag to compare

@mbolli mbolli released this 19 Oct 14:48

Views can declare what their Render read, topics form a tree by their : segments, and the hub numbers its commits, for caches of derived values. A read can skip the statement caches, and a stream the hub stops keeps its clean end over HTTP/2. No API was removed. What to check in an app:

  • Topics with : now count for the levels above them: a commit to board:42 touches board too, so a view on a bare kind such as board renders at every board:<id> commit.

  • Per-tab, session and presence state belongs under a root of its own (tab:…, sid:…, here:…), not under a root that shared views listen to. lith's site moved its demos' state out from under each demo's slug; the template needs no change.

  • sse: a write under way when the hub stops its stream gets up to 250 ms to finish, like the end frame, where it had a deadline of now. Over HTTP/2 that deadline reset the stream with INTERNAL_ERROR, and the page lost its clean end and its Reconnect event on a deploy.

  • live: Depends(ctx, topics...) declares, in a Render, topics the view read. They mark it dirty like Topics and replace those of the Render before, so a view keyed by the tab follows the tab's route without Unwatch and Watch.

  • live: Hub.Seq numbers the commits, Hub.Version(topics...) is the last commit to touch one of them, and Seq(ctx) is, in a Render, the last commit its snapshot is sure to hold. A cached count stays valid while its topics' version is not above the number it was computed at. The hub keeps the versions of the Options.Versions topics changed last (32,768 by default, 4.3 MiB at most); a topic pushed out reads as changed then.

  • live: topics form a tree by their : segments. A commit to src:A:7 marks the views of src:A and src too and moves their versions; one to src:A:* marks every view at or below src:A, and * alone is MarkAll. Topic(parts...) builds one from outside ids such as urn:uuid:…, escaping :, * and %; Below(topic) adds the :*. The levels above keep their versions in a store of their own, Options.PrefixVersions (32,768 by default), so a sync of more items than Options.Versions no longer pushes out the version of a space. A topic with an empty segment or a * panics in Serve, Watch and Depends. A commit of two topics at depth 2 costs about 60 ns more (BenchmarkChanged).

  • sqlite: Uncached(ctx) runs a query without the statement caches, on the read pool, in a Render and in a command. SQL built at runtime, such as an IN list sized to the results, no longer pushes the statements that repeat out of a connection's cache.

  • Docs and skill: recipes for a topic hierarchy, a per-tab view that navigates with Depends, and an aggregate cached against a version; views in a shared database, keyed by the person and the route; large updates and deletes, by rowid range or in Exclusive; live without lith's writer, for an app whose own store calls Hub.Changed after each commit; and pinning Readers, CacheKiB and the hub's Workers in a desktop app, since they grow with GOMAXPROCS. New benchmarks measure large writes and the hub (BenchmarkLargeWrite, BenchmarkChanged, BenchmarkTick), and bench/RESULTS.md compares the board with v0.3.1.

v0.3.1

Choose a tag to compare

@mbolli mbolli released this 19 Oct 11:21

newest-frame.js now only drops frames in the page by default, and once a page reports that it fell behind, the server paces its stream by the page's acknowledgements until it keeps up again. The template and the site pace this way by default, and pacing is no longer experimental.

  • live: newest-frame.js is the build of github.com/mbolli/datastar-attribute-newest-frame; live/VERSION names the commit. The template and the site leave __reopen off: at CPU 6x it held the Game of Life to 4 frames a second, 0.37 s behind where no fix was 0.10 s, and the page stayed on the slow stream for good. data-newest-frame__reopen keeps v0.3.0's behaviour. The slow-link check as the page starts is unchanged.
  • live: a stream with Spec.Ack starts unpaced; Hub.Pace paces it once its page is behind, and then it sends a write only once the page has acknowledged the last (Hub.Ack). Spec.Release (DefaultRelease, 5 s) ends the pacing once the page keeps up, measured against the round trip the page reports (Hub.RoundTrip) with Spec.ReleaseSlack (DefaultReleaseSlack, 100 ms). Spec.Window paces a stream from the start.
  • live, prom: Hub.Behind(id) counts a page's report that it fell behind its stream. prom exports it as <namespace>_live_newest_frame_behind_total and <namespace>_live_newest_frame_behind_streams_total, and the pacing as <namespace>_live_paces_total, _releases_total and _acks_total; the "lith apps" dashboard draws them.
  • Template: <body> posts each newest-frame-behind to POST /pace, which counts it and paces the stream, and each paced write has the page post POST /ack after the paint. Both routes reach only the streams of the request's session; /pace is limited per session and /ack per stream. A slow page gets about a third of the bytes; on a slow phone on a 300 ms link it lags 0.60 s at 1.45 frames a second, against 0.47 s at 2.74 with the plugin alone. A page that keeps up pays nothing. Without Ack in the stream's Spec, /pace only counts.
  • Template: the binary serves its metrics for Prometheus on a loopback listener of its own (-metrics, by default -addr's port + 100). The unit now allows loopback and no other host. lith-new requires lith/prom and lith/dbmetrics/prom, and every release tags prom/vX.Y.Z.
  • Docs: what a frame costs the page (mostly its elements, not its bytes), counting the pages that fall behind, and pacing a stream by the page, with what it costs. The slow-page numbers come from the corrected TestLifeDrift: 27.15 s behind at the median in 150 s, not 42.6 s after 2 minutes.

To update an app made from an earlier template:

  • Copy internal/web/pace.go, add its routes POST /pace and POST /ack with their limiters (paces, acks) to the server, and add both routes to edge.CSRF's Datastar routes.
  • In the stream handler, give the Spec the ID streamID(sid, tab) and Ack: ackEvent(tab). An app that calls Watch or Unwatch passes the same ID.
  • Copy the template's <body> attributes, data-newest-frame data-on:newest-frame-behind={ behind }, and behind in internal/views/expr.go.
  • For the metrics, copy cmd/app/metrics.go, the unit's network lines and deploy/observability/app.metrics.yml, and install the "lith apps" dashboard again. An app that calls out adds its addresses to the unit's IPAddressAllow.

v0.3.0

Choose a tag to compare

@mbolli mbolli released this 16 Oct 12:59

The new package events adds event sourcing to the writer, with inline and catch-up projectors, rebuilds, retention and reads as of any position. A page that is too slow for its stream now stays close to the present with newest-frame.js, a Datastar plugin that lith serves. It applies only the newest frame of each view, and asks for a slower stream when it skips most of them. No API was removed. What to check in an app:

  • To bound a slow page's lag, serve live.Scripts' newest-frame.js under /static/ (static.Add("lith/"+live.NewestFrame, js) on the app's edge.Static), load it after Datastar through the import map, and put data-newest-frame on <body>. Open the stream with headers: $_slow ? {'Save-Data': 'on'} : {} at the same URL as before, and set live.Spec.Slow from edge.SlowLink(r). The template does all of it.

  • The Grafana dashboard and alert rules for lith apps are in deploy/observability at the repository's root and no longer filter on a scrape job: install them again on a host that has the old ones. They match the metric names and the label app, and the dashboard has a row for events.

  • Apps opt in to minified JavaScript and CSS by importing the new module github.com/zweiundeins/lith/minify and passing edge.Minify(minify.Bytes) to edge.NewStatic, outside development; an app that does not import it serves its files as before and pulls in nothing. The template does.

  • A stream whose client falls behind no longer costs renders: a dirty view waits while every stream that watches it has a frame of it waiting, and then sends the frame that waited and the newest a tick later. Nothing to change, unless a test counts a slow stream's frames.

  • events: a new package for event sourcing on the writer. A command calls Execute in its Apply: it reads the stream's state row, which a Decider folds from the stream's own events, checks the expected version (Any, NoStream, Exactly, or ErrConflict), decides, appends, saves the row and runs the inline projectors (Handle), all in the command's savepoint. Streams are typed by their key, int64 or string, and their events' interface; each event type is defined once with Define, through a helper of one line per stream that makes the compiler check that the type belongs to it. Events record their time in Unix milliseconds, an actor, a correlation and a causation. Append and One append without a decision; Load and Since read as iterators over any sqlite.Querier; Upcast reads an old event type as a new one. New refuses a projector that neither handles nor ignores a type of its stream, and duplicate names. A Runner rebuilds an inline projector whose version changed into copies of the tables it names (Tables), a command per batch, and swaps them in with the last batch, so pages never read a table half built; a decider whose Version changed folds each stream again at its next command. Catch-up projectors (After) run after the commit from a checkpoint, woken by the writer, with the event's position as the idempotency key; an event that keeps failing parks the projector with its error in Runner.Stats. Forget and Prune delete events, and positions, from AUTOINCREMENT, are never handed out again. The tables lith_events, lith_event_state and lith_projections are created by the first command that needs them. events/eventstest tests a decider as given, when, then, and checks a JSON sample of every event type. While a projector rebuilds, new events still reach its live table through its new handlers, in a savepoint whose failure is logged and leaves the command alone; the batches drop the topics they touch, and only the swap touches the projector's topic. A projector whose table is missing fails Runner.Run, and in commands only those on its own stream kind. LoadUntil, SinceUntil and StateAt read a stream, the log and a decider's state as of a position. Through the writer with one inline projector, it appended 18,342 events a second with 200 producers and NORMAL, and rebuilt a projection of 100,000 events in about a second (bench/RESULTS.md).

  • events and prom: Runner.Stats has the last position committed (Head) and each projector's Lag behind the last event of its stream kind, kept by the commits without a query, and Store.Stats counts the events committed by stream kind, the conflicts and the refolds. prom.Sources takes the runner (Events) and the store (Store) and exports <namespace>_events_*: appended_total{stream}, conflicts_total, refolds_total, head_position, and by projector projector_position, projector_lag, projector_status (a series per status, 1 for the current one) and projector_attempts. The lith apps dashboard has an Events row, and two alert rules fire for a projector parked for a minute and for a live catch-up projector more than 1,000 positions behind for 10 minutes.

  • events: Prune takes an events.Retention: Since keeps the events stored from a time on, Newest the newest so many, and with both an event goes only when both let it go; Limit caps the events one call takes. Compact is Prune that closes the books: each stream that loses events keeps one, its last, rewritten in place with its position, version, time and metadata as a snapshot of the stream's state, an event type that the app defines on the stream and that the decider's Evolve and every projector handle. A rebuild after it makes the tables it made before, as-of reads stay right from the newest snapshot on, and a stream keeps one snapshot at most. It is the one place where a stored event changes; a fourth table, lith_event_kinds, records how far each kind was compacted. Bounds returns a kind's first and last position, 0 and 0 without events, the newest snapshot first after Compact. Load, LoadUntil, Since, SinceUntil, StateAt and Bounds read a database whose tables the first command has not made yet as one without events. StateAt refuses a position before a stream's snapshot.

  • writer: Tx.Local and Tx.SetLocal keep a value per writer, for packages built on it that serve many databases, such as what events knows of each.

  • writer: Tx.Untouched(fn) runs fn and drops the topics it touched, keeping its changes: for writes that no view reads yet.

  • Docs and skill: the guide Event sourcing; the commands guide's event-sourced command and the board example run on events, and the review checklist has a section on events.

  • Website: a ninth demo, Town in time: a town of tiles built together and event-sourced on events, a stream per tile with a decider and two inline projectors, and a slider that takes each tab back to any position of the log through a history table, the town as of each position a view that every tab there shares. The site runs an events.Runner and exports lith_site_events_*. The town's sweep keeps its newest 5,000 events and a snapshot of each tile from before them (Compact), so a rebuild makes the same town, and the badge in the stage's corner says past instead of live while a tab looks back.

  • Website: a demo can be featured (demo.Demo.Featured): its card comes first and takes four cells, so nine cards fill every row on /demos, and on the landing page, which has five columns when the page is wide enough, it takes two. Town in time is featured.

  • Template: the Caddy block is deploy/app.caddy, for /etc/caddy/sites/, and deploy/observability holds the app's blackbox probes and an Alloy file that ships its journal, with slog's level as a label, and its access log to Loki. deploy/setup-host.sh installs them into the folders a shared host's Prometheus and Alloy load, each only where that service is installed, and reloads Alloy, putting the old file back if Alloy refuses the new one. The CI deploy key is now optional. The guide Metrics and monitoring lists the folders and the labels; an app made from an earlier template can copy the folder and the script.

  • lith-new: app_ in the template becomes the app's name with - as _, for identifiers such as Alloy's component labels.

  • The Grafana dashboard and alert rules for lith apps moved from site/deploy/observability to deploy/observability at the repository's root, with a README on installing them. They no longer filter on a scrape job: they match the metric names and the label app, the dashboard also filters by slot, and each alert fires for one app and slot.

  • edge: SlowLink reports whether a request says its client is on a slow link: Save-Data: on, which browsers send when their user asked to save data, and lith's pages send on their stream's request from their own measurement; Chromium's client hints, ECT of slow-2g, 2g or 3g, RTT over 300 ms or Downlink under 0.8 Mbit/s, which Chromium sends once a document has asked for them with Accept-CH: ECT, RTT, Downlink (NetworkHints); the query parameter net=slow, which pages put in their stream's URL before they sent Save-Data, for the tabs still open from then.

  • live: Spec.Slow marks a stream to a client on a slow link. It sends each view's newest frame 250 ms or more after the last, about four a second. A view's first frame goes out at once, and so does the frame of a view that sent nothing for 250 ms, which takes the frames that wait along: views that change together still arrive together, and an outcome is not held up. Heartbeats go out as before. On a phone with its CPU slowed four times on DevTools' Slow 4G and 3G presets, a tab of a Game of Life that steps ten times a second kept its main thread 54 to 56% busy instead of 89 to 90%, and its stream sent 2.7 to 2.9 KB/s instead of 5.2 to 6.2. A phone too slow for the page needs it on a fast link too: with the CPU slowed 15 times on an unthrottled link, the tab fell 94 to 98 s behind the board in five minutes without it, while the hub dropped no frame and no write waited, since the browser and the kernel held the fra...

Read more

v0.2.0

Choose a tag to compare

@mbolli mbolli released this 14 Oct 15:39

Streams are measured by memory: the budget books what each stream holds, sizes its window from its frames, and with Refuse caps the streams instead of a count. No API was removed. What to check in an app:

  • A budget's Limit now covers each stream's encoder, 32 KiB besides and the frames the hub keeps, not only the encoders. The default, half the memory limit, needs no change; a Limit sized for encoders alone now lets fewer streams compress.

  • ZstdWindow and BrotliWindow are the largest window a stream may get: each stream takes the smallest window that holds its frames. MinWindow no longer defaults to 18; set it only for pages whose frames refer back further than one write.

  • With Refuse, a stream gets in only with room for a window that holds its frame, and its headers go out with its first frame. Open streams with retry: 'error' and leave live.Options.MaxStreams at its default, as a backstop.

  • A view that one stream watches, such as a tab's own, no longer keeps its frame: a stream that joins it gets a frame rendered in the next tick.

  • sse: ReadSignals reads a request's Datastar signals for any method: the datastar query parameter of GET, HEAD and DELETE, the JSON body of the others, @query's QUERY included. It caps the body at 1 MiB (ReadSignalsLimit for another cap), takes a request without signals as none, and reads the body to its end, so that over HTTP/1.1 a handler that waits before it answers still notices a client that left. Called after Open, it returns ErrBodyConsumed.

  • edge: Compress compresses the answers to QUERY as it does GET's.

  • sse: PatchElements and the other events split HTML without a carriage return with bytes.IndexByte: a 230 KB page takes 40 µs instead of 342.

  • sse: a budget books what its streams hold, not only their encoders: each stream's tier, 32 KiB for the rest of the stream (its state, its goroutines' stacks, net/http's buffers), and what Budget.Hold books, such as the frames a hub keeps. Limit, half the process's memory limit by default, now covers all of it; the other half stays for the rest of the app and for the garbage between collections. BudgetStats.Frames reports what Hold booked. Without Refuse, uncompressed streams may take the total beyond the limit.

  • sse: a stream's tier is picked at its first write, from its largest write: the window must hold that frame with a sixteenth to spare, or a frame cannot refer to the one before. A brotli stream takes that window and no larger one, at quality 4 while the budget stays at most half full, then at quality 3, and only when that does not fit, quality 3 at 2^16, which compresses each frame alone; it goes out uncompressed only when even that does not fit. A zstd stream takes the largest tier within its fair share, moves down no further than its frame's window, takes 2^15 only when that window does not fit, and moves up again at a write once room frees up. MinWindow no longer defaults to 18: unset, the frames decide; set, it is the smallest window a stream's frames need, for pages whose frames refer back further than one write. With a limit, an unset ZstdWindow or BrotliWindow lets the budget go up to MaxWindow. In 150 MiB, 114 brotli tabs with a 230 KB view each hold 2^18, and 168 zstd tabs. The tiers are booked from measurements at 50 to 230 KB: brotli's 7 to 31% lower, zstd's 32 to 48 KiB higher. Apps that set MinWindow to their largest frame can drop it.

  • sse: Stream.Upgrade returns a channel that the stream's budget closes when the stream holds a brotli window below its frames, or went out uncompressed, and a stream opened now would get a window that holds them: at most one stream a second. BudgetStats.Reopened counts them.

  • live: the hub books the frames it keeps in the streams' budget, once for all the streams that watch a view: a []byte frame at its length and 8 KiB, another by its Len() int method. A stream with Spec.Reconnect ends with sse.Reconnect when its budget has a better tier for it, and the page opens it again; the same stream ID at most once a minute, a stream without an ID after a minute.

  • live: the hub keeps a view's newest frame only while two or more streams watch it. Of a view that one stream watches, such as a tab's own, it keeps the frame's length and hash once the stream has the frame: a render that repeats it still goes to no stream, the budget books nothing for it, and a stream that joins the view gets a frame rendered in the next tick. That saves about 235 KiB a tab with a 233 KB view of its own, 34 MiB for 150 such tabs, and a budget with Refuse no longer goes past its limit by the frame of a tab that Open has just let in. A frame that is not []byte has no hash, and the hub keeps it as before. Each []byte frame is hashed once as it renders, about 16 µs for 233 KB.

  • sse: with Refuse, a budget lets a stream in only with room for 32 KiB and the smallest tier of its coding whose window holds its frame: Options.Frame, which live's Serve sets from the stream's first frames (unset, the coding's smallest tier). It also turns away a stream that compresses nothing (its client accepts no coding, or NoCompression is set) when its 32 KiB do not fit. A stream let in keeps a window that holds its frame until it closes: a zstd stream keeps that tier booked before its first write and while it hibernates, and moves no lower when the budget fills or a lower limit asks it to; a brotli stream takes no window below its frame. When the budget turns a stream away, the zstd streams above their frame's window move down to it at their next write, until the stream would fit. A stream whose frames outgrow its window later keeps its tier and sends more bytes a frame until it gets a larger one; it is never cut off. So no stream the budget lets in goes out uncompressed, in raw blocks or with a window below its frame, and memory caps the streams: an app with Refuse leaves live.Options.MaxStreams at its default as a backstop, and its pages open their streams with retry: 'error'. In 120 MiB, 134 zstd tabs with a 233 KB view each get in, all at 2^18, or 91 brotli tabs, and the others get a 503. Before, 150 MiB let in 175 zstd tabs with a 230 KB view, most of them at 2^15 to 2^17, where a frame took 4 to 35 KB instead of about 60 bytes.

  • live: Serve opens a stream once the first frames of its views are there, or the tick that renders them has ended, and gives their length to the budget (sse.Options.Frame), which with Refuse answers 503 when no window that holds them fits. The response's headers now go out with the first frame: on a hub that ticks every 50 ms, 24 ms later at the median, while the first frame comes as soon as before. A view that Watch gives a stream that the budget then turns away waits for the next stream with its ID, as it does while none is open.

  • prom: sse_budget_frames_bytes and sse_budget_reopened_total; the limit and the reservation cover what the streams hold.

  • sse: a brotli stream flushes its encoder every 16 KiB of a frame at quality 3 and every 32 KiB at 4. Without those flushes brotli extends a copy into the next input block without hashing it, so after a change in a large frame it lost the frame before and coded the rest in short pieces: a 230 KB board at quality 3 and 2^18 took 3,173 bytes a frame and now takes 259, a 230 KB data table that scrolls 5,756 and now 227 (quality 4: 2,382 and 140, 3,450 and 134). The encoder's heap no longer grows with the frame: at 230 KB, quality 3 at 2^18 held 1,554 KiB against the 1,312 its tier booked, and now holds 1,083; quality 4 at 2^18 2,370 against 2,192, now 1,603. Each flush costs a metablock header: a 230 KB list with 20 changes spread through it takes 676 bytes a frame instead of 363, and a 50 KB board at quality 4 50 instead of 34. The CPU per frame of a 230 KB frame changes by 13% less to 30% more.

v0.1.1

Choose a tag to compare

@mbolli mbolli released this 14 Oct 07:59
  • sse: with ZstdLowMemory set and a memory budget (every live hub has one), a client that accepts brotli but not zstd, such as Safari, got its stream uncompressed: the low-memory filter dropped the brotli tiers too. It now narrows zstd's tiers only. Apps that set ZstdLowMemory should update.

v0.1.0

Choose a tag to compare

@mbolli mbolli released this 13 Oct 21:36

The first release: the runtime for Go, SQLite and Datastar apps, with a template for new apps, a skill for coding agents and the website at lith.zweiundeins.gmbh. It needs Go 1.27 and builds with CGO_ENABLED=0.

Packages

  • sqlite: one write connection and a query_only read pool; pragmas; migrations by user_version or a project's own runner; read-only opens; View and ViewValue in one snapshot, joining a transaction a ctx carries; VACUUM INTO backups. Each read connection keeps the 128 statements it ran last prepared.
  • writer: the single writer. Send, Do and DoValue queue commands, which run in batches of one transaction with a savepoint each, on statements prepared once; Once and OnceResult for requests a client retries; Flush; Exclusive for work between batches; checkpoints and PRAGMA optimize.
  • live: commits touch topics, topics mark views dirty, and each tick renders every dirty view once, in one read transaction per worker, for all the streams that watch it. Presence, Every, Watch and Unwatch, WriteTick, a stream cap, heartbeats, and Spec.Reconnect, which ends each stream with sse.Reconnect when the server stops, so that pages open their stream again on the next process.
  • sse: Datastar's events, formatted once and written to many streams; zstd, brotli or gzip per stream, with a deadline on every write; a memory budget that shrinks the streams' windows as more open; idle streams that give their encoder back (Hibernate).
  • edge: sessions, CSRF, the client's address behind a proxy, a CSP with a nonce, compression, precompressed static files, rate limiters, request logs, health checks.
  • tenant: one open database per account, refcounted, evicted at a soft cap.
  • dbmetrics: statement timing and a slow-query log through a wrapping driver.
  • loadtest: the writer and the render loop under load, in process, without HTTP.
  • backup (a module of its own): Litestream to S3, a restore when the file is missing, snapshots, a daily verify.
  • dbmetrics/prom and prom (modules of their own): the statements', the writer's, the hub's and the stream budget's numbers in Prometheus.

Defaults to know

  • synchronous=NORMAL: a power loss or a kernel crash can undo the commits of the last 35 seconds or so; a crash of the process loses none. sqlite.Options{Synchronous: "FULL"} syncs every batch.
  • The streams share a memory budget of half of GOMEMLIMIT, or of the cgroup's limit: a few streams compress with 512 KiB windows, many with 256 KiB, and the rest go out uncompressed, or not at all with Refuse.
  • An idle stream keeps its encoder until live.Options.Hibernate is set.
  • A page opens its stream with retry: 'error' from an element with an id, the one Spec.Reconnect names.

Tools

  • cmd/lith-new copies the template into a new module that requires the lith it ran from: a guestbook with tests, CI, a systemd unit, a Caddy block, a deploy script with rollback and a backup timer.
  • skills/lith: the rules for coding agents, with review commands, Datastar notes, recipes and performance numbers.
  • bench: the writer, render and stream benchmarks, with the commands to run them again (bench/RESULTS.md).

Website

lith.zweiundeins.gmbh runs on lith, from site/: the landing page, the docs (guides that follow one app, Datastar notes, performance, the package reference from the doc comments, a review checklist and this changelog), and eight live demos: a guestbook, a writer race, a stream inspector, a pixel canvas, a shared board, a cursor party, a Game of Life and a million checkboxes.