Skip to content

v0.2.0

Choose a tag to compare

@mbolli mbolli released this 14 Oct 15:39
· 277 commits to main since this release

Streams are measured by memory: the budget books what each stream holds, sizes its window from its frames, and with Refuse caps the streams instead of a count. No API was removed. What to check in an app:

  • A budget's Limit now covers each stream's encoder, 32 KiB besides and the frames the hub keeps, not only the encoders. The default, half the memory limit, needs no change; a Limit sized for encoders alone now lets fewer streams compress.

  • ZstdWindow and BrotliWindow are the largest window a stream may get: each stream takes the smallest window that holds its frames. MinWindow no longer defaults to 18; set it only for pages whose frames refer back further than one write.

  • With Refuse, a stream gets in only with room for a window that holds its frame, and its headers go out with its first frame. Open streams with retry: 'error' and leave live.Options.MaxStreams at its default, as a backstop.

  • A view that one stream watches, such as a tab's own, no longer keeps its frame: a stream that joins it gets a frame rendered in the next tick.

  • sse: ReadSignals reads a request's Datastar signals for any method: the datastar query parameter of GET, HEAD and DELETE, the JSON body of the others, @query's QUERY included. It caps the body at 1 MiB (ReadSignalsLimit for another cap), takes a request without signals as none, and reads the body to its end, so that over HTTP/1.1 a handler that waits before it answers still notices a client that left. Called after Open, it returns ErrBodyConsumed.

  • edge: Compress compresses the answers to QUERY as it does GET's.

  • sse: PatchElements and the other events split HTML without a carriage return with bytes.IndexByte: a 230 KB page takes 40 µs instead of 342.

  • sse: a budget books what its streams hold, not only their encoders: each stream's tier, 32 KiB for the rest of the stream (its state, its goroutines' stacks, net/http's buffers), and what Budget.Hold books, such as the frames a hub keeps. Limit, half the process's memory limit by default, now covers all of it; the other half stays for the rest of the app and for the garbage between collections. BudgetStats.Frames reports what Hold booked. Without Refuse, uncompressed streams may take the total beyond the limit.

  • sse: a stream's tier is picked at its first write, from its largest write: the window must hold that frame with a sixteenth to spare, or a frame cannot refer to the one before. A brotli stream takes that window and no larger one, at quality 4 while the budget stays at most half full, then at quality 3, and only when that does not fit, quality 3 at 2^16, which compresses each frame alone; it goes out uncompressed only when even that does not fit. A zstd stream takes the largest tier within its fair share, moves down no further than its frame's window, takes 2^15 only when that window does not fit, and moves up again at a write once room frees up. MinWindow no longer defaults to 18: unset, the frames decide; set, it is the smallest window a stream's frames need, for pages whose frames refer back further than one write. With a limit, an unset ZstdWindow or BrotliWindow lets the budget go up to MaxWindow. In 150 MiB, 114 brotli tabs with a 230 KB view each hold 2^18, and 168 zstd tabs. The tiers are booked from measurements at 50 to 230 KB: brotli's 7 to 31% lower, zstd's 32 to 48 KiB higher. Apps that set MinWindow to their largest frame can drop it.

  • sse: Stream.Upgrade returns a channel that the stream's budget closes when the stream holds a brotli window below its frames, or went out uncompressed, and a stream opened now would get a window that holds them: at most one stream a second. BudgetStats.Reopened counts them.

  • live: the hub books the frames it keeps in the streams' budget, once for all the streams that watch a view: a []byte frame at its length and 8 KiB, another by its Len() int method. A stream with Spec.Reconnect ends with sse.Reconnect when its budget has a better tier for it, and the page opens it again; the same stream ID at most once a minute, a stream without an ID after a minute.

  • live: the hub keeps a view's newest frame only while two or more streams watch it. Of a view that one stream watches, such as a tab's own, it keeps the frame's length and hash once the stream has the frame: a render that repeats it still goes to no stream, the budget books nothing for it, and a stream that joins the view gets a frame rendered in the next tick. That saves about 235 KiB a tab with a 233 KB view of its own, 34 MiB for 150 such tabs, and a budget with Refuse no longer goes past its limit by the frame of a tab that Open has just let in. A frame that is not []byte has no hash, and the hub keeps it as before. Each []byte frame is hashed once as it renders, about 16 µs for 233 KB.

  • sse: with Refuse, a budget lets a stream in only with room for 32 KiB and the smallest tier of its coding whose window holds its frame: Options.Frame, which live's Serve sets from the stream's first frames (unset, the coding's smallest tier). It also turns away a stream that compresses nothing (its client accepts no coding, or NoCompression is set) when its 32 KiB do not fit. A stream let in keeps a window that holds its frame until it closes: a zstd stream keeps that tier booked before its first write and while it hibernates, and moves no lower when the budget fills or a lower limit asks it to; a brotli stream takes no window below its frame. When the budget turns a stream away, the zstd streams above their frame's window move down to it at their next write, until the stream would fit. A stream whose frames outgrow its window later keeps its tier and sends more bytes a frame until it gets a larger one; it is never cut off. So no stream the budget lets in goes out uncompressed, in raw blocks or with a window below its frame, and memory caps the streams: an app with Refuse leaves live.Options.MaxStreams at its default as a backstop, and its pages open their streams with retry: 'error'. In 120 MiB, 134 zstd tabs with a 233 KB view each get in, all at 2^18, or 91 brotli tabs, and the others get a 503. Before, 150 MiB let in 175 zstd tabs with a 230 KB view, most of them at 2^15 to 2^17, where a frame took 4 to 35 KB instead of about 60 bytes.

  • live: Serve opens a stream once the first frames of its views are there, or the tick that renders them has ended, and gives their length to the budget (sse.Options.Frame), which with Refuse answers 503 when no window that holds them fits. The response's headers now go out with the first frame: on a hub that ticks every 50 ms, 24 ms later at the median, while the first frame comes as soon as before. A view that Watch gives a stream that the budget then turns away waits for the next stream with its ID, as it does while none is open.

  • prom: sse_budget_frames_bytes and sse_budget_reopened_total; the limit and the reservation cover what the streams hold.

  • sse: a brotli stream flushes its encoder every 16 KiB of a frame at quality 3 and every 32 KiB at 4. Without those flushes brotli extends a copy into the next input block without hashing it, so after a change in a large frame it lost the frame before and coded the rest in short pieces: a 230 KB board at quality 3 and 2^18 took 3,173 bytes a frame and now takes 259, a 230 KB data table that scrolls 5,756 and now 227 (quality 4: 2,382 and 140, 3,450 and 134). The encoder's heap no longer grows with the frame: at 230 KB, quality 3 at 2^18 held 1,554 KiB against the 1,312 its tier booked, and now holds 1,083; quality 4 at 2^18 2,370 against 2,192, now 1,603. Each flush costs a metablock header: a 230 KB list with 20 changes spread through it takes 676 bytes a frame instead of 363, and a 50 KB board at quality 4 50 instead of 34. The CPU per frame of a 230 KB frame changes by 13% less to 30% more.