Web: loading large session history transfers the full event stream (25MB / 42k events; 54% is streaming chunks) — request WS compression and lazy loading

[0xK8oX]

Environment

• dsh 0.1.6-alpha.1 (npm registry), web profile, macOS, served behind a reverse proxy
• one long-running coding session from daily usage as the test case

Measurement (real session)

Metric	Value
Log on disk (zstd)	12 MB
Uncompressed event stream	22.5 MB
Events	41,794
Turns / assistant settlements	76 / 468

Event-type breakdown of the transferred payload:

type	count	bytes
reasoning-chunks	18,128	5.0 MB
tool-call-chunks	8,162	3.2 MB
assistant/chunk	6,133	3.1 MB
tool/code-dispatch	1,154	3.5 MB
assistant/message (settlements)	468	2.3 MB

54% of the transferred bytes are streaming chunk events that are superseded by the settlement events in the same log (settlements fully cover all 76 turns). Stripping chunk events yields 10.2 MB / 5,072 events with zero content loss.

Where the time goes

• Server-side read is not the bottleneck: decompress + JSON-parse of all 42k events ≈ 0.5 s; the projection cache is effective once warm.
• The cost is (a) transferring ~25 MB over the browser WebSocket, then (b) rendering 42k events in the DOM. Over a proxied HTTPS path this is a 10–40 s blocking wait for a single session open.

Feature requests

1. permessage-deflate on the session-stream WebSocket — the stream is JSON text and compresses ~3–5×; no protocol change needed.
2. Settlements-first history load: when a client opens a session for viewing, send settlements only (or a compact projection), and fetch chunk streams lazily only when the UI needs them.
3. Virtualized / paginated rendering for very long sessions in the web UI.

Happy to provide more traces. Thanks for the great harness — the session-log rebuildability and the 0.1.6 perf work are already a big step up.

[0xK8oX]

Follow-up data point from the same deployment: a single malformed session log makes the web session-list API fail for every workspace at once — listArtifacts() appears to throw on the first bad log instead of skipping it. We hit this when an experimental compacted copy violated the frame layout; the failure mode was "all workspaces show no sessions" until the file was removed.

Suggest: per-artifact try/catch with a warning (skip + surface the bad id), consistent with how optional plugin failures are isolated elsewhere in 0.1.6. This also matters for crash-orphaned logs (truncated at step/start) from unclean restarts, which we have also seen.