Skip to content

mkio 1.3.0

Choose a tag to compare

@markuskimius markuskimius released this 19 Sep 03:06
· 3 commits to main since this release

mkio 1.3.0 keeps a service's live updates flowing when one peer is slow or gone. A minor release: two config keys and one envelope field, all additive, with wire protocol 1.2 → 1.3. No change to existing config, commands, the expression language or the on-disk layout.

Fixed

  • A service listener sent each change to its subscribers in turn and awaited every send, and websockets had no heartbeat. One peer that stopped reading — a laptop asleep with a page open, a frozen tab — held up every other subscriber of that service once its socket buffers filled, which takes real traffic, so it showed up only after volume. When that peer then disconnected, on_unsubscribe replaced the subscriber list while the listener was suspended in the send, self._subscribers.remove(sub) raised ValueError against the new list, and nothing guarded the listener task. It died silently: snapshots kept working, but live updates never came again — for any subscriber, across page reloads — until the server was restarted. Seen from mkfix as a blotter that "stops updating" after many orders and fills.
  • Listeners in all three services now log what an event raises and go on; dead subscribers are pruned by identity against the current list.
  • Query subscribers are pending from before the snapshot read, so a change committed while the snapshot is read, sent or paged is buffered rather than lost; a buffered insert of a row the snapshot carried arrives as an update. Stream and SubPub register right after the snapshot send.
  • Query pagination state is found by (ws, subid): two pages that both name a subscription mkui-table-3 no longer clobber each other's page sequence.

Added

  • Per-connection outbox (OutboxWebSocket): sends queue on the connection and return, and one task per connection writes, so a listener never waits on a peer. Covers query, stream, subpub, monitors and custom services.
  • ws_send_buffer_mb (default 16): a backlog past it closes that connection with 1013 so the client takes a fresh snapshot. Frames over a sixteenth of the buffer — snapshots — are not counted, or a large snapshot would trip the limit on the first update behind it and the reconnect would loop.
  • ws_heartbeat_s (default 30, 0 = off) drops peers that stop answering.
  • A full change queue no longer drops silently: ChangeBus marks it and the service resyncs — SubPub and a Query over joined tables re-run their SQL and publish what differs; a plain Query and a Stream (buffer reloaded) reset their subscribers.
  • A reset is a nack with "code": "reset". The JS and Python clients answer it by subscribing again, three times in a row at most, instead of deleting the subscription.

See the README section "Slow peers, lost places and resets".

Install: pip install mkio==1.3.0 — https://pypi.org/project/mkio/1.3.0/