Skip to content

levain 0.8.0

Choose a tag to compare

@phillipclapham phillipclapham released this 11 Oct 22:36
· 4368 commits to main since this release

The cockpit's write side: K2a (one write route, the write token, writes bound to what was rendered) and K2b (the broker, which turns a write that needs more than the token into a consented, journalled fire). They ship together.

Added

  • The cockpit broker. A T2 or T3 write from the cockpit no longer fires inside the request: it is issued as a pending that holds a consent record (what will change, on which row and version, at which tier, and when the pending expires), decided in one transaction, then applied store by store and finished. Every step is journalled in cockpit.db under the install's cockpit data root, so a crash at any point is recovered at the next start and an apply that hit a transient fault is retried in the background, with a backoff that caps at two minutes. One broker owns a data root (an exclusive lock on cockpit.lock beside it); a second server on the same root waits in standby and its broker-backed reads say so.
  • Signing a pending. A pending waiting for a signature shows on its row with a cancel button. It is signed with an enrolled OpenSSH key over the record's exact bytes (ssh-keygen -Y sign -n levain-cockpit) and the signature is posted to /cockpit/pending/<id>/signature. An unsigned pending expires after 15 minutes, and each key holds a bounded number of waiting pendings.
  • levain cockpit enroll | revoke | keys manage a reported install's signing keys: a laptop-presence key signs T2 and T3, a display-phone key T2. The last laptop-presence key is revoked only with --compromised-since, and from that time on its signatures stop reading verified.
  • A reported install (levain serve --install-class reported) may fire a T2 write on the write token alone; the write is recorded as reported and the page labels it claims cockpit token. T3 always needs a signature. A governed install (a root-written marker) is not built yet: a governed marker disables the broker and every write route says so.
  • An episode can be tombstoned from the cockpit (the entity's own episodes; T2, irreversible, confirmed first), and the neocortex State section is editable again, written through anneal's own section writer on anneal's section version (anneal-memory 0.9.43's Store.replace_section), so an edit against a section another writer changed is refused.
  • Recent edits shows the broker's applied writes beside the ledger's, and the decisions panel shows the broker's waiting pendings.

Changed

  • One lock protocol for levain's two single-writer locks, and the install lock now fails closed. The install lock (.levain/install.lock) and the codex home lock (<CODEX_HOME>/levain.lock) are one flock code path. Where flock answers "not supported" (some SMB, WebDAV and old FUSE mounts), both now refuse before writing anything but the empty lock file, which a re-run treats as no content; set LEVAIN_UNLOCKED=1 to run without the lock when no other init or update writes that install or codex home at the same time. LEVAIN_CODEX_HOME_UNLOCKED=1 still unlocks the codex home lock, but the install lock is taken first, so it needs LEVAIN_UNLOCKED=1 too. Before this, the install lock went ahead unguarded there: reproduced on Linux with flock answering "not supported", 6 of 20 concurrent levain init --force pairs into one install failed mid-install; with this change all 20 refuse before writing. Every platform here means macOS and Linux, the two levain supports.
  • The lock holds across NFS clients, and a killed holder never leaves it stuck. Measured on an NFSv4.2 export with two clients: 19 of 20 concurrent init --force pairs had one run refused as busy and none failed, including with one client on levain 0.7.2. A holder killed by SIGTERM or SIGKILL released the lock at once: the kernel owns it, so there is no stale lock file to clear.
  • Every write to the web cockpit needs the write token, on loopback too. levain serve --write prints a token for this server run and a link that carries it (#write_token=... in the fragment, which the browser never sends anywhere); the page takes the token from the link and keeps it. A write with no token, or the wrong one, is refused 403 credential_required before its body is read. Measured on Linux: another local user (nobody) posting to a levain serve --write on the same machine got 403 on the cockpit write route, on every /edit kind and on /action. A server given its own tokens (make_server(surface_tokens=...), one per surface) refuses to start if two surfaces share one, or if a token is not printable ASCII without spaces. Under levain daemon install the token changes each time the daemon starts and is printed only to the daemon's log; install now says where that log is, and creates it (and the error log) owner-only, because systemd creates a missing append: log readable by other users (0664, measured on Ubuntu 24.04). On Linux levain's log directory (~/.local/state/levain) is made owner-only too, so a log deleted or rotated later is still private.
  • levain init --web needs its write token too. Another local user could post the onboarding form to 127.0.0.1 while init --web was running, and its answers became your new entity's seed (measured on Linux: user nobody, 200, the seed written). The command now prints a token for this run and a link that carries it in the URL fragment; the form sends it, and a post without it is refused 403 credential_required before anything is read or written. If the link opens without its token, the form asks for the printed one.
  • An off-box cockpit no longer shows "Chat is unavailable". The chat panel asks /chat.json whether the server has chat; a writable server bound off loopback (which never has chat) answered that with the read token's 403 instead of 404, so the panel showed an error instead of staying away. It now answers 404.
  • Writes go through one route, POST /cockpit/verb, and /edit and /action are aliases of it. The kernel decides each write's tier from the stored row, never from the caller: a C1 write (touch, snooze, a retier that does not park, a resolve, your state line) fires; a write that needs your signature or your laptop (T2, T3) is refused "needs the broker" until K2b. A panel offers only its own verbs, and a write names the row version or value version it was shown: a row changed since, or a value changed since, is refused 409 stale. For a spore, that compare runs inside anneal's own write transaction, so a change that lands between the server's read and the write is refused too.
  • /edit bodies now bind to what was rendered. An operator_state edit carries panel_version (the page sends the version its state line came with; the first line of a new install sends "absent"); a spore_descend carries the expect_disposition it was rendered with; an /edit on a spore that no Tray, Open loops or Keep panel shows is refused 409.
  • The state line has one lock. levain state, the TUI and the web cockpit take one lock on the context file (on its real path, so a symlinked file and its target share it); the cockpit's version check runs under it. Where the lock cannot be taken (a filesystem without working locks), the state line is not written and the command says why. A writer outside levain does not take it.
  • Requires anneal-memory 0.9.43 or later (the spore compare uses expected_version, first in 0.9.42; the cockpit broker is built and tested on 0.9.43).
  • Open loops comes before the Tray, and the Tray has a By age view: the same box, the same rows, oldest first, with each item's age. The page switches between the Tray's order and By age without re-sorting anything itself: the server sends both orders.
  • The Episodes panel can show other agents' episodes. An embedding server passes episode_feeds={name: read} to build_default_cockpit; each feed's rows appear with their agent, and the page gets an agent filter. A feed that fails, or sends rows that are not episodes, is named on the panel and shows nothing, and the entity's own episodes stay. Only the entity's own episodes can be tombstoned. The entity's own episodes are its newest few dozen, and the panel says so; other agents' rows are not cut to that window, so a busy entity never hides them.
  • The web dashboard draws itself from the cockpit manifest by default, and writes through it. The legacy page is /?source=substrate; its writes post /edit and /action, where a T2 write is refused because it fires only through the broker. On the manifest page every affordance posts /cockpit/verb with the row it was shown. A write that may rise above its rendered tier first asks the server for a preview (dry_run), shows the tier it rises to and what it does, and fires the exact request you saw only on a second click; a fire that names an intent must also name the previewed tier_seen and gesture_seen, and the server refuses 409 tier_moved / gesture_moved when either has changed. A T2 write in a reported install is labelled claims cockpit token. Each fire above C1 carries one idempotency key, kept for the tab, so a retry after a lost answer lands once. A pending your signature is waiting on shows on its row with a cancel button. The manifest names the server's write capability (transport: {writable, write_token_required}), and a resolve or promote action names the kinds its row's type takes (options). A row whose text holds characters the page cannot show faithfully (a zero-width space, a line separator) offers no inline edit; edit it with the CLI. Rename shows only where the cockpit offers it.
  • A spore's domain is editable (spore_update takes domain; the inline edit has a domain field). The broker refuses a type change on a Keep note or an open loop, as the legacy route does.
  • A downstream verb can fire through the broker. An embedding server passes make_server(extra_effects={verb: DownstreamEffect(store, canonical)}): an EffectStore and its canonical_effect for that verb, checked at construction. An ActionVerb may declare fields, which the manifest publishes and the route enforces.

Known limits

  • A filesystem with no lock support at all cannot be locked: there levain refuses unless LEVAIN_UNLOCKED=1 says no other writer runs.
  • An NFS mount with local_lock=flock or local_lock=all keeps the lock on that client, so two clients do not exclude each other.
  • Two NFS clients that present the same NFSv4 client identity (cloned machines or containers sharing a hostname and client identifier) are one client to the server, and their locks do not exclude each other; measured with two network namespaces sharing the host's identity.
  • On a Linux NFS client the lock belongs to the process, so two threads of one process that each take it do not exclude each other. No levain caller does that: the web init server runs one install at a time behind its own lock.
  • The TUI's spore writes are compared on the item's list (expect_disposition), not on its version.
  • Without flock (Windows), a version-bound write (the cockpit's state line and spore writes) is refused; an unchecked levain state writes as before.
  • A downstream verb with no extra_effects entry is T2 and has no broker effect, so it is refused (needs_broker on /action, not_built on /cockpit/verb). An embedding server that sent messages through /action before this release needs an effect store for that verb.
  • The preview binds the tier and the gesture, not the params; the page sends exactly the params it previewed.
  • A downstream verb declared job=True runs no job. The broker applies the verb's effect store once and never calls its handler; /action, the route that runs handlers, refuses a T2 downstream verb. Asynchronous downstream work has no path through the broker yet.
  • An embedding server's downstream write verb that worked through /action before this release (a message send, for example) is refused after the upgrade until that server passes an effect store for it (extra_effects). Upgrade the embedder and levain together.
  • Neither the page nor the CLI signs a pending yet: a T3 write, or a T2 write outside a reported install, waits until something posts an ssh-keygen -Y sign signature over its record. The full pending card comes with the governed install.
  • Erased means erased at the API. A tombstoned episode's content can remain in freed pages of the SQLite files and in anneal's tombstone record (a content hash), and a State section edit keeps the section's previous text in cockpit.db with no retention bound.
  • A write that touches more than one store takes no lock across them, and a store records no receipt per effect: the broker's journal is the record of what it applied.
  • In a reported install the write token's secret has no levain-owned file: a server mints a fresh token each start, and under levain daemon install it is printed only to the daemon's owner-only log.
  • An embedding server that calls make_server itself with neither write_token nor surface_tokens gets a fresh write token on each start, kept only on the server object (levain serve prints its own). Pass write_token to keep one across restarts.
  • One broker owns a data root: two servers over the same data root (two listeners for one install, say) leave the second in standby, so it fires no T2/T3 write and shows no pending rows. Run one server per data root.
  • While an embedder's own decision source cannot be read, the composed decisions panel shows that error and hides the broker's waiting pendings with it. A waiting pending still cannot fire unsigned, and it expires after 15 minutes.
  • The broker refuses to start when its data root or a registered store is on a network filesystem (NFS, SMB, FUSE), because its lock may not hold there; the cockpit's writes above C1 are then refused.