Skip to content

Build Test and Release

Wijanarko Putra Rajeb edited this page Nov 18, 2026 · 13 revisions

Build, test and release internals

Build & Dev Loop

  • Development has ONE entry point; production has two builds. bun run dev runs src/server/index.ts, which imports index.html; Bun bundles that page's script, styles and assets and serves them from its own routing table (/_bun/asset/* and /_bun/client/*), so there is no client watcher, no dist/client to have been built first, and no second port. bun run build is different: the client is built from index.html with splitting: true and the server from src/server/index.ts with splitting: false and *.html external, because a single build cannot split the client without emitting the server's own chunks into dist/client/static/ — a path plugins/static.ts serves to anyone (measured: 325 chunks, 67-98 of them the server's). Production therefore reads the shell from dist/client/index.html instead of importing the HTML bundle; see the build bullet below.
  • HMR is real and Bun owns it. Bun.serve({ development: true }) injects its own client and pushes updates over import.meta.hot; saving a client file is applied in the browser, saving a server file restarts the process (bun run --hot). The dev-assets.ts proxy that forwarded rsbuild's hot-update chunks is gone with rsbuild.
  • Dev assets are proxied through the server, because Bun's own dev asset routes cannot be cached and that is what made every page load re-download the bundle. In development: true Bun answers /_bun/client/* and /_bun/asset/* from its routing table with an ETag it then ignores (If-None-Match → always 200 + full body), no Cache-Control, and no compression — measured on 1.4.2: a 19.7 MB unminified dev bundle was re-sent in full on every load, against 9.5 MB once in production. lib/assets/dev-assets.server.ts closes it: ssr.ts rewrites the shell's /_bun/client|asset/ urls to /_dev-assets/* (Bun's table wins for the paths it declares, so a route on /_bun/* is never reached — verified), and that module fetches the bytes back from the listener to add Cache-Control, a content ETag and the 304, plus the compression plugins/compress.ts already provides (19.7 MB → 3.3 MB zstd). /_bun/hmr (the WebSocket) and /_bun/unref (the beacon) deliberately stay on Bun. The rewrite runs BEFORE the bootstrap payload is injected, so a /_bun/ string inside a user's settings is untouched. no-cache, never immutable: the bundle's name is a build-generation hash that changes per rebuild, but the CSS asset names are NOT content-addressed — appending a rule to tailwind.css changed the bytes at the same /_bun/asset/<hash>.css URL — so immutable there would pin stale styles for a year.
  • The dev bundle is held by the service worker, not the HTTP cache, because the HTTP cache physically cannot hold it. Chrome declines to store a response whose decoded body reaches ~5 MB — measured with Cache-Control: no-cache and a working ETag: 1 MB and 4 MB answered 304 on the next load, while 5/6/7/8/16/20 MB were re-fetched in full and never sent If-None-Match. So the proxy alone still left the 19.7 MB bundle re-downloading every load. public/sw.js therefore caches /_dev-assets/client/* in Cache Storage (no per-entry limit; verified: the second load reports transfer: 0 and the cache holds one entry). It keeps ONE generation — storing a new bundle deletes the others, so a rebuild cannot accumulate bundles — and it deliberately does NOT cache /_dev-assets/asset/*, whose names survive a content change; those go to the network and revalidate through the proxy's ETag.
  • development is derived, not configured: Elysia's Bun adapter passes development: !isProduction to Bun.serve from NODE_ENV, so bun run dev and bun run start differ by that one variable and nothing else. The proxy and the shell rewrite are gated on the same variable (DEV_ASSETS_ENABLED), so production is untouched: its assets are content-hashed and already answered with a working 304 and immutable by Bun's table (runtime) or plugins/static.ts (AOT).
  • Production assets and the shell ARE compressed; Bun's runtime-prod chunks are not, and cannot be from here. maybeCompress reaches the API routes (route-adapter.ts), the dev proxy and plugins/static.ts (a built asset, keyed per ETag in lib/assets/compressed-cache.server.ts) and the SSR shell — the shell is no-store and served on every navigation, and /fonts.css is 4 KB on every load, so both were the repeated cost. Measured on this repo's own bundle: 10.5 MB → 2.26 MB zstd / 2.45 MB gzip / 2.16 MB brotli, one 20 ms pass per build (2 ms warm). A Vary: accept-encoding rides EVERY response of a compressible type, the 304 included — a shared cache that stored the identity variant would otherwise hand it to a client that accepts only br. The shell itself is now ~5 KB of markup: the two generated stylesheets (182 KB) are a content-hashed, immutable /theme-<hash>.css instead of inline <style> blocks.
  • A dev server walks toward the descriptor cliff, and the surface now says so before the breakage. development: true keeps the whole client module graph open for HMR — measured 3,545 descriptors after a single page load for this app, against 17 with development: false — and a macOS reload never closes the previous generation's (bun#40706; the pre-exec sweep exists only on Linux), so the count climbs with every rebuild (measured +2…+9 per edit, ~17/min on this repo while agents edit it). At Darwin's OPEN_MAX (10,240) posix_spawn starts answering EBADF: bad file descriptor, posix_spawn '/bin/sh' — verified A/B: 10,230 descriptors spawn fine, 10,240 fails — and from then on every spawn in the process fails at once: the PTY shell, the omp child, git, rg, until the instance is restarted. lib/lifecycle/fd-pressure.ts keeps the count visible: GET /api/health carries fds: { open, highest, limit, nearCliff } (a fcntl(F_GETFD) sweep capped at the cliff, ~1 ms, null where the host cannot answer) and kabmo status prints it with a restart hint. A long-lived session belongs on a production server: 17 descriptors, not 3.5k.
  • Process-wide state that owns a descriptor belongs on globalThis, not in a module binding. bun --hot re-evaluates a changed module in place, so a module-level cache is reset while its handle lives on and the old one is never closed. db.server.ts leaked a bun:sqlite handle per reload of that file — measured db.sqlite and db.sqlite-wal at 19 descriptors, +2 per edit, each one 2 descriptors closer to the cliff above. The slot is globalThis.__kabmoDb, for the same reason the terminal store, the flock host and the provider cache are anchored there.
  • lib/assets/conditional.server.ts owns the one "is the client's copy still good?" decision, because two routes hand out bytes and they must agree: plugins/static.ts (a file on disk) and lib/assets/dev-assets.server.ts (bytes fetched back through Bun's dev asset routes). A rule applied to one and not the other would make one asset kind revalidate differently from another for no reason a caller could see. The load-bearing part is precedence — RFC 9110 §13.1.3: a recipient MUST ignore If-Modified-Since when the request carries If-None-Match, so the presence of If-None-Match settles the response on its own and a non-match means "stale" (200), never a reason to consult the date. Every branch errs toward "not modified = false": a false costs one full response, a wrong true hands the client a stale asset with no way to notice. The file route mints a WEAK validator from size+mtime (W/"<size>-<mtimeSec>") — a content hash would mean reading every asset on every request, the opposite of the point, and a wrong 304 from size+mtime needs both a same-size edit and a preserved mtime, which no build or editor does. The mtime is floored to whole seconds before comparing: HTTP dates carry one-second resolution, so a file whose mtime has a fractional part would otherwise be permanently "newer" than the Last-Modified this server just sent for it and could never revalidate (measured: IMS on an unfloored mtime answered 200 every time). Verified end to end in both prod modes — /sw.js (the only max-age=3600 asset, so the only one that ever revalidates) now answers 304 to If-None-Match and to a fresh If-Modified-Since, and 200 + the new bytes once the file actually changes. The AOT bundle had no validator at all before this, so every hour each client re-downloaded it in full; runtime mode already had one from Bun.
  • A unit suite's temp directory is the suite's to remove, and afterAll is the only hook that actually runs. bun test does NOT fire a process.on('exit', …) a test file registers (measured: the marker file never appeared), so src/server/lib/btw/schema.server.test.ts's exit hook had never deleted anything — three .sqlite + -wal + -shm triples per run, forever. The measured leak inventory for one full bun test in an isolated TMPDIR, before and after: 30 entries / 3 GB-scale accumulation → 2 entries. The shapes that leaked, each now fixed: an array of mkdtempSync roots collected and never swept (fs-path.test.ts, 5 dirs); removing the CHILD of a mkdtemp instead of the parent (plugins/static.test.ts removed public/, plugins/build-server.test.ts removed the copied plugin — 3 empty parents); path.join(roots[0] ?? tmpdir(), …) falling through to $TMPDIR itself once afterEach had emptied the array (db.server.test.ts's blocker files); and nine inline mkdtempSync calls in one file with no teardown at all (marketplace/plugins/command-runner/src/server/resolve.test.ts). The audit command is TMPDIR=/tmp/audit bun test && ls /tmp/audit — a fresh $TMPDIR is the only way to see the leak, because the developer's own $TMPDIR is already full of it. Two entries are expected and correct: kabmo-test-bins (the compiled-fixture cache, pruned by age) and ripgrep-wasm-<hash>.wasm (the ripgrep package's own cache, keyed by its wasm hash).
  • bun run build (scripts/build-client.ts) runs TWO builds into dist/client: the client from index.html with splitting: true and minChunkSize: 16384, then the server from src/server/index.ts with splitting: false and external: ['*.html']. The server bundle is what kabmo serve executes (from dist/client as its cwd); the client bundle is what the shell links. Why split: without it every import() is inlined into one entry — measured 13.40 MB raw / 3.26 MB gzip, with @shikijs/langs 3.07 MB, @shikijs/themes 1.26 MB, mermaid 1.60 MB and xlsx 0.48 MB all on the boot path, which defeats the lazy grammar/theme design in shiki-langs.ts. With it the boot chunk is 1.90 MB raw / 590 KB gzip and 143 dynamic chunks resolve over HTTP (verified 143/143 200). Why two builds: one build with splitting: true also splits the server, and its chunks land under dist/client/static/ — publicly served, and /static/ is auth-exempt. Bun.build via the JS API, never bun build: the CLI does not read bunfig.toml's [serve.static] plugins (verified: bun build --target=bun --production emitted Tailwind's @theme verbatim, zero utility classes).
  • bunfig.toml's [serve.static].plugins is the dev and prod plugin list. One entry, src/server/lib/bundler/css.ts, which expands Tailwind v4 (@theme/@import "tailwindcss" are PostCSS syntax Bun's CSS parser rejects), flattens nested at-rules (nested-at-rules.ts, registered after Tailwind), and lifts @font-face blocks out of the bundle. scripts/build-client.ts passes that same module to Bun.build explicitly.
  • That plugin makes the Tailwind toolchain a RUNTIME dependency, not build tooling. Bun.serve loads it whenever an HTML route is bundled — every kabmo serve --dev and every bun run start — so @tailwindcss/postcss, postcss and tailwindcss are all declared in dependencies. They were not: @tailwindcss/postcss was a devDependency and postcss was declared nowhere, which no local run could reveal (a checkout has the whole tree) but a published install cannot load. Measured on a production-only install of the published tarball: the server came up and answered 503 on every page, logging error: Failed to load plugins for Bun.serve: Cannot find package 'postcss'; bun add -g failed one dependency later on @tailwindcss/postcss. tailwindcss itself is direct rather than transitive because @import "tailwindcss" in src/client/tailwind.css is resolved by Tailwind's own resolver from the project root — verified by forcing the package out of the root (nested under @tailwindcss/node only): the bundle then fails with an enhanced-resolve error. src/server/lib/bundler/dependencies.test.ts walks every bare import in src/ and fails if one is missing from dependencies (a devDependency counts as missing), which is what keeps a build-shaped module from drifting back into the build-only list.
  • Nested at-rules must be flattened before Bun sees them, because Bun's minifier re-nests them into a form Chrome drops. For target: 'bun' (which this build needs — its entrypoint is the server) Bun emits a rule's nested at-rule as @supports (…) { & { … } }, and a nested rule whose parent selector names a pseudo-element is inert in Chrome: & under ::placeholder or ::-webkit-scrollbar-thumb never applies, so the rule's earlier fallback declaration wins. Tailwind emits its colour fallbacks in exactly that shape (color: var(--x); @supports (color: color-mix(…)) { color: color-mix(…) }), so the whole app's placeholder and scrollbar colours collapsed to full ink — and the same shape also cost --ink-secondary/--ink-muted/--ink-hairline, --theme-info-hover/--theme-warning-hover, and the .prose-content/.shiki .find-match refinements. flattenNestedAtRules (nested-at-rules.ts) hoists each nested at-rule out of its rule with the parent selector resolved, preserving declaration order and the containing @layer, so the emitted CSS is @supports (…) { .sel::placeholder { … } } at the top level and no minifier has anything left to re-nest. The rsbuild pipeline this replaced compiled for browsers and flattened the nesting itself, which is why the migration surfaced it. css.test.ts builds a fixture through the real plugin with the production options and fails if & survives — asserting on the flattener's own output would not catch it.
  • Font faces cannot be bundled. Bun's CSS loader resolves EVERY local url(): with file it inlines the font as base64 (592 kB stylesheet, ~377 kB gzipped, every face downloaded whether or not a formula renders) and with dataurl it writes the ABSOLUTE FILESYSTEM PATH into the CSS. A same-origin path is not an escape hatch — url("/fonts/x.woff2") fails the build with "Could not resolve"; measured against Bun 1.4.2, only data:, http(s): and protocol-relative urls are left alone. So the faces leave the bundle: the plugin rewrites them to /fonts/<basename> and strips them from the module, and the server re-declares them on /fonts.css by reading the same source stylesheets under src/ on every request (lib/assets/font-faces.ts holds the extraction, shared by both ends) while lib/assets/fonts.server.ts resolves each basename back to the installed package. The server derives the sheet from source rather than from a directory the bundler wrote, because that artifact is produced once per process and can never be repaired — an rm -rf while the server runs, or a fresh clone that never built, left /fonts.css empty for the rest of the process and the terminal's Nerd symbols face silently vanished (tofu boxes, no error). The shell links it through app-head — a <link href="/fonts.css"> in index.html would be resolved by the HTML loader and fail the same way.
  • index.html's own asset references are relative (./public/icon.svg), because the HTML loader resolves them and emits hashed copies; an absolute /icon.svg fails the whole build. The manifest's OWN icon entries stay absolute (/icon.svg): Bun does not rewrite inside a copied JSON file, and public/ is served directly.
  • public/sw.js is served by a route, not by the bundler — it is referenced only from an inline <script>, which the HTML loader does not scan, so it is never copied. tryServeStatic answers it from public/.
  • Asset roots are resolved from the package, not the cwd (lib/assets/fonts.server.ts walks up from import.meta.dir). An installed package runs the AOT bundle with dist/client as its cwd, where a cwd-relative public/ or node_modules/ does not exist. Two questions, two resolvers, and they are NOT the same directory in every install shape: packageRoot() is the first root holding node_modules (where the DEPENDENCIES are — fonts are served from there), while packageDir() is the first root holding src/ (where the PACKAGE is — the source stylesheets /fonts.css is assembled from live there). A hoisted install puts the package under node_modules/kabmo/ and its dependencies in the parent node_modules/, so packageRoot() is the PARENT — resolving src/ through it found nothing, served an empty /fonts.css, and silently dropped every @font-face (the Nerd symbols face among them). Verified against a simulated hoisted published install: empty sheet through packageRoot(), full 4064-byte sheet through packageDir().
  • The shell is a Bun HTML route in development and a built file in production. Development fetches the server's own /_shell route, because Bun renders an HTML route only while serving and there is no in-process API for an HTMLBundle — new Response(bundle) yields "[object HTMLBundle]" and app.handle('/_shell') answers 404, because the route lives in Bun.serve's routing table rather than Elysia's. Production reads dist/client/index.html once and caches it for the process (a rebuild needs a restart, exactly as Bun's cached route did), which also removes a loopback HTTP request and a full HTML transfer from every navigation. The import is dynamic and guarded (lib/lifecycle/shell-entry.server.ts) so the server build can mark *.html external; a static import would bundle the client graph into the server and production would resolve an HTML bundle it never serves.
  • A shell that would not render reports the EDIT that broke it, and a 200 is not proof it rendered. renderShell used to check only !response.ok, so a failed client bundle surfaced as "Shell route answered 500." — while the file, line and message sat inside Bun's own "Build Failed" page as a binary payload (Uint8Array.from(atob(…))) with no server-side reader. lib/bundler/build-errors.server.ts obtains them the supported way instead: after a failure, it rebuilds the same entrypoint with Bun.build({ target: 'browser', throw: false }) and the plugins bunfig.toml names, and formats each log.position as file:line:column message plus the offending source line. The probe is not a subset of the route's failures, deliberately: a missing named export is answered 200 by the route while Bun swaps the bundle for a 104-byte stub that only calls location.reload(), so the page cannot run and is reported as the real failure it is; an unresolved import that tree-shaking drops fails nothing and stays silent (verified against the route on Bun 1.4.2 — both 500 shapes reported, both 200 shapes handled as above). The check is on the MARKUP, not the status: with development: false Bun caches the HTML route's output, and a build that failed on the FIRST request leaves it caching an empty body answered 200 for every later request — measured req1 500 len=0, req2..n 200 len=0, still empty after the source was fixed, because production never rebuilds that cached output. SHELL_MARKER (<!--app-bootstrap-->, shared with the SSR route's substitution) is what distinguishes real markup from a cached failure; when the bundle builds cleanly but the markup is not the shell, the page says the fix is a RESTART, because that is the only thing that rebuilds it. Both the reason and the marker travel through the same 503 page, HTML-escaped — the reason carries arbitrary source lines, and a < in one would truncate the page at the first tag.
  • serve.routes and Elysia's routes coexist: new Elysia({ serve: { routes: { '/_shell': shell } } }) — Bun's table wins for the paths it declares, everything else reaches app.fetch, and Elysia's websocket handler is composed alongside.

Browser E2E Layer (tests/e2e/)

  • The layer above bun test, and the only one that sees the assembled product. Unit and component tests mount a tree in happy-dom; route suites call handlers in-process. Neither sees the SSR shell, the client bundle through Bun's asset routes, the realtime socket, a spawned omp child, or a real panel width. Every bug that reached a user this cycle lived there — a stream indicator that died 450 ms in, a toolbar clipped at 296px, a wiki link that opened a tab. The specs are the regression net for that class.

  • The MOCK update popup opens over the app on every boot, and a spec must dismiss it — with a locator HANDLER, not a click. MOCK=true answers "an update is available" (MOCK_CURRENT_VERSION/MOCK_LATEST_VERSION in lib/updates/check.ts), so the Update dialog's fixed inset-0 backdrop swallows every click. The check is ASYNCHRONOUS, so a click issued at domcontentloaded lands before the dialog exists and the later popup then blocks the next action — the flake this pins. fixtures/app.ts's openApp/installOverlayHandler register a page.addLocatorHandler on the popup's "Later" button, which fires whenever it appears (and again on a reload). Do not "fix" this by disabling the popup or by sleeping.

  • A spec waits on a CONDITION, never a duration, and the condition must be the one the action needs. Three real flakes were found and pinned this way: a send issued before the sidebar's folder list loaded had no cwd, fell to the mock path, and rendered nothing (the folder's name on screen is the gate); a reload issued before the 600 ms session-state debounce landed read the previous layout back (page.waitForResponse on the POST is the gate); and keystrokes typed as soon as [data-terminal-root] was visible were written to a socket with no shell behind them and lost (the ready frame on the terminal socket is the gate). A waitForTimeout would have been too short on a loaded runner and wasted time on an idle one, in all three cases.

  • A gate must be the thing the action needs, not a state the app is free to skip — and an absent state is not a gate. editor-scroll.e2e.ts's reload spec waited for the editor to read Select a file to open before reopening its file. But the tab list is itself persisted (layout.openedFiles), so whether the panel draws that placeholder or the file it already had open depends on the order the restored blob lands in — a race the spec does not own (and the root-change wipe is gone: useFileTabs now closes tabs only on a genuine root CHANGE, never on the first known root or a transient null). Measured on the failing CI leg: the placeholder was never drawn and the assertion timed out for 15s while the panel was already correct, and the spec then passed on retry. The session-state GET is the gate (the blob is what the click needs); reopening a file that already has a tab reuses that tab, so the placeholder adds nothing.

  • A "platform limit" is a claim to verify, not a reason to skip. Both terminal-capacity.e2e.ts specs and terminal.e2e.ts skipped Windows on the belief that the PTY child could not stay alive there. It could: the cause was Kabmo's own spawn (detached on Windows = DETACHED_PROCESS = no console, so cmd.exe exited 1 immediately), and once fixed all three specs pass on that leg — the ConPTY byte-difference the terminal.e2e.ts skip also cited never mattered, because neither of its assertions compares bytes (a prompt character class and a unique marker). The cap's WINDOWS BRANCH stays covered independently by store-liveness.test.ts, which injects platform: 'win32'. A skip that outlives its cause is worse than no skip: it reports the platform as unverifiable and hides the product bug that made it look that way. Before adding one, reproduce the failure against the real spawn and name the exact option/API that causes it. The same file now carries ONE skip that passed that test, and it names its two inputs: a stream can be scrolled and selected from types a POSIX sh loop into the shell and copies with Meta+C. On Windows the shell is cmd.exe, so clear resolved to Git-for-Windows' clear.exe, printed its usage, and left the while loop as ARGUMENTS — the buffer never grew past one screen, the wheel had nothing to scroll into and Latest never appeared (measured on shard 4/4 of run 43112230252, with the screenshot in that run's artifact); copy is Meta+C only in ghostty-web's dist/ghostty-web.js, a chord Windows does not deliver. Revisit when the spec streams through PowerShell or a terminal copy chord that is not macOS-only — and note the product gap the skip exposes: a Windows desktop has no keyboard copy in the terminal, because TerminalKeyBar renders only on touch.

  • Assert the observable that actually changes, not the control you clicked. The editor's saved state is an icon swap and its aria-label does not move, so the spec asserts the FILE ON DISK; the terminal paints to a canvas with no text nodes, so the terminal spec asserts the terminal socket's frames — and, where the claim is about paint, the canvas pixels (the cell padding, the DOM caret), never the wrapper's class list.

  • A dialog's state is initialized by MOUNTING it fresh, never by a reset effect — and this spec is what found that. NewWorkspaceModal kept one mounted form and cleared its fields in a useEffect keyed on isOpen; Preact runs effects AFTER paint, so the dialog was on screen and its fields focusable before the reset landed, and anything written in that window was discarded. Measured through this layer: a path typed into the field was wiped and the listing stayed on the home directory; the same bug in DirectoryList's new-folder field left it empty with the submit button disabled. The fix is structural — if (!isOpen) return null and mount the form — so there is no ordering question to get wrong. The related lesson is that fill() is a legitimate way to find this: it writes synchronously after the element exists, which is exactly the window an effect-based reset loses.

  • getByRole(name:) matches a SUBSTRING, so a fixture's locator handler can click the wrong thing. installOverlayHandler addresses the update popup's "Later" button; a spec that put a row named appeared-later on screen had that row matched and CLICKED by the handler, which navigated the dialog into the folder and made the row vanish — a failure that looked like a broken refresh. The handler's locator carries exact: true. Check every locator used in addLocatorHandler (and any handler that fires on a page-wide match) for this: the popup's own control is exactly "Later".

  • Two runners, disjoint globs, on purpose. bun test owns *.spec.ts; Playwright owns *.e2e.ts (testMatch in tests/e2e/playwright.config.ts). The first version of this layer used .spec.ts and bun test imported the Playwright file into its own process, where test.describe throws "Playwright Test did not expect test.describe() to be called here". Never let the globs overlap — neither runner can be told to ignore a file the other owns without also hiding a real suite.

  • The runner is Playwright's own, in tests/e2e/, never bun test. A browser spec needs a per-file context, a per-file server and a trace artifact — a lifecycle Bun's runner does not own, and one that fights its single-process, shared-globalThis model (the reason test-support/isolated-db.ts exists).

  • fixtures/server.ts boots a REAL server subprocess, not an in-process Elysia app, because the thing under test is the assembled product. It is runtime-agnostic (node:child_process, node:net, globalThis.fetch) — it runs in a Playwright WORKER, which is Node, so a Bun.spawn there is a ReferenceError.

  • Isolation is the contract, and it is why getDatabasePath was fixed. Every path the server writes is redirected under one temp root removed at teardown: KABMO_DATA_DIR, KABMO_DB_PATH, PI_CODING_AGENT_DIR, plus SYNC_WORKSPACE=false. getDatabasePath put the MOCK branch BEFORE the KABMO_DB_PATH check, so a MOCK=true server started from a checkout wrote the repository's own workspace.db no matter what the caller set — verified, and pinned by src/server/db.server.test.ts. An explicit override now wins over BOTH defaults. Do not reorder those branches; a test that leaks into ~/.kabmo or the checkout's workspace.db corrupts the developer's machine.

  • Readiness is polled on GET /api/health, and the PORT is compared, never parsed from the boot log. The route is a stable shape (ok, mock, port); a log line is prose a future edit may reword. Comparing the port is what stops a stale instance or a lost port race from being adopted silently.

  • fixtures/fake-omp.ts is the keystone, and it is a real protocol CLIENT, not a mock. It speaks omp's NDJSON wire (ready, {type:'response'} with command/success/data, and the session event frames), so the code under test is the real spawn path, frame reader, frame fold, realtime topic and renderer. A double that stubbed any of those would test the double — the same argument test-support/realtime-server.ts makes. It honors the argv Kabmo actually builds (--mode rpc-ui --cwd … --resume … --approval-mode …) and prompts select behaviour: delay:<ms>, tool:<name>, edit:<pairs>, ask:<text>, error:<text>, retry:<n>, slow, default Response to: <prompt>. edit:<pairs> emits a real edit card whose details.diff is omp's EXCERPT dialect of that many removed/added PAIRS — every row a change, because the card renders contentMode: 'diff' and a patch of context lines collapses back under the 288px ceiling, where the scroller's range is 0 and a spec built on it proves nothing (measured). It is the only way a browser spec can observe the diff card's own scroller.

  • The fake must emit the FULL accumulated message_update frames Kabmo requests. configureChild sends set_event_filter {messageUpdates:"full"}, so every message_update carries the whole message again — that is what the client folds. The fake re-sends the accumulated content on each delta (and the tool-call fixture re-sends its block with the name/id), so a spec observes the same shapes a real child sends.

  • A builtin that opens no turn must be acked agentInvoked:false, and the ack must be the ONLY response. The wrapper reads agentInvoked:true as "this dispatch claimed the turn slot" and holds promptRunning until the agent-start watchdog fires; the prompt_result {agentInvoked:false} that follows is then suppressed by the run-attribution rule in frame-fold.ts (see Chat-Stream-and-Tools), so nothing clears the flag early. Measured on the release run of 2026-11-18 (44249018348): the fake acked every prompt true before dispatching on the message, so the client's /kabmo-mode plan-review republish left the session reporting busy for ~10 s after the answer landed — rename.e2e.ts was refused with "This session is working" on a session that had finished, 4 of 10 shards red, and 1 flake in 6 locally. /rename and /kabmo-mode now answer false and nothing else; the dispatcher's own decline path emits the settle frame when it owns the slot.

  • FAKE_OMP_RECORD_PATH appends every RECEIVED frame as JSONL, so a spec can assert what Kabmo sent (that the event filter was requested, that a reload reached a live child) rather than inferring it.

  • Two modes, chosen per spec. mock uses the seeded demo presets and needs no omp install (fast, no fake). real runs MOCK=false with KABMO_OMP_BIN pointed at a shim that execs fake-omp.ts under Bun — the shim exists because the server spawns the path directly, so it must be executable. There is no model and no network in either mode.

  • Fixture data is created through the REAL API (fixtures/seed.ts → POST /api/folders), never by writing the database. A hand-written row skips the route that resolves project_path and the scan the sidebar reads, so it tests a state the app cannot produce.

  • Every temp directory is REGISTERED the moment it exists, because the per-spec teardown is not the whole story. fixtures/temp-root.ts owns the registry and is the only place this layer mints a temp dir (makeTempDir for test scope, makeProcessTempDir for the compiled fake omp, which outlives the test that built it). Three shapes leak without it, and each was measured: a spec that FAILS or is retried mid-body skips every dispose() at the end of that body (a probe spec seeding a workspace and a working repo left 2 + 2 directories on each of its two attempts), a Ctrl-C kills the worker before any teardown runs, and a SIGKILLed worker runs nothing at all. So the registry has two scopes and one sweep: the _tempRoots auto fixture opens a test scope before every body and sweeps it after — which is the failure path a dispose() cannot reach — and process exit plus SIGINT/SIGTERM sweep the process scope. auto: true is what makes the ORDER right (an automatic fixture tears down after the spec's own), so server.stop() has already released the directory handles; a workspace the server still had open would fail to delete on Windows. HarnessServer.stop() still removes its own run root and then calls untrackTempDir, and its removals are individually try/caught — a fixture teardown that throws is reported as the SPEC's failure. The one case no in-worker hook can reach is a SIGKILL, and that is globalSetup's job: sweepStaleTempDirs() runs once before the first worker and deletes omc-e2e-* directories older than two hours (mtime, not birthtime — a directory keeps being written while its spec runs), which is what stops the accumulation. Measured on this machine before the fix: 964 directories and 1.3 GB in $TMPDIR over four days, dominated by 310 seeded workspaces and 369 run roots; after it, a passing run, a failing run and a Ctrl-C'd run each leave 0. sweepStaleTempDirs takes an injectable root because it deletes BY NAME — the first version of its unit test ran against the real $TMPDIR and reclaimed the developer's 964 directories, which is exactly the hazard the option removes. fixtures/temp-root.test.ts covers both scopes and every sweep rule against a private root, and tests/e2e/tsconfig.json excludes **/*.test.ts because those files import bun:test while the program deliberately has types: ["node"] only (adding bun-types there would also let a SPEC reach Bun.*, a ReferenceError in a Playwright worker).

  • The compiled-fixture cache is NOT a temp root: it is a shared, deliberately persistent directory, and its key is a hash of the stub's SOURCE. src/test-support/executable-stub.ts compiles a stub to a real executable (Bun.build({compile:true}), ~90 MB on disk because the Bun runtime is embedded) into $TMPDIR/kabmo-test-bins/<label>-<sha256(source)>, because libuv's search_path starts only .com/.exe and a #!/bin/sh fixture is unspawnable on Windows. Two consequences, both measured: a source that embeds a PER-RUN value — a mkdtempSync path baked in with JSON.stringify — misses the cache on every run and leaves the previous binary forever (bun test grew the directory by ~200 MB per run from two suites, reaching 40 GB before anyone noticed; a version answer baked per case did the same for the omp --version stub), and a source that does NOT is compiled once and reused indefinitely (a full bun test now leaves 12 directories / 1.2 GB on the first run and the same 12 / 1.2 GB on the second — verified). Pass per-run values through the environment instead: every spawn site that runs a fixture hands the child an explicit env built from Bun.env at spawn time (childEnv()), so a Bun.env.X = … assignment made in-process DOES reach the child — the claim in the old comment, that Bun.spawn only ever sees the STARTUP environ, holds only for a spawn that passes no env at all (verified on Bun 1.4.4: runtime-set visible with {...process.env}, invisible without). The sweeper for it is src/test-support/stub-cache.ts (pruneStubCache, a week by mtime, node-only so both layers can import it): executableStub runs it once per process, and the E2E globalSetup runs it too — sweepStaleTempDirs deletes by the omc-e2e- name prefix, so it can never reach this directory, and a stale binary from a changed source would otherwise sit there for good.

  • data-testid only where no stable accessible name exists. The preference is getByRole + accessible name (the toolbars already carry title/aria-label on every icon-only button), then visible text, then a testid for a container to scope a query, a virtualized row, or a canvas. Measured: src/ had ZERO testids before this layer; the set added is app-root (both layouts) and composer-input. Message rows already carry id={msg.id}, so a spec addresses a row by its id rather than by a new attribute. Add a testid as part of the spec that needs it, with a stable domain name — never speculatively, never layout detail.

  • The E2E project is typechecked separately (bun run typecheck:e2e → tsc -p tests/e2e) because it runs under Node, not Bun, and the root tsconfig covers only src/ and packages/. ci.yml runs this gate so a broken spec fails the cheap job, not only the browser one.

  • The browser layer runs on a pull request only when the e2e label is applied, plus nightly and manual dispatch (.github/workflows/e2e.yml). A push to main runs the SAME suite inside release.yml, whose e2e job calls e2e-specs.yml and needs: it before anything is tagged — so the merge is verified without a second workflow, and the label stays the cheap-PR escape hatch. Measured on 26333b1, a push: trigger on e2e.yml made every merge run the matrix twice (E2E 25m17s beside E2E specs / Chromium in the Release run), and the duplicate pair is a second chance for a flake to read as a red main. The nightly cron still covers main's tip when nothing has been pushed. Chromium is required; firefox/webkit are opt-in projects not yet run in CI. Failure artifacts (trace, screenshot, HTML report) are uploaded only on failure.

  • The browser suite is SHARDED across runners, and workers is not the lever. Measured on the release run of 2026-11-11 (42955898928): 33m02s wall, of which the gate was 28m45s and E2E specs / Chromium (Windows) alone was 28m45s — 87% of the run, with every other job under 8 min. The suite is 163 tests at workers: 1; raising it to 2 bought 12% (1.7m → 1.5m over 21 tests) and made two specs flake, because two servers and two browsers contend for one CPU and the assertions are timing-sensitive. A shard is a separate MACHINE, so it keeps the one-worker-per-machine property the config depends on and scales linearly: 3 shards on Linux (60/53/50 tests) and 4 on Windows (41/48/36/38). Windows is the bound of the run at ~7 min; the Linux leg lands at ~5 min, so it is no longer sized to finish together with Windows. --shard balances by TEST COUNT, not duration, so the shards are not exactly equal — the job timeouts are ~3x the expected shard, which is still a hang detector. fail-fast: false is load-bearing: a red shard must not cancel the others, because every shard's failure artifacts are what a reader needs.

  • bun test --shard is NOT usable on this suite yet, and the reason is a real bug. Measured 2026-11-12: --shard=1/2 failed 6 cases in src/server/lib/sessions/spawn.server.test.ts with UNIQUE constraint failed: workspace_folders.id, --shard=2/2 failed 3 more, while the unsharded run was 11259 pass / 0 fail. The file passes alone and passes in the full run, so the cause is cross-file env leakage: an earlier file leaves PI_CODING_AGENT_DIR (or SYNC_WORKSPACE) set, getDb() then seeds workspace folders from discovery into the database the next file expects to be empty, and the seed takes id 1. Sharding only changes which file runs before it. Fix the leak before sharding the unit gate — and note it buys nothing today anyway, because the browser shards land at ~7 min and the Windows unit job is 6m36s.

  • A suite that installs a DOM window puts it back through test-support/pristine-globals.ts, and Event travels WITH EventTarget. The Linux leg of release run 43379328083 failed 15 cases across files that could not have caused them. terminal-panel/scroll-hold.test.ts died with Argument 1 ('event') to EventTarget.dispatchEvent must be an instance of Event because workspace/editor/plugin-rail.test.tsx installed a happy-dom window per case and never restored it, and hooks/ui/ui-listeners.test.ts installed eight globals in beforeAll while its own comment claimed a matching afterAll that did not exist. Bun discovers files from the filesystem, so which suite follows changes between checkouts — such a leak is latent until an unrelated file is added. The shared helper captures the runner's values on the FIRST install; a local save-and-restore loop captures whatever the previous file left behind and then restores the leak. EventTarget is in its list for the same reason Event is: a target from one realm refuses an event from another, so installing half the pair turns any later dispatchEvent into a TypeError. fetch is on that list too, and a stub that the TEST settles is the worst kind to leave behind. packages/kabmo-ui/src/fetch.test.tsx replaced globalThis.fetch with one that returns a promise its own case resolves later, and its afterEach restored the DOM globals but not fetch. Every later file that made a real request then awaited a promise nothing would ever settle: the Linux gate's four src/cli/lib/commands/session.test.js cases each died at bun's 5000ms timeout, in a suite that names neither the installer nor the component. Reproduced deterministically by running the two files in one process (bun test packages/kabmo-ui/src/fetch.test.tsx src/cli/lib/commands/session.test.js — 4 pass, 4 fail before; 8 pass after). Restore through Bun.fetch, never a captured value: the handle is the runner's own whatever ran before. The same run-wide sharing is why a suite that mints a PERSISTED slice id must mint it process-wide — editor/find-harness.test-util.ts keyed its session id on a per-file counter, so the second file's test-session-2 read the first file's persisted alpha back and the "an empty selection seeds nothing" case failed with Received: "alpha" on the Ubuntu gate; the counter now lives on globalThis. The same lesson applies to a helper that SPAWNS something: assert the state the code under test actually reads, not the state you asked for. test-support/peer-harness.ts waited for the session lease to be held and returned, but the ownership guard matches the holder's PARENT against an instance record and the sh -c wrapper reparents it a moment LATER — so the record was written with a guessed pid: 1 and the Linux gate (43568904108) answered locally with ownerCalls === []. The helper now waits for the reparenting and returns { pid, parentPid }, and each caller writes the record with the pid it got back.

  • A module-level client cache is voided together with its in-flight read. fetchModelsData keeps inFlight to dedupe concurrent callers, and invalidateModelsCache used to clear only cached — so a read started before an invalidation was handed to a caller that invalidated on purpose (the stale-picker bug it exists to prevent), and a read that never settled owned the slot for the rest of the process. Measured on the same run's Linux leg: five cases timed out at the runner's 5 s budget (models.test.ts, provider-ui.test.ts) and three more waitForDoms gave up with a healthy stub installed, one file after another. The fix is a generation counter — an invalidation bumps it and drops inFlight, and only a read from the current generation may write the cache — pinned by models.test.ts's "an invalidation also retires a read that is still in flight".

  • A preact tree that is DETACHED is still subscribed, and its next re-render happens after the window is gone. Release run 44213261035's Windows shard failed 8 cases in client/lib/plugins/list-slots.test.ts with ReferenceError: document is not defined inside preact's flush — a file that mounts nothing. The cause was one file earlier in the shard: chat-timeline/MessageList.test.ts mounted two MessageList trees per test and only dropped the container, so AiMessageFooter's plugin strip kept its useListSlotRows('messageAction') listener alive; the first registerListSlot in the next file published, preact queued an update, and the flush read document after afterAll had removed it. The same tree never failed on macOS, where the deferred flush landed inside a test. Unmount (render(null, container)) in afterEach and put the globals back through pristine-globals.ts; a 0 unmount count in a file that calls render() is the shape to grep for. Order-dependent failures name the file that SURFACED them, not the file that leaked.

Route Method Contract

  • Route modules keep the ported Remix shape: an exported loader (GET) and/or action (mutating verbs). action owns method dispatch — it branches on request.method and returns its own 405 { error: 'Method not allowed' }.
  • Because of that, bindingsFor/actionBindings mount action on all four mutating verbs, and add a 405 GET fallback when the path has no loader. Registering only the "supported" verb would turn a wrong-verb call into a 404 and hide the real problem.
  • One parameter name per path position. Elysia's router rejects /api/files/:sessionId and /api/files/:fileId/toggle in the same tree. When two routes share a segment position, unify the name (the handlers map it to their own local variable). Current shared names: sessions/:sessionId (also carries a folder id for GET /api/sessions/:sessionId), files/:fileId.
  • A STATIC segment that shares a position with a parameter route is registered BEFORE it, and as its own explicit binding. /api/sessions/batch-delete and /api/folders/batch-delete are POST-only entries placed above the :sessionId / :folderId families, because a parameter route winning the match answers 405 (its handler dispatches on verb) and the batch silently becomes "method not allowed" instead of an unknown path. actionBindings is deliberately not used for them either — it mounts all four mutating verbs, and a batch verb should not answer PUT/PATCH.

Clone this wiki locally