P 07 Lazy Cached Engine

P-07: Lazy-load and cache the engine

Context: The engine is DuckDB compiled to WebAssembly: about 34 MB, loaded in a worker. The baseline (P-01, finding 6) found that "the warm start isn't warm". In warm mode the benchmark still downloaded 34.3 MB and the engine was ready no sooner than cold, so the 1 s warm budget was missed. Production already serves the hashed engine files with public, max-age=31536000, immutable, so the question was whether the app or the benchmark was at fault.

Hypothesis: Starting the engine after the first paint keeps it off the critical path of the app shell, and serving it from the HTTP cache on a return visit brings warm engine-ready under 1 s.

Change: #151, closing #51.

• The engine starts after the first paint. The workbench used to start it in its mount effect. It now starts it once the app shell has been painted (afterNextPaint), so creating the worker can't hold up the first frame. A query sent before then still starts it straight away.
• The preview server mirrors vercel.json's caching. It used to apply only the /(.*) rule, so hashed files under /assets/ were sent with Vite's no-cache instead of production's immutable. It now applies every matching rule per path. The e2e header spec and verify-headers check Cache-Control on /assets/ too.
• The benchmark's warm mode uses a profile on disk. This is why P-01 saw no cache. A Playwright context from browser.newContext() keeps its HTTP cache in memory, and Chromium's in-memory cache won't store a 34 MB response, so every "warm" run downloaded the engine again. Warm runs now use launchPersistentContext, as a returning visitor's browser does.
• No <link rel="preload">. It would fetch 34 MB before the first paint, and which engine build is needed (eh or mvp) is only decided at runtime.

Method:

• Scenarios: the cold-start scenarios from P-01, cold (fresh profile) and warm.
• Builds: main at 3639160 and the branch at a024d05, run in the same session. "Before" uses main's harness, whose warm mode was in memory, so its warm numbers are what P-01 measured, not what a returning visitor sees.
• Iterations: 3 warm-up and 10 measured iterations per scenario.
• Environment: the reference laptop from P-01 (Intel Core i5-8250U, 24 GB, Windows 11 Pro), Chromium 156, served locally.

Times are given as median / p95.

Cold and warm start

Metric	Before (main)	After (branch)	Change (median)
timeToEngineReady, cold	1,609 / 1,673 ms	1,702 / 2,361 ms	+5.8% (noise)
timeToEngineReady, warm	1,598 / 1,697 ms	1,581 / 1,795 ms	−1.1%
bytesTransferred, warm	34,251,466	3,060	−99.99%
timeToAppShell, cold	360 / 387 ms	412 / 833 ms	see finding 4
timeToAppShell, warm	345 / 363 ms	347 / 366 ms	unchanged

Budget	Target	After	Status
Engine ready, cold	≤ 4 s	1.7 s	✅ Met
Engine ready, warm	≤ 1 s	1.6 s	❌ Not met

Findings

1. The cold budget is met. The engine is ready in about 1.6 to 1.7 s with nothing cached, well inside 4 s.
2. The benchmark's warm mode was wrong, not the app. With a profile on disk, a warm start transfers 3 KB instead of 34 MB: the engine comes entirely from cache. P-01's "warm" runs were measuring a cache that can't hold a 34 MB entry.
3. The warm budget is still missed. Taking the download away didn't make the engine ready sooner: 1,581 ms against 1,598 ms. On localhost the download was never the cost. About 1.6 s goes to compiling and instantiating 34 MB of WebAssembly and booting DuckDB, and that happens on every start, cached or not.
4. The cold app shell is unchanged. The branch's 412 ms median includes an outlier (p95 833 ms). Re-running interleaved gave main 346 and 380 ms against the branch's 378 and 400 ms, within main's own spread. The app-shell mark is recorded before the engine starts in both versions, so this change can't move it.
5. The hypothesis held for bytes, not for time. Caching works, and on a real network a returning visitor no longer re-downloads 34 MB. But reaching 1 s needs the compiled module cached as well, not just the file.

Trade-offs:

• Engine start moves later by one frame. A user who types a query before the first paint doesn't wait for it, since a query starts the engine straight away.
• Warm benchmarks are slower to set up. A profile on disk per run instead of an in-memory context.
• Incognito windows have the same in-memory cache limit, so they always download the whole engine. The service worker (#52) covers that.

Limitations:

• Localhost only. Network time is close to zero, so the gain from not re-downloading 34 MB doesn't show in timeToEngineReady. On a real connection it would.
• Production was likely already cached. It already sent immutable, so returning visitors with a normal profile were probably served from disk before this change. What changed is that the benchmark can now see it.
• One session, Chromium only. Cold p95s are wide (2,361 ms for the branch), as on earlier runs on this laptop.

Evidence:

• Raw results: 2026-10-26T13-02-00Z-a024d05.json (branch) and 2026-10-26T12-59-19Z-3639160.json (main).
• Tests: the unit test "starts the engine once the shell has been painted" failed before the change, and the e2e Cache-Control test on /assets/ failed against the old preview ("Received no-cache").
• To reproduce: pnpm bench:data, then pnpm bench.

Conclusion: Keep. The cold budget is met, the engine is served from cache on a return visit, and the benchmark now measures a warm start correctly. The warm budget is not met: iterate by caching the compiled WebAssembly module (V8's code cache, or keeping a compiled WebAssembly.Module), as a follow-up.