Skip to content

feat(performance): priority scheduler + read coalescing on the SQLite worker (0228 Phase 1)#279

Merged
crs48 merged 3 commits into
mainfrom
claude/0228-parallel-sqlite-reads-worker-pool-and-dispatcher
Jun 26, 2026
Merged

feat(performance): priority scheduler + read coalescing on the SQLite worker (0228 Phase 1)#279
crs48 merged 3 commits into
mainfrom
claude/0228-parallel-sqlite-reads-worker-pool-and-dispatcher

Conversation

@crs48

@crs48 crs48 commented Jun 26, 2026

Copy link
Copy Markdown
Owner

Implements Phase 1 of exploration 0228.

Context

The natural follow-up to 0227 was "can we have a pool of reader workers + a dispatcher?" The exploration found that the naive version is impossible: the web adapter's opfs-sahpool VFS holds exclusive file handles (a second worker on the same DB falls back to in-memory), and the replica-based version that would give true parallelism is gated on a benchmark proving CPU saturation and on enabling production cross-origin isolation (which has embedding risk). So the doc's recommended "do now" is a scheduler on the single worker — most of the felt latency win at a fraction of the cost.

What landed

A priority scheduler + read coalescing inside SQLiteWorkerHandler (the one chokepoint both the main thread and the data worker share, via connectPort):

  • Interactive reads drain ahead of queued writes — a write/sync-apply burst can no longer starve a read. Generalizes the 0227 head-of-line fix to any queued op.
  • Identical concurrent reads coalesce into a single execution (request de-dup).
  • getSchedulerSnapshot() exposes queue depth for diagnostics.
  • No parallelism added (opfs-sahpool is single-connection) — this is ordering only, and serial execution matches SQLite's single-connection model (safer than today's interleave-at-awaits).

New worker-scheduler.ts + 7 deterministic tests proving the ordering/coalescing guarantees (these double as the exploration's "benchmark harness": an interactive read is provably served ahead of queued writes). @xnetjs/sqlite typecheck + full unit suite green.

Deliberately deferred (per the doc's own gating)

  • Transaction chunking / cooperative yield — splitting one atomic applyNodeBatch into multiple transactions changes atomicity semantics; correctness-sensitive, wants its own change.
  • Live p50/p95 boot probe + explicit bulk-tagging — refinements; the scheduler default (reads→interactive, writes→write) already delivers the benefit.
  • Devtools panel — a getSchedulerSnapshot() hook exists; wiring the panel is a follow-up.
  • All of Phase 2 (replica pool + dispatcher) — explicitly conditional on a saturation benchmark and on production COOP/COEP; not built speculatively.

The real worker path is exercised by the editor-ux/capture e2e checks.

🤖 Generated with Claude Code

xNet Test and others added 3 commits June 25, 2026 18:48
…dispatcher vs OPFS exclusivity)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… worker

All storage work now funnels through one priority scheduler inside the SQLite
worker (the single chokepoint both the main thread and data worker share):
interactive reads drain ahead of queued writes, so a write/sync-apply burst can
no longer starve a read — generalizing the 0227 head-of-line fix. Identical
concurrent reads are coalesced into one execution. No parallelism is added
(opfs-sahpool is single-connection by construction); this is pure ordering.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…r (0228)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@crs48 crs48 temporarily deployed to pr-279 June 26, 2026 01:58 — with GitHub Actions Inactive
@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Preview removed for PR #279.

github-actions Bot added a commit that referenced this pull request Jun 26, 2026
@crs48 crs48 merged commit c690de5 into main Jun 26, 2026
14 checks passed
@crs48 crs48 deleted the claude/0228-parallel-sqlite-reads-worker-pool-and-dispatcher branch June 26, 2026 02:11
github-actions Bot added a commit that referenced this pull request Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant