Skip to content

feat(local-first): A0.1 perf harness → B0.1 PoC → ADR-0008 → B1.1 shadow sync#140

Merged
qnbs merged 16 commits into
mainfrom
feat/perf-harness-a01
Jun 16, 2026
Merged

feat(local-first): A0.1 perf harness → B0.1 PoC → ADR-0008 → B1.1 shadow sync#140
qnbs merged 16 commits into
mainfrom
feat/perf-harness-a01

Conversation

@qnbs

@qnbs qnbs commented Jun 15, 2026

Copy link
Copy Markdown
Owner

User description

Context

First concrete delivery of the Local-First track from the strategic audit (Y.Doc-as-source-of-truth, ADR-0008). The app was offline-first but not local-first in the CRDT sense — Yjs was an inert bolt-on. This branch lays the foundation and proves the direction is sound, all behind a flag that is off by default (Redux stays the source of truth).

Six commits, each independently green:

Commit What
86438db A0.1 — large-manuscript perf harness (pnpm bench, deterministic ~120k-word fixture)
8d77f67 B0.1 — CRDT PoC + decision-gate proofs (services/localFirst/projectDoc.ts)
56ce87f ADR-0008 — local-first data model (Y.Doc as source of truth)
cebeded B1.1 binding — incremental shadow write-through (docBinding.ts)
4230bf8 B1.1 wiringenableLocalFirstSync flag + listenerMiddleware shadow sync + y-indexeddb

Why it matters

  • A0.1 baseline already surfaced a real risk: typing on 120k words hits p99 ≈ 15 ms / p999 ≈ 50 ms (over the 16 ms frame budget) and full-manuscript word count ≈ 34 ms. This is the before/after gate the migration is judged against.
  • B0.1 decision gate is green: round-trip fidelity, char-level concurrent merge (both edits kept — the win a string-LWW model loses), section-level merge, and Y.UndoManager undo all proven.
  • B1.1 wires a flag-gated, debounced shadow sync that mirrors Redux → per-project Y.Doc (+ y-indexeddb). On read-verify drift it logs a count (no PII) and self-heals via re-projection — never affects the user.

Scope / deliberate deferrals

  • enableLocalFirstSync is off by default; Redux remains the source of truth. This branch does not flip anything.
  • Parity check (pnpm run parity:check) was left to CI — the repo-wide grep swaps on the low-end dev host. Flag consistency verified by construction (defaults + locale + section + handler + runtime all present across 11 locales).
  • An E2E flag spec + the Playwright typing-latency spec are follow-ups (E2E is CI-only here).

Verification (local, all green)

  • typecheck ✅ · lint ✅ (1346 files) · targeted unit tests ✅ (76 — incl. new localFirst/ suites + all 8 FeatureFlagsState mocks) · i18n:check ✅ (2697 × 11) · pnpm audit ✅ (clean; new dep y-indexeddb@^9.0.12).

Next (later sessions)

  • B2.x: per-section Y.Text data migration + the gated Redux→Y.Doc flip (one-way door — keeps POJO backup).
  • Track A in parallel: editor perf hardening (A2.2, informed by A0.1), coverage to C-7, Tauri release loop.

🤖 Generated with Claude Code


CodeAnt-AI Description

Add a gated local-first sync shadow, with performance baselines and docs

What Changed

  • Added an experimental Local-First sync flag to Settings so the new shadow sync can be turned on and off from the UI.
  • When enabled, the app now mirrors project changes into a Yjs document in the background, loads saved shadow data on startup, and tears the shadow down when the flag is turned off.
  • Added support for saving the shadow doc per project and keeping it in sync without affecting the current Redux-based source of truth.
  • Added a large-manuscript benchmark suite plus a deterministic fixture to measure typing, undo, save, snapshot, word count, and ProForge assembly on a ~120k-word project.
  • Added ADR 0008 and localized the new feature flag label across supported languages; updated tests to cover the new flag and local-first behavior.

Impact

✅ Experimental local-first sync can be enabled from Settings
✅ Shadow project data now persists across reloads when enabled
✅ Clear before/after baseline for large-manuscript editor performance

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

@codeant-ai

codeant-ai Bot commented Jun 15, 2026

Copy link
Copy Markdown

CodeAnt AI is reviewing your PR.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@vercel

vercel Bot commented Jun 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
storycraft-studio Ready Ready Preview, Comment Jun 16, 2026 9:06am
worldscript-studio Ready Ready Preview, Comment Jun 16, 2026 9:06am

@qnbs

qnbs commented Jun 15, 2026

Copy link
Copy Markdown
Owner Author

@CodeAnt-AI review

@codeant-ai

codeant-ai Bot commented Jun 15, 2026

Copy link
Copy Markdown

CodeAnt AI is running the review.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai Bot added the size:XXL This PR changes 1000+ lines, ignoring generated files label Jun 15, 2026
@codeant-ai

codeant-ai Bot commented Jun 15, 2026

Copy link
Copy Markdown

Sequence Diagram

This PR adds an experimental Local-First sync feature that, when enabled, mirrors the Redux project state into a per-project Yjs document with IndexedDB persistence, while Redux remains the source of truth and a shadow binding verifies and self-heals any drift.

sequenceDiagram
    participant User
    participant SettingsUI
    participant ReduxStore
    participant ListenerMiddleware
    participant LocalFirstBinding
    participant IndexedDB

    User->>SettingsUI: Enable Local-First sync
    SettingsUI->>ReduxStore: Set enableLocalFirstSync = true

    ReduxStore->>ListenerMiddleware: Project state changed with flag enabled
    ListenerMiddleware->>LocalFirstBinding: Create or reuse project doc with project id
    LocalFirstBinding->>IndexedDB: Attach persistence and load existing doc state

    ListenerMiddleware->>LocalFirstBinding: Sync from current ProjectData
    LocalFirstBinding->>LocalFirstBinding: Incrementally update Y.Doc and verify against ProjectData
    LocalFirstBinding->>IndexedDB: Persist updated doc in background

    User->>SettingsUI: Disable Local-First sync
    SettingsUI->>ListenerMiddleware: Flag turned off, teardown binding and persistence
Loading

Generated by CodeAnt AI

Comment thread services/localFirst/docPersistence.ts Outdated
Comment thread tests/unit/localFirst/docPersistence.test.ts Outdated
Comment thread tests/unit/localFirst/docPersistence.test.ts Outdated
Comment thread tests/unit/localFirst/docPersistence.test.ts Outdated
qnbs added a commit that referenced this pull request Jun 15, 2026
…oundary (CodeAnt #140)

- docPersistence.test.ts: drop the fixed setTimeout(100) flush; poll a fresh
  provider via vi.waitFor (condition-driven, no wall-clock guess). Fake timers
  can't advance fake-indexeddb's async IDBRequest resolution, so waitFor is the
  correct deterministic mechanism. Tests now run in ~half the time.
- docPersistence.ts: document why this is the centralized local-first
  persistence chokepoint (mirroring logger/proForgeMemoryBank/aiInferenceCache
  IDB wrappers) and why a Yjs update-log provider cannot route through the
  high-level StorageBackend API.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread services/localFirst/docBinding.ts Outdated
Comment thread services/localFirst/docBinding.ts Outdated
Comment thread services/localFirst/docBinding.ts
Comment thread services/localFirst/docBinding.ts Outdated
Comment thread app/listenerMiddleware.ts Outdated
Comment thread app/listenerMiddleware.ts
@codeant-ai

codeant-ai Bot commented Jun 15, 2026

Copy link
Copy Markdown

CodeAnt AI finished running the review.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

qnbs added a commit that referenced this pull request Jun 15, 2026
#140)

Move every persistProjectDoc teardown (destroy/clearData) into finally so a
failing assertion or a vi.waitFor timeout can never leak an open IndexedDB
provider into later tests. Addresses the missing-cleanup + fixed-delay-race
findings on docPersistence.test.ts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@qnbs

qnbs commented Jun 15, 2026

Copy link
Copy Markdown
Owner Author

@CodeAnt-AI review

@codeant-ai

codeant-ai Bot commented Jun 15, 2026

Copy link
Copy Markdown

CodeAnt AI is running the review.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai Bot added size:XXL This PR changes 1000+ lines, ignoring generated files and removed size:XXL This PR changes 1000+ lines, ignoring generated files labels Jun 15, 2026
@codeant-ai

codeant-ai Bot commented Jun 15, 2026

Copy link
Copy Markdown

Sequence Diagram

This PR adds an experimental Local-First sync flag that, when enabled, mirrors the Redux project state into a per-project Yjs document with IndexedDB persistence, verifies for drift, and self-heals while remaining shadow-only.

sequenceDiagram
    participant User
    participant SettingsUI
    participant ReduxStore
    participant Listener
    participant LocalFirstSync
    participant Storage

    User->>SettingsUI: Change Local-First sync setting
    SettingsUI->>ReduxStore: Update enableLocalFirstSync flag
    ReduxStore-->>Listener: Notify flag and project state change

    alt Flag enabled
        Listener->>LocalFirstSync: Create project doc and attach persistence
        LocalFirstSync->>Storage: Load persisted state into project doc
        Listener->>LocalFirstSync: On debounced project change mirror Redux project
        LocalFirstSync->>LocalFirstSync: Update doc incrementally and verify vs project
        LocalFirstSync-->>Listener: Log drift count and reproject on mismatch
    else Flag disabled
        Listener->>LocalFirstSync: Tear down doc binding and persistence
    end
Loading

Generated by CodeAnt AI

Comment thread services/localFirst/docPersistence.ts Outdated
Comment thread tests/unit/localFirst/docPersistence.test.ts
Comment thread tests/unit/localFirst/docPersistence.test.ts
Comment thread tests/unit/localFirst/docPersistence.test.ts
Comment thread tests/unit/localFirst/projectDoc.test.ts Outdated
Comment thread services/localFirst/projectDoc.ts Outdated
Comment thread app/listenerMiddleware.ts Outdated
@codeant-ai

codeant-ai Bot commented Jun 15, 2026

Copy link
Copy Markdown

CodeAnt AI finished running the review.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

qnbs added a commit that referenced this pull request Jun 15, 2026
Six valid findings from the second review pass:

docBinding.ts:
- reconcileSection: a field cleared to `undefined` is now deleted from the
  Yjs map (was left stale — `key in section` stays true for explicit undefined).
- reconcileMeta: nextKeys built from DEFINED values so cleared meta fields are
  actually removed, not left stale.
- observe(): binding writes now run in `doc.transact(fn, this.origin)` (new
  public readonly origin symbol) so consumers can echo-filter self-updates.
- verify(): compares EVERY meta field reconcileMeta writes (logline, author,
  goals…), not just title — non-title drift now triggers self-heal.

listenerMiddleware.ts:
- Handle create/teardown serialized behind a promise lock — overlapping sync
  and enable/disable runs can no longer race on the module-global handle.
- OFF→ON now performs an immediate initial projection via a dedicated enable
  listener (shared runLocalFirstShadowSync helper) instead of waiting for the
  next edit.

Tests: +4 docBinding (undefined-clear section + meta, non-title verify drift,
observe origin). typecheck + lint + 22 localFirst tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codeant-ai

codeant-ai Bot commented Jun 15, 2026

Copy link
Copy Markdown

CodeAnt AI is reviewing your PR.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai Bot added size:XXL This PR changes 1000+ lines, ignoring generated files and removed size:XXL This PR changes 1000+ lines, ignoring generated files labels Jun 15, 2026
Comment thread tests/unit/localFirst/docPersistence.test.ts Outdated
@codeant-ai codeant-ai Bot added the size:XXL This PR changes 1000+ lines, ignoring generated files label Jun 16, 2026
@codeant-ai

codeant-ai Bot commented Jun 16, 2026

Copy link
Copy Markdown

Sequence Diagram

This diagram shows how enabling the Local-First sync feature flag causes the app to mirror the Redux project state into a Yjs document with IndexedDB persistence, and how subsequent edits trigger debounced shadow sync updates.

sequenceDiagram
    participant User
    participant SettingsUI
    participant ReduxStore
    participant Listener
    participant LocalFirstSync
    participant IndexedDB

    User->>SettingsUI: Toggle Local-First sync on
    SettingsUI->>ReduxStore: Dispatch enableLocalFirstSync true
    ReduxStore->>Listener: Feature flags state updated
    Listener->>LocalFirstSync: Start shadow sync with current project
    LocalFirstSync->>IndexedDB: Attach y-indexeddb and load project doc
    IndexedDB-->>LocalFirstSync: Return persisted doc state
    LocalFirstSync->>LocalFirstSync: Project to Y.Doc projection and verify
    ReduxStore->>Listener: Project manuscript edited
    Listener->>LocalFirstSync: Debounced shadow sync from Redux
Loading

Generated by CodeAnt AI

Comment thread services/localFirst/docPersistence.ts
Comment thread app/listenerMiddleware.ts
…ent (#140 wave-4)

- Privacy: the shadow doc was persisted as PLAINTEXT to y-indexeddb even when
  at-rest encryption (enableIdbAtRestEncryption) was active. getLocalFirstHandle
  now uses NOOP_PERSISTENCE (in-memory only) when isIdbEncryptionReady(), so no
  unencrypted manuscript copy hits disk; shadow-sync still validates vs Redux.
- docPersistence: `active` is now a getter reflecting live state (false once
  teardown begins, incl. the async whenSynced-rejection path) instead of a
  constant true; destroy() memoizes the real teardown promise so concurrent/repeat
  calls share one in-flight destroy (no double-destroy, no flag flipped before
  destroy completes).
- docBinding.reconcileSection: if the content slot holds a non-Y.Text
  (corrupted/legacy) value, recreate a Y.Text seeded with the content instead of
  silently dropping the edit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codeant-ai

codeant-ai Bot commented Jun 16, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@qnbs

qnbs commented Jun 16, 2026

Copy link
Copy Markdown
Owner Author

@CodeAnt-AI review

@codeant-ai

codeant-ai Bot commented Jun 16, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai Bot added size:XXL This PR changes 1000+ lines, ignoring generated files and removed size:XXL This PR changes 1000+ lines, ignoring generated files labels Jun 16, 2026
@codeant-ai

codeant-ai Bot commented Jun 16, 2026

Copy link
Copy Markdown

Sequence Diagram

This PR adds an experimental Local-First sync feature that, when enabled, mirrors the Redux project state into a Yjs document with optional IndexedDB persistence using a debounced listener and self-healing verification.

sequenceDiagram
    participant User
    participant SettingsUI
    participant ReduxStore
    participant LocalFirstSync
    participant YDocBinding
    participant IndexedDb

    User->>SettingsUI: Toggle Local-First sync on
    SettingsUI->>ReduxStore: Dispatch enableLocalFirstSync true
    ReduxStore->>LocalFirstSync: Feature flag is enabled
    LocalFirstSync->>YDocBinding: Initialize project doc with current project data
    YDocBinding->>YDocBinding: Project data into Y.Doc and verify, reproject on drift
    YDocBinding->>IndexedDb: Persist doc state when persistence is active

    User->>ReduxStore: Edit manuscript section
    ReduxStore->>LocalFirstSync: Debounced project change event
    LocalFirstSync->>YDocBinding: Incremental sync from updated project and verify
Loading

Generated by CodeAnt AI

Comment thread tests/unit/localFirst/docPersistence.test.ts
Comment thread tests/unit/localFirst/docPersistence.test.ts
Comment thread services/localFirst/docBinding.ts
Comment thread services/localFirst/docBinding.ts
Comment thread app/listenerMiddleware.ts Outdated
…ence in tests (#140 wave-5)

- Privacy: the LocalFirst handle chose NOOP vs y-indexeddb only at creation, then
  was reused by projectId — so if at-rest encryption activated AFTER a plaintext-
  persisting handle was made, it kept writing plaintext. getLocalFirstHandle now
  re-checks isIdbEncryptionReady() on cache hit and, if encryption is now active on
  a still-persisting handle, wipes the plaintext (clearData) and tears it down so
  it's recreated in-memory.
- docPersistence tests: the persistence/clear assertions read while the writer
  provider was still alive, so the value could arrive via live cross-provider sync
  rather than persisted IndexedDB (false positive). Destroy the writer, then
  re-read — the value can now only come from (or be absent in) the IDB store.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codeant-ai

codeant-ai Bot commented Jun 16, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@qnbs

qnbs commented Jun 16, 2026

Copy link
Copy Markdown
Owner Author

@CodeAnt-AI review

@codeant-ai

codeant-ai Bot commented Jun 16, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai Bot added size:XXL This PR changes 1000+ lines, ignoring generated files and removed size:XXL This PR changes 1000+ lines, ignoring generated files labels Jun 16, 2026
@socket-security

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​y-indexeddb@​9.0.121001009480100

View full report

@codeant-ai codeant-ai Bot added size:XXL This PR changes 1000+ lines, ignoring generated files and removed size:XXL This PR changes 1000+ lines, ignoring generated files labels Jun 16, 2026
@codeant-ai

codeant-ai Bot commented Jun 16, 2026

Copy link
Copy Markdown

Sequence Diagram

This PR adds an experimental Local-First sync flag that, when enabled, mirrors the Redux project into a Yjs document with optional IndexedDB persistence, verifies for drift, and runs as a background shadow sync without affecting the user.

sequenceDiagram
    participant User
    participant SettingsUI
    participant FeatureFlagsStore
    participant ListenerMiddleware
    participant LocalFirstSync
    participant IndexedDB

    User->>SettingsUI: Toggle Local-First sync on
    SettingsUI->>FeatureFlagsStore: Dispatch enableLocalFirstSync = true
    FeatureFlagsStore-->>ListenerMiddleware: Feature flags state updated
    ListenerMiddleware->>LocalFirstSync: Run initial shadow sync for current project
    LocalFirstSync->>IndexedDB: Attach or skip project doc persistence
    LocalFirstSync->>LocalFirstSync: Mirror Redux project into Y doc and verify
    alt Drift detected between Redux and Y doc
        LocalFirstSync->>LocalFirstSync: Reproject from Redux and log drift
    end
Loading

Generated by CodeAnt AI

@qnbs
qnbs merged commit ce0c1f2 into main Jun 16, 2026
18 checks passed
@qnbs
qnbs deleted the feat/perf-harness-a01 branch June 16, 2026 10:38
qnbs added a commit that referenced this pull request Jun 16, 2026
… CF Pages rename (#166)

* chore(release): finalize v1.23.0 changelog + rename CF Pages project

Promote the [Unreleased] changelog to [1.23.0] — 2026-06-16 and add this
release's remaining entries (Local-First foundation #140, OpenRouter reactive
policy #163, Local AI i18n #164, WorldScript icons + URL cleanup #165, and
Dependabot maintenance). Rename the Cloudflare Pages project in wrangler.toml
storycraft-studio → worldscript-studio (CF is not actively deployed; the
project rename on the CF side is a later step).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(tauri): resolve Vite base from TAURI_ENV_PLATFORM (fixes blank desktop window)

The Tauri build detected desktop mode via `TAURI_PLATFORM`, the Tauri 1.x env
var that Tauri 2.x never sets. Every desktop build therefore fell through to the
GitHub Pages base (`/WorldScript-Studio/`) and 404'd its hashed assets under
`tauri://localhost/`, leaving only the native file/help menu over an empty
webview on Windows/macOS.

Base resolution now checks `TAURI_ENV_PLATFORM` (legacy `TAURI_PLATFORM` kept as
a fallback) and is extracted into a pure, unit-tested `config/resolveViteBase.ts`
so the regression cannot silently return. No runtime landmines: locale/community
fetches use `import.meta.env.BASE_URL`, which resolves correctly under `./`.

Linux `.deb`/AppImage still require webkit2gtk-4.1 (Ubuntu 22.04+) — a Tauri 2
platform minimum documented in the changelog, not a bug.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL This PR changes 1000+ lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants