m2: DB + schemas + migrations - #304
Merged
Merged
Conversation
…otes/task_tags/inbox_items/saved_filters/settings)
…on_feedback, reminders)
…initions, sync_devices/queue/state/history)
…ers, signing_public_key, current_device unique index)
…ent_searches + table rebuilds, capture_source)
…s/note_metadata/property_definitions/folder_configs/calendar tables and alters)
Per-table domain structs derive specta::Type for Phase F binding generation. Project includes clock/synced_at to match the final schema after migrations 0015 + 0017 + 0018 (plan snippet elided them; SQL is authoritative). projects_and_tasks_roundtrip exercises INSERT + SELECT roundtrip via from_row for all three tables.
Per-table domain structs for the note vault tables (0006 + 0022). Each derives specta::Type for Phase F binding generation. PropertyDefinition uses r#type with explicit serde(rename) since type is a Rust keyword; the DB column stays "type". One roundtrip smoke test per table covers INSERT + SELECT via from_row.
… default apply_pending now disables foreign_keys at connection scope before replaying pending migrations and re-enables (with foreign_key_check) after. SQLite silently no-ops the in-SQL PRAGMA foreign_keys=OFF inside a transaction, so Drizzle's CREATE __new_X / INSERT / DROP X / RENAME rebuild migrations (notably 0018) cascade-delete child rows on populated DBs when the caller opened the connection with FK enforcement on. New regression test rebuild_migrations_preserve_rows_when_fk_enforcement_is_on applies 0000-0017, inserts FK-related rows, then runs apply_pending — without the toggle the DROP TABLE projects step wipes statuses + tasks. Also pass --features test-helpers in the cargo:test script so the migrations_test target (which is required-features gated) actually runs in the default verification path.
Phase D batch 1: 4 calendar domain structs covering migrations 0024 (foundation) + 0025 (event target_calendar_id) + 0026 (event field_clocks) + 0027 (rich fields on events + external_events) + 0028 (source last_error). Field set is the post-migration union. Each module exposes the canonical struct name expected by the Phase F specta generator (CalendarEvent, CalendarSource, CalendarExternalEvent, CalendarBinding) with a from_row helper. No CRUD yet — those land per domain in later milestones. Smoke tests: 4 roundtrip tests in migrations_test.rs. The CalendarEvent test embeds Turkish UTF-8 (çğıöşü + uppercase variants) to guard against encoding bugs at the SQLite/rusqlite boundary.
…structs Phase D batch 2: 5 user-collection structs covering migrations 0000 (inbox_items base) + 0001 (bookmarks) + 0002 (inbox rebuild + filing_history/inbox_item_tags/inbox_stats) + 0004 (reminders) + 0005 (inbox viewed_at, inbox_stats reminder count) + 0007 (tag_definitions) + 0011 (inbox sync metadata + local_only) + 0016 (tag_definitions clock) + 0018 (rebuild reset) + 0019 (inbox capture_source) + 0023 (folder_configs). Each module exposes the canonical struct name expected by the Phase F specta generator (InboxItem, Bookmark, Reminder, TagDefinition, FolderConfig) with a from_row helper. inbox_jobs is intentionally deferred — that struct lands with the M8.7 snooze scheduler. Notable column quirks: - InboxItem.local_only is Option<bool>: the column was added by ALTER TABLE in 0011 with no NOT NULL, so backfilled rows were nullable. The 0018 rebuild kept the same nullable shape with default 0. Strict type follows the schema (Phase D gotcha #1). - InboxItem.r#type uses the raw identifier escape; serde renames it back to `type` via #[serde(rename = ...)]. - TagDefinition uses `name` as primary key (no `id` column) and has no synced_at column even after the 0018 rebuild — only clock was added in 0016. Smoke tests: 5 roundtrip tests in migrations_test.rs, all asserting row.get path matches column names. Total tests: 11 (Phase B/C) + 4 (calendar) + 5 (collections) = 20 passing.
Two correctness bugs flagged by Codex review of the M2 migration runner: 1. Concurrent apply_pending could crash the second runner. Two app processes (dev hot-reload, double-launch) both bootstrapped, both read the same applied set, both computed the same pending list, both opened a BEGIN DEFERRED tx. Whichever lost the write-lock race crashed on either non-IF-NOT-EXISTS DDL (0001 `CREATE TABLE bookmarks`) or on the schema_migrations.name PRIMARY KEY. Fix: each per-migration tx now uses BEGIN IMMEDIATE so runners serialize on the SQLite write lock, and re-checks schema_migrations under that lock — if a sibling already recorded this migration, commit the empty tx and continue. The replay is now convergent. 2. apply_pending toggles PRAGMA foreign_keys = OFF for the rebuild migrations (0002/0009/0013/0018 use the CREATE __new_X / INSERT / DROP X / RENAME pattern that needs FK off at connection scope). If a migration failed mid-replay, the early ? return skipped the restore block, leaving FK enforcement permanently off on a connection that Db wraps in Arc<Mutex<Connection>> and reuses for every subsequent app write. Fix: scope the restore to run on both success and failure paths via an early-binding result, and make it best-effort so a restore failure doesn't mask the original migration error. TDD: two regression tests added before the fix. - apply_pending_restores_fk_state_on_migration_failure: pre-creates a colliding bookmarks table so 0001 fails, asserts FK is back to ON. RED: "PRAGMA foreign_keys must be restored to ON". GREEN with fix. - apply_pending_is_safe_under_concurrent_runners: temp file DB, WAL set on a setup connection, two threads with a Barrier race apply_pending, asserts both succeed and schema_migrations has exactly one row per embedded migration. RED: "UNIQUE constraint failed: schema_migrations.name". GREEN with fix and stable across 5 consecutive runs.
Adds the first real feature-domain IPC slice: a thin Setting KV store backed by SQLite via three Tauri commands. - db/settings.rs: get/set/list helpers with UPSERT semantics matching the 0018 schema (key TEXT PRIMARY KEY, value TEXT, modified_at strftime'd). - db/saved_filters.rs: SavedFilter struct + from_row mapper, present so the Phase F bindings stress-test exercises it without yet needing CRUD. - commands/settings.rs: 3 #[tauri::command] + #[specta::specta] handlers, thin adapters over the db layer with input structs gated on rename_all = camelCase. - lib.rs: invoke_handler! registers the 3 commands directly so the acceptance grep that scans lib.rs for settings::settings_* hits. - tests/settings_test.rs (test-helpers gated): 5 cases covering missing key, roundtrip, upsert, sorted list, and Turkish-character UTF-8 integrity through rusqlite TEXT. Mock router stays in place for every other settings_* route; only settings_get/set/list will flip to real Tauri once Task 15 lands.
Replaces the M1 stub generator with a tauri-specta Builder pipeline that emits TypeScript bindings for the full M2 domain surface plus the three settings commands. Adds specta-typescript as a direct dep so the generator binary can call Typescript::default(); previously it was only available as a transitive feature flag of tauri-specta and unreachable from our bin. Output (354 lines): - 1 commands const with settingsGet/settingsSet/settingsList wrappers that return tauri-specta's typed Result envelope - AppError discriminated union (kind + message tag/content) - 22 domain types (Bookmark, Calendar*, FolderConfig, InboxItem, NoteMetadata, NotePosition, Project, PropertyDefinition, Reminder, SavedFilter, SearchReason, Setting, Status, Sync*, TagDefinition, Task) - 2 input types for settings commands - typedError runtime helper camelCase rename hygiene verified: no created_at/modified_at/project_id fields leak through. The bindings:generate + bindings:check pipeline runs cargo against the generator bin to keep the file deterministic; the checked-in copy now matches a fresh regeneration. Renderer typecheck stays clean — nothing imports from @/generated/bindings yet. Phase F Task 15 hooks the Setting type up via the useSettings slice.
…wired Phase F's renderer slice. The IPC wrapper now lights up the three settings commands while every other domain stays on the JS-side mock router. - src/lib/ipc/invoke.ts: realCommands set gains settings_get/set/list. The router gate stays untouched, so unrelated mocks (settings_get_section, settings_set_general_settings, calendar/journal/tab/AI groups, etc.) continue to serve their M1 mock shapes until each domain's Rust handler lands in a later milestone. - src/hooks/useSettings.ts: new file. Tanstack Query bindings for the three commands — useSetting(key) / useSettings() / useSetSetting() — typed against `Setting` from generated/bindings.ts. Mutation onSuccess invalidates the per-key + list query keys. - tests/useSettings.test.tsx: end-to-end roundtrip through the real invoke wrapper using a stubbed @tauri-apps/api/core. Follows invoke.test.ts's pattern of vi.unmock'ing @/lib/ipc/invoke first so we bypass the global setup's auto-mock and exercise the live shouldUseMock branching. Three cases: roundtrip, list reflection after mutation, null for unset key. Settings package residue removed from acceptance grep paths (src/lib/ipc/**, src/hooks/useSettings.ts, src/pages/settings/**) by rehoming the two type-only contracts touched by this slice into src/types/settings-schemas.ts: ShortcutBinding (used by pages/settings/shortcuts-section.tsx) and CalendarSettings (used by pages/settings/calendar-section.tsx). Deeper consumers in src/lib/ and src/hooks/ still import @memry/contracts/settings-schemas and travel forward via the Phase G carry-forward ledger. Manual dev smoke (toggle setting → sqlite3 confirms row) is PENDING — no UI consumer wires useSetting/useSetSetting yet. The hook + bindings + Rust path are already proven by 3 vitest cases + 5 cargo cases, but end-to-end UI verification waits on either Phase G's broader rollout or a one-off temporary toggle the user can drive.
…V path
Codex adversarial review on Phase F flagged that the live onboarding flow
still routed through the JS-side mock router because only the new generic
useSettings hook was added — none of the existing domain-specific section
hooks were migrated. Onboarding completion appeared to persist in dev
because the mock module's state object survives between hook remounts, but
nothing actually reached SQLite.
This commit migrates the production caller (useGeneralSettings) onto the
KV path while keeping the hook's external contract intact.
- Read: invoke('settings_get', { input: { key: 'general' } }) returns the
serialized JSON blob (or null on first run). parseStoredSettings merges
any stored partial against DEFAULTS so forward-compat additions survive
schema drift.
- Write: settingsRef tracks the latest committed value; updateSettings
derives `next = { ...current, ...patch }`, JSON.stringifies it, and
pushes through invoke('settings_set', ...) before mutating local state.
The optimistic update is skipped until the IPC succeeds.
- Event subscription stays — settings-changed broadcasts still merge into
local state for cross-window propagation when the eventing layer comes
online in M3+.
Regression coverage in use-general-settings.test.tsx (TDD red→green):
1. Asserts the hook only invokes settings_get / settings_set against the
real Tauri core — never the legacy settings_get_general_settings or
settings_set_general_settings mock routes.
2. onboardingCompleted survives unmount + remount through the real KV
store (this was the exact case Codex flagged).
3. Multiple partial updates compose without losing prior fields (proves
the JSON merge layer doesn't drop properties on second write).
Test setup follows invoke.test.ts: vi.unmock('@/lib/ipc/invoke') bypasses
the global setup-dom auto-mock so the assertion is on the real wrapper,
and vi.mock('@tauri-apps/api/core') exposes a __resetStore knob that
beforeEach uses to isolate the in-memory KV between cases.
- scripts/dev-reset.sh, new-migration.ts, schema-diff.ts (DB tooling) - scripts/command-parity-audit.ts (renderer↔mock↔Rust classification ledger) - Replace electron-log/renderer logger with a Tauri-safe console wrapper - Harden port:audit to flag electron-log + bare 'electron' imports and track @memry/* residue (114 in production code, informational ledger) - Align updater mocks with use-app-updater hook (get_state / check_for_updates / download_update / quit_and_install) using the contract's AppUpdateState shape; legacy updater_check / _download / _install names removed - Add notify_flush_done as a real no-op Rust command so the renderer's flush-on-quit hook stops 404-ing through the mock router; full lifecycle coordinator deferred to M8.0 - Register db:reset, db:new-migration, db:schema-diff, command:parity scripts in package.json - Update setup-dom.ts mock target from electron-log/renderer to @/lib/logger so logger spies still work in tests
…cking This commit introduces the first real feature-domain IPC slice for managing settings through a thin KV store backed by SQLite. It includes: - Implementation of settings_get, settings_set, and settings_list commands in Rust. - Database helpers for get/set/list operations with UPSERT semantics. - Integration of these commands into the Tauri command handler. - Tests covering various scenarios including missing keys and UTF-8 integrity. The mock router remains for other settings routes, with a transition to real Tauri commands planned for future milestones.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Mutex<Connection>+ WAL PRAGMAssrc-tauri/migrations/*.sqlschema_migrationsbookkeepingspecta::Typederivesettings_get/settings_set/settings_listnotify_flush_donereal Rust shim (full lifecycle deferred to M8.0)MEMRY_DEVICE=A/Bprofile support +dev:a/dev:bscriptsdb:reset,db:new-migration,db:schema-diff,command:parityelectron-log/rendererto a Tauri-safe wrapperuseAppUpdater(updater_get_state,updater_check_for_updates,updater_download_update,updater_quit_and_install)Parent spec:
docs/superpowers/specs/2026-04-24-electron-to-tauri-full-migration-design.mdPlan:
docs/superpowers/plans/2026-04-25-m2-db-schemas-migrations.mdTest plan
All scripts live under
apps/desktop-tauri/package.json— run from the reporoot with
--filter @memry/desktop-tauri. Root-levelpnpm devlaunchesthe Electron app (
apps/desktop); use the filter prefix orcd apps/desktop-taurifor every command in this section.pnpm --filter @memry/desktop-tauri cargo:check && pnpm --filter @memry/desktop-tauri cargo:clippy && pnpm --filter @memry/desktop-tauri cargo:testgreen (32 Rust tests passing locally)pnpm --filter @memry/desktop-tauri bindings:checkclean (no drift)pnpm --filter @memry/desktop-tauri command:parityclean (89 commands classified, M2 invariants enforced)pnpm --filter @memry/desktop-tauri port:auditreports 0 Electron-era hitspnpm --filter @memry/desktop-tauri testgreen (3589 TS tests)pnpm --filter @memry/desktop-tauri typecheck && pnpm --filter @memry/desktop-tauri lintcleanpnpm --filter @memry/desktop-tauri db:resetthenpnpm --filter @memry/desktop-tauri dev:acreates all 29 rows inschema_migrationscd apps/desktop-tauri/src-tauri && cargo run --release --bin bench_m2 --features test-helpersp50 < 20ms (local: 285µs / 75× headroom)Schema parity vs Electron data DB
Ran
pnpm db:schema-diff <electron-vault>/.memry/data.db <tauri-data.db>:__drizzle_migrations,fts_tasks*,fts_inbox*schema_migrationsTriage:
__drizzle_migrations↔schema_migrations— equivalent migration bookkeeping; expected drift.fts-tasks.ts/fts-inbox.ts, not via Drizzle migrations. Deferred to M7 (search + sqlite-vec).Every shared table has matching column and index sets, confirming spec §6.2 Risk #17 (Drizzle semantics missed) is not violated by the M2 port.
Carry-forward ledger
The
command:parityaudit catalogues 81 renderer commands deferred to later milestones. Breakdown:114
@memry/*workspace import references remain in renderer production code (informational, mostly@memry/contracts/*types —@memry/rpc/*and@memry/db-schema/*are M3+ work).Risk coverage
--releaseper docstring🤖 Generated with Claude Code