Skip to content

Add DB migration framework and split initial archive schema into migrations#2

Closed
t41372 wants to merge 1 commit into
mainfrom
codex/redesign-project-to-meet-new-requirements
Closed

Add DB migration framework and split initial archive schema into migrations#2
t41372 wants to merge 1 commit into
mainfrom
codex/redesign-project-to-meet-new-requirements

Conversation

@t41372
Copy link
Copy Markdown
Owner

@t41372 t41372 commented Apr 6, 2026

Motivation

  • Make archive schema evolution explicit and idempotent so installations without a stamped baseline can be upgraded safely.
  • Ensure new recoverability/import columns and indexes are applied in a controlled, versioned way rather than ad-hoc DDL.
  • Improve documentation formatting consistency in several markdown tables for readability.

Description

  • Introduce a schema_migrations table and an apply_migration helper used by create_schema to run idempotent migrations.
  • Split the monolithic archive SQL into migration files migrations/0001_initial_archive.sql and migrations/0002_import_recoverability.sql and wire them up via MIGRATION_0001_SQL / MIGRATION_0002_SQL constants.
  • Add migration SQL that creates the initial table set and the import/recoverability indexes/columns, and update schema creation to call ensure_ai_schema and ensure_insight_schema after migrations.
  • Add and update unit tests to validate migration stamping and legacy-schema upgrades, and tidy up related test logic in schema_migration_helpers_and_text_renderers_are_stable.

Testing

  • Ran the crate unit tests (including schema_migration_helpers_and_text_renderers_are_stable and create_schema_upgrades_legacy_archives_without_baseline_stamping) via cargo test in the vault-core crate, and they all passed.
  • Verified migration count and applied migration names are recorded in schema_migrations during tests. - All automated tests in the modified module succeeded.

Codex Task

@t41372 t41372 closed this Apr 7, 2026
@t41372 t41372 deleted the codex/redesign-project-to-meet-new-requirements branch April 11, 2026 02:14
t41372 added a commit that referenced this pull request May 22, 2026
… i18n

Why
- Codex review flagged three Blocking/High issues against feat/v0.3-redesign-2:
  (1) the ⌘K palette in shell.tsx sent `{ search, limit, offset }` to
      query_history and tried to read `response.rows`, but the real
      contract is `{ q, page, cursor, ... }` returning `items`. Result:
      palette searches were silently empty on desktop, and the existing
      tests had been mocking the wrong shape.
  (2) Paper UI still leaked raw English copy (char/chars counter, "Remove
      tag {tag}" aria, "Calendar" dialog label, "now"/"first" year-rail
      footers, dashboard greetings). i18n is a shipping contract, not
      polish.
  (3) Theme was held twice — once by the shell as a private useState,
      once by Settings → Appearance via applyPaperPreferences. Toggling
      one didn't update the other, so the buttons drifted.

What
- shell.tsx: palette now calls `backend.queryHistory({ q, limit, sort })`
  and maps `response.items` with the real HistoryEntry shape (id/url/
  domain/title/visitedAt/visitTime). Removed the orphan PaletteRow
  interface. shell.test.tsx mocks `backend.queryHistory` with valid
  HistoryQueryResponse fixtures instead of the legacy rows shape.
- i18n: added 3-language entries for the paper detail panel's char
  counter (singular + plural template), the remove-tag aria label, the
  calendar dialog aria, the year-rail now/first footer captions, and
  the dashboard morning/afternoon/evening greetings. The dashboard's
  hand-rolled greeting map is gone — it now reads
  `dashboard.greetingMorning/Afternoon/Evening` like every other copy
  in that route. PaperDetailPanelCopy + PaperCalendarPopoverCopy +
  PaperYearRailProps + paper-view.tsx copy interface all gained the
  new keys; buildPaperDetailPanelCopy + buildPaperContactSheetCopy
  thread them through. i18n parity 100% (2798 keys × 3 locales).
- paper-preferences.ts: applyPaperPreferences now dispatches a
  `pathkeep.paperPreferencesChanged` CustomEvent (with the resolved
  preferences in `detail.preferences`) after applying + persisting.
  shell.tsx subscribes to that event so the topbar theme button stays
  in sync with Settings → Appearance toggles; settings/appearance-
  section.tsx subscribes too so flipping theme via the shell button
  updates the radio without re-mount. shell.tsx's handleToggleTheme
  now routes through applyPaperPreferences instead of mutating a
  private useState — single source of truth, one persist call.

Context
- Codex findings #1, #5, #6. Findings #2 (paper pagination), #3
  (og:image fetch trigger), #4 (coverage gate restoration) tracked
  separately and remain pending in BACKLOG.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
t41372 added a commit that referenced this pull request May 22, 2026
…age gate

Why
- Codex review flagged three more issues:
  (2) Paper Explorer Browse exposed no pagination — the underlying
      useExplorerUrlState had `goToHistoryPage` / `setHistoryPageSize`
      handlers, but the paper UI dropped them. For a 1440 M-row archive
      this caps the user at the first response page (≤200 rows).
  (3) Card-mode og:image cache only loaded existing rows — there was no
      production path that enqueued a fetch for cache-miss URLs, so a
      fresh archive permanently rendered favicons.
  (4) `check:coverage` ran `coverage:rust:quality` (a triage slice) not
      the accepted full-scope `coverage:rust`. That's an unauthorised
      gate weakening vs. docs/plan/program/quality-matrix.md.

What
- PaperContactSheet: new optional `pagination` prop renders a footer
  with newer/older buttons (paper goes newest → oldest, so "next" is
  the older direction), a "page X of Y · N rows" summary, and a
  rows-per-page selector. Disabled buttons clamp at archive bounds.
  Footer omitted entirely when route doesn't supply pagination (e.g.
  the day-only insight surface).
- PaperExplorerView + Explorer route: thread the new pagination
  descriptor through from useExplorerUrlState's existing
  handleNextHistoryPage / handlePreviousHistoryPage / setHistoryPageSize
  handlers. `visibleTimeResults` carries the authoritative page /
  pageCount / total / hasNext / hasPrevious; the footer reads them.
- i18n: 5 new keys across all three locales — paginationOlder, Newer,
  Summary, SummaryPending, PageSizeLabel. 2803 keys × 3 locales,
  parity 100%.
- use-explorer-og-images.ts: after `loadHistoryOgImages` resolves,
  the hook now diffs the response and enqueues
  `triggerOgImageRefetch(batch)` for URLs that came back with a
  non-`ok` status (missing or never-cached). Bounded at 20 URLs per
  render so the worker pool can't be stampeded; `enqueuedFetchRef`
  prevents re-enqueuing within the same cache epoch. The promise's
  .catch swallows rate-limit / disabled-fetch rejections silently —
  the Rust worker persists negative-cache rows on its own.
- og_images_fetch.rs: extracted `read_capped_bytes<R: Read>` from
  `read_response_body` so the body-read fall-throughs (Io error,
  TooLarge) can be unit-tested directly with synthetic Read impls,
  closing line 336 of the production source. Three new cargo tests:
  errors-mid-stream → Io, oversize → TooLarge, short stream → Ok.
- package.json: check:coverage now calls `coverage:rust` (full
  scope), not the `:quality` triage slice. The Rust full gate still
  has ~20 uncovered defensive lines (worker pool mutex poison,
  sender disconnect, mid-stream Io in the live HTTP pipeline) — those
  surface on `bun run check` so a release gate failure is honest.

Coverage delta
- JS: unchanged at 99.71 stmts / 98.89 branches / 99.77 funcs /
  99.89 lines (existing threshold still 99/99/98/99 pending the
  separate JS residual sweep).
- Rust full scope: 12 → 7 uncovered lines in og_images_fetch.rs.
  archive_flows.rs worker pool internals (13 lines) still need an
  integration harness — tracked in WORK-V03-RUST-COVERAGE-RESIDUAL.

Context
- Codex findings #2, #3, #4. Codex flagged the gate-weakening change
  as a merge blocker — the check:coverage script is now restored to
  full coverage. The remaining residual is documented in BACKLOG and
  the gate fails until it closes, which matches the codex requirement
  of "merge 前必須恢復".

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
t41372 added a commit that referenced this pull request May 27, 2026
Why
  Codex review finding C1: the user's Settings → Link previews copy
  promises "Off — No fetching anywhere." That promise was leaking. The
  Browse `useExplorerOgImages` hook fired `triggerOgImageRefetch` for
  every visible cache miss without consulting the effective fetch mode,
  and the matching backend IPC entry (`refetch_og_images_impl`) only
  checked the legacy `fetch_enabled` kill switch — not the modern
  `fetch_mode` tri-state. So a user who picked "Off" via the modern
  control while the legacy switch stayed `true` (default for installs
  upgrading from the pre-tri-state schema, or any directly-edited
  config.json) still saw outbound HTTP fetches every time a card
  scrolled into view.

  This is a data-sovereignty contract violation per AGENTS.md core
  principle #2, not a paper cut.

What
  Backend (defence-in-depth, so a buggy caller can never bypass the
  policy):
    - Add `vault_worker::effective_og_image_fetch_mode()` that
      hydrates the unlocked config and returns
      `config.og_image.effective_mode()` (which already folds the
      legacy kill switch into the tri-state).
    - `worker_bridge::archive::refetch_og_images_impl` consults the
      effective mode first. `Off` → return `Ok(0)` without ever
      touching the worker pool. The explicit "Rebuild now"
      affordance lives behind `prefetch_og_images_impl` and is
      intentionally untouched (its docstring already declares that
      Rebuild ignores `fetch_mode` because clicking Rebuild *is*
      the explicit override).
    - New unit assertion in
      `worker_bridge::tests::annotations_and_og_image_impls_cover_local_desktop_flows`
      sets `fetch_enabled=true, fetch_mode=Off` and confirms the
      command short-circuits.

  Frontend (primary gate, so we don't even send the IPC for no-op
  work):
    - `useExplorerOgImages` takes a new optional
      `fetchMode: OgImageFetchModeConfig` prop (default
      `'background'` so older test callers behave unchanged).
      When `'off'` the hook still calls `loadHistoryOgImages` to
      hydrate cached rows from local SQLite, but skips the
      `triggerOgImageRefetch` enqueue branch entirely.
    - `pages/explorer/index.tsx` folds the legacy `fetchEnabled`
      switch into the effective mode (`fetchEnabled=false` ⇒ `'off'`)
      and threads it into the hook from `snapshot.config.ogImage`.
    - New hook test: with `fetchMode='off'` and a mix of cached + uncached
      rows, the cached row hydrates, the cache-miss URL is stored as
      null (so we don't re-issue the lookup), and `triggerOgImageRefetch`
      is never called.

How
  Two layers of gating, not one. The hook is the fast path (skips the
  IPC for cases the user shouldn't pay for), the backend is the truth
  (the policy holds even when the frontend is bypassed by automation,
  scripts, the dev-IPC bridge, or future callers we haven't written
  yet). Each path tested independently.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant