Skip to content

chore(m5): drop core connector engine + DB v5 migration#115

Merged
graydawnc merged 2 commits intomainfrom
chore/m5-pr4-core
Apr 28, 2026
Merged

chore(m5): drop core connector engine + DB v5 migration#115
graydawnc merged 2 commits intomainfrom
chore/m5-pr4-core

Conversation

@graydawnc
Copy link
Copy Markdown
Collaborator

Summary

Continuation of #112 / #113 / #114. Removes the connector subsystem from @spool-lab/core — engine, scheduler, registry, plugin loader, capabilities, prerequisite checker, trust store, all capture queries — and adds a v5 schema migration that drops the connector tables on existing user DBs and narrows the stars CHECK constraint to session-only.

The renderer (#112), main process (#113), and CLI (#114) already stopped consuming any of this surface, so the cut is contained to packages/core/. Nothing else needed to change.

-6007 / +317 net.

Migration story

  • Existing user upgrading from v4 → v5 migration drops captures / captures_fts / captures_fts_trigram / capture_connectors / connector_sync_state + their triggers, deletes capture stars, rebuilds stars with the narrowed CHECK, drops the 'connector' source row. Captures data is gone — by design (users were/will be directed to Spool Daemon for connector functionality).
  • Fresh install on the post-v5 schema → schema block creates only the session tables; v1–v3 migrations no-op via try/catch (they reference tables that no longer exist on a fresh install); v4 creates stars; v5 narrows it via the rebuild round-trip.

Schema diff (packages/core/src/db/db.ts)

  • Schema block: removed captures, captures_fts, captures_fts_trigram, capture_connectors, connector_sync_state tables and their indexes; narrowed stars CHECK to item_type = 'session'; dropped the 'connector' source row from the seed insert
  • Trigger block: removed captures_fts_insert / captures_fts_delete triggers (kept their DROP IF EXISTS for upgrade cleanup)
  • v1–v3 migrations: wrapped in try/catch (table references no longer present on fresh install)
  • New v5 migration that does the actual table drops + stars rebuild
  • Maintenance section: dropped the captures FTS rebuild and the capture-orphan-stars sweep

Other changes

  • src/connectors/ — deleted entirely (27 files, ~3700 LOC)
  • src/db/queries.ts — dropped searchCaptures, searchAll, searchCaptureRows, mapCaptureRows, mergeCaptureGroups, insertCapture, getCaptureCount, compareSearchResultRelevance, getMatchTypePriority; narrowed getStarredUuidsByType and listStarredItems to session-only
  • src/types.ts — deleted CaptureResult, Capture, CapturedItem exports; narrowed SearchResult, StarKind, StarredItem
  • src/index.ts — dropped all connector + connector-sdk re-exports
  • package.json — dropped @spool-lab/connector-sdk, effect, semver, tar deps + their @types; removed connector-sdk filter from build script

New tests

  • src/db/migration-v5.test.ts — seeds a hand-built v4 DB with a session, a capture, a capture star, a connector_sync_state row; runs getDB(); verifies tables dropped, capture stars gone, session star kept, capture inserts now rejected, 'connector' source row gone, user_version = 5. Plus a fresh-install no-op case.
  • src/db/stars.test.ts — rewritten session-only (5 capture-specific cases removed, seedCapture helper gone)

Features that should still work post-PR (verified)

Feature Verification
Core build pnpm build clean (no connector-sdk filter; just tsc)
Core unit tests 48/48 pass (parsers, syncer, watcher, search-query, stars, migration-v5)
searchFragments shape kind: 'fragment' round-trip via spool search
getStarredUuidsByType new shape { session: string[] } typed end-to-end
listStarredItems session-only union, app already filters at boundary
App typecheck pnpm exec tsc --noEmit clean
App electron build pnpm build:electron clean — main bundle 56.65 kB → 55.72 kB, loader chunk 308.57 kB → gone (replaced by syncer chunk at 68 kB)
CLI build + tests pnpm build clean, 21/21 tests pass
spool sync Indexes 5 Claude fixture sessions
spool status Reports 5 total (claude: 5, codex: 0, gemini: 0)
spool search XYLOPHONE_CANARY_42 Returns matching session
Final DB schema sqlite3 dump shows only session tables — no captures/connector_sync_state/etc. — user_version = 5
Migration on real v4 data Seeded 2 stars (1 session + 1 capture), upgraded → capture star gone, session star kept, connector tables gone, narrow CHECK rejects new capture inserts

Test plan

  • pnpm test (core) 48/48 pass including new migration-v5 test
  • pnpm test (cli) 21/21 pass
  • pnpm build:electron (app) clean
  • spool sync / status / search smoke against real Claude fixtures
  • Verified DB schema after migration — only session tables remain, user_version = 5
  • Smoke session-search/agent-search/star/unstar in the running Electron app (will do once CI is green)
  • Smoke upgrade path with a real previous-version DB (preserved for the prod-style install smoke test that's tracked separately for the v0.4.0 release)

🤖 Generated with Claude Code

graydawnc and others added 2 commits April 28, 2026 17:24
Removes the connector subsystem from @spool-lab/core: the engine,
scheduler, registry, plugin loader, capabilities, prerequisite checker,
trust store, all capture queries and tests. Adds a v5 schema migration
that drops connector tables on existing user DBs and narrows the stars
CHECK constraint to session-only. The renderer / main process / CLI
already stopped using these in earlier PRs.

src/connectors/: deleted entirely (27 files, ~3700 LOC) — engine,
scheduler, registry, loader, npm-install, prerequisites, trust-store,
capabilities, types, all tests.

src/db/queries.ts:
- Drop searchCaptures, searchAll, searchCaptureRows, mapCaptureRows,
  mergeCaptureGroups, insertCapture, getCaptureCount,
  compareSearchResultRelevance, getMatchTypePriority
- Narrow getStarredUuidsByType return to { session: string[] }
- Narrow listStarredItems to session-only
- Drop CaptureResult / Capture / CapturedItem / SearchResult / Source
  type imports

src/db/db.ts:
- Schema block: remove captures / captures_fts / captures_fts_trigram /
  capture_connectors / connector_sync_state tables, their indexes, and
  the 'connector' source row insert; narrow stars CHECK to
  item_type = 'session'
- Trigger block: remove captures_fts_insert / captures_fts_delete (kept
  the DROP TRIGGER lines so existing-user upgrade cleans them out via v5)
- Wrap historical v1-v3 migrations in try/catch — they reference tables
  that fresh installs no longer create
- Add v5 migration: drops connector tables/triggers/FTS shadow tables,
  deletes capture stars, rebuilds stars with narrow CHECK, drops the
  'connector' source row
- Drop maintenance: rebuildFtsTableIfEmpty for captures, capture-orphan
  stars prune (no captures table to filter against)

src/types.ts:
- Delete CaptureResult, Capture, CapturedItem export
- Narrow SearchResult to fragment-only
- Narrow StarKind to 'session'
- Narrow StarredItem to session-only

src/index.ts: drop all connector exports + connector-sdk re-exports.

package.json: drop @spool-lab/connector-sdk, effect, semver, tar deps
and their @types; drop the connector-sdk filter from the build script.

Tests:
- Add src/db/migration-v5.test.ts: seeds a v4 DB with capture data, runs
  getDB(), verifies tables dropped + stars narrowed + session stars
  preserved + capture inserts now rejected; plus a fresh-install no-op
  case
- Rewrite src/db/stars.test.ts to session-only (drops 5 capture-specific
  cases, removes seedCapture helper)

Bundle/footprint:
- Net -6007 / +124 in core
- App main bundle 56.65 kB → 55.72 kB
- App loader chunk 308.57 kB → gone (replaced by syncer chunk at 68 kB)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous commit removed @spool-lab/connector-sdk, effect, semver,
tar (and @effect/vitest, @types/semver dev deps) from
packages/core/package.json without re-running pnpm install. CI's
--frozen-lockfile install rejected the drift.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@graydawnc graydawnc merged commit f5a775a into main Apr 28, 2026
4 checks passed
@graydawnc graydawnc deleted the chore/m5-pr4-core branch April 28, 2026 09:37
graydawnc added a commit that referenced this pull request Apr 28, 2026
The 0.4.0 npm publish shipped stale dist artifacts (commands/connector.js
and connector-shared.js from before #114 deleted them, plus core's
connectors/ subtree from before #115). tsc only writes new files; it
does not delete outputs whose source has been removed. The published
tarballs were functional (the CLI entry doesn't import the orphans)
but ~37 kB heavier than they should be.

Fix: prepend `pnpm run clean` (already defined as `rm -rf dist`) to
the build script so every fresh build starts from an empty dist/.
Verified locally — cli dist now contains only the 5 surviving
commands and core dist no longer carries a connectors/ tree.

Bump cli + core to 0.4.1 so the clean build can be published. App,
landing, and root version stay at 0.4.0 — the Electron release on
GitHub is unaffected by this fix and we don't need to cut another
DMG just for npm hygiene.

Co-authored-by: Chen <99816898+donteatfriedrice@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant