Context
scripts/migrate-selfhost-sqlite-to-postgres.ts (~480 lines) is the one-shot data migration a
self-host operator runs to move their SQLite backend to Postgres (npm run selfhost:postgres:migrate).
It is high-stakes by design: it copies every table's rows across engines, validates row counts and
key-matching after copy, resets Postgres sequences, prunes schema-init seed rows, and defaults to a
transactionally-rolled-back dry run unless --execute is passed. Despite that, it has zero test
coverage anywhere under test/** — confirmed via find test -iname "*migrate-selfhost*" /
*sqlite-to-postgres*", both empty.
This is a real gap, not a hypothetical: scripts/** is excluded from Codecov's coverage.include
(codecov.yml), so nothing in CI currently forces this file to be tested, and a regression here would
only ever surface when an operator runs a real, potentially destructive migration against their own
data.
Several of this script's exported pure functions are directly testable without a real Postgres server,
following the exact mocking convention test/unit/selfhost-pg-adapter.test.ts already established for
src/selfhost/pg-adapter.ts (a hand-built Pool/PoolClient mock whose query() records every SQL
statement + params to a shared log, so a test can assert on exact generated SQL):
quoteIdent (identifier-injection guard — throws on any non-[A-Za-z0-9_] identifier)
insertSql (batched multi-row INSERT ... VALUES (...), (...) [ON CONFLICT (...) DO NOTHING] builder)
valuePlaceholder (the _selfhost_vectors ::vector/::jsonb cast special-case)
normalizePostgresValue (NUL-byte replacement for Postgres text/json columns — SQLite text can contain
\0, Postgres cannot)
parseArgs (flag parsing + the --postgres-url/DATABASE_URL scheme validation, the
--batch-size positive-integer guard, the SQLite-source/migrations-dir existence checks)
The higher-level orchestration (copyAll, copyTable, countTargetRowsMatchingSourceRows,
countConflictingTargetRowsForSourceKeys, resetPgSequences) needs the same mock-PoolClient
approach test/unit/selfhost-pg-adapter.test.ts/test/unit/selfhost-pg-queue.test.ts already use, paired
with a real in-memory node:sqlite DatabaseSync source (the script already imports
node:sqlite's DatabaseSync, and the repo already relies on in-memory node:sqlite elsewhere —
test/helpers/d1.ts's TestD1Database).
Requirements
- Add
test/unit/migrate-selfhost-sqlite-to-postgres.test.ts (or a -script suffix matching this
repo's existing naming for CLI-wrapper test files, e.g. selfhost-env-reference-script.test.ts).
- Cover the pure functions listed above directly (import them from the script — they are already
top-level exported or straightforward to export if currently unexported; check current export
status and add export only where genuinely needed for testability, not a broad refactor).
- Cover
copyAll's real decision branches using a mock PoolClient (mirroring
test/unit/selfhost-pg-adapter.test.ts's makeMockPool pattern) + a real in-memory DatabaseSync
loaded with a couple of tiny fixture tables:
- happy path: empty target, common columns present, rows copied, sequences reset.
_selfhost_vectors skipped without --include-vectors, copied with it.
- throws when the target Postgres schema is missing a source table.
- throws when the target already has rows and
--allow-non-empty is not set.
--allow-non-empty with a non-conflicting key overlap succeeds; with a conflicting one throws.
- throws when a non-empty target table has no comparable primary key overlap.
- the post-copy validation branch (
targetCount < targetRowsBefore throws; the
_selfhost_job_stats "at least" comparison vs. the exact-match comparison for other tables).
- Cover
parseArgs's validation branches (missing/invalid --postgres-url, non-numeric/<1
--batch-size, missing SQLite source file, missing migrations dir, --help exit).
Deliverables
Test Coverage Requirements
scripts/** is excluded from Codecov's coverage.include (codecov.yml), so this PR owes no
codecov/patch percentage — there is no automated coverage gate for this path today, which is exactly
the gap this issue closes. Verification instead is: the new test file passes under
npx vitest run test/unit/<new-file>.test.ts, exercises every branch listed in Requirements (both the
throw and success side of each), and the full npm run test:coverage run stays green. Do not add a
scripts/** entry to codecov.yml's include as part of this change — that's a separate,
repo-wide decision out of scope here.
Expected Outcome
A regression in scripts/migrate-selfhost-sqlite-to-postgres.ts's SQL-generation, validation, or
conflict-detection logic is caught by npm run test:coverage locally and in CI, instead of only being
discoverable by an operator running a real (potentially destructive) Postgres migration.
Links & Resources
scripts/migrate-selfhost-sqlite-to-postgres.ts (the file needing tests)
test/unit/selfhost-pg-adapter.test.ts (the mock-Pool/PoolClient pattern to mirror)
test/unit/selfhost-pg-queue.test.ts / test/unit/selfhost-pg-vectorize.test.ts (siblings using the
same mock convention)
test/helpers/d1.ts (existing in-memory node:sqlite usage pattern in this repo's test suite)
codecov.yml (confirms scripts/** is excluded from coverage.include)
Context
scripts/migrate-selfhost-sqlite-to-postgres.ts(~480 lines) is the one-shot data migration aself-host operator runs to move their SQLite backend to Postgres (
npm run selfhost:postgres:migrate).It is high-stakes by design: it copies every table's rows across engines, validates row counts and
key-matching after copy, resets Postgres sequences, prunes schema-init seed rows, and defaults to a
transactionally-rolled-back dry run unless
--executeis passed. Despite that, it has zero testcoverage anywhere under
test/**— confirmed viafind test -iname "*migrate-selfhost*"/*sqlite-to-postgres*", both empty.This is a real gap, not a hypothetical:
scripts/**is excluded from Codecov'scoverage.include(
codecov.yml), so nothing in CI currently forces this file to be tested, and a regression here wouldonly ever surface when an operator runs a real, potentially destructive migration against their own
data.
Several of this script's exported pure functions are directly testable without a real Postgres server,
following the exact mocking convention
test/unit/selfhost-pg-adapter.test.tsalready established forsrc/selfhost/pg-adapter.ts(a hand-builtPool/PoolClientmock whosequery()records every SQLstatement + params to a shared log, so a test can assert on exact generated SQL):
quoteIdent(identifier-injection guard — throws on any non-[A-Za-z0-9_]identifier)insertSql(batched multi-rowINSERT ... VALUES (...), (...) [ON CONFLICT (...) DO NOTHING]builder)valuePlaceholder(the_selfhost_vectors::vector/::jsonbcast special-case)normalizePostgresValue(NUL-byte replacement for Postgres text/json columns — SQLite text can contain\0, Postgres cannot)parseArgs(flag parsing + the--postgres-url/DATABASE_URLscheme validation, the--batch-sizepositive-integer guard, the SQLite-source/migrations-dir existence checks)The higher-level orchestration (
copyAll,copyTable,countTargetRowsMatchingSourceRows,countConflictingTargetRowsForSourceKeys,resetPgSequences) needs the same mock-PoolClientapproach
test/unit/selfhost-pg-adapter.test.ts/test/unit/selfhost-pg-queue.test.tsalready use, pairedwith a real in-memory
node:sqliteDatabaseSyncsource (the script already importsnode:sqlite'sDatabaseSync, and the repo already relies on in-memorynode:sqliteelsewhere —test/helpers/d1.ts'sTestD1Database).Requirements
test/unit/migrate-selfhost-sqlite-to-postgres.test.ts(or a-scriptsuffix matching thisrepo's existing naming for CLI-wrapper test files, e.g.
selfhost-env-reference-script.test.ts).top-level
exported or straightforward to export if currently unexported; check current exportstatus and add
exportonly where genuinely needed for testability, not a broad refactor).copyAll's real decision branches using a mockPoolClient(mirroringtest/unit/selfhost-pg-adapter.test.ts'smakeMockPoolpattern) + a real in-memoryDatabaseSyncloaded with a couple of tiny fixture tables:
_selfhost_vectorsskipped without--include-vectors, copied with it.--allow-non-emptyis not set.--allow-non-emptywith a non-conflicting key overlap succeeds; with a conflicting one throws.targetCount < targetRowsBeforethrows; the_selfhost_job_stats"at least" comparison vs. the exact-match comparison for other tables).parseArgs's validation branches (missing/invalid--postgres-url, non-numeric/<1--batch-size, missing SQLite source file, missing migrations dir,--helpexit).Deliverables
test/unit/migrate-selfhost-sqlite-to-postgres.test.ts(or equivalently named) covering thefunctions and branches listed above.
exportadditions toscripts/migrate-selfhost-sqlite-to-postgres.tsneeded to makethe above testable, with no behavior change to the script itself.
Test Coverage Requirements
scripts/**is excluded from Codecov'scoverage.include(codecov.yml), so this PR owes nocodecov/patchpercentage — there is no automated coverage gate for this path today, which is exactlythe gap this issue closes. Verification instead is: the new test file passes under
npx vitest run test/unit/<new-file>.test.ts, exercises every branch listed in Requirements (both thethrow and success side of each), and the full
npm run test:coveragerun stays green. Do not add ascripts/**entry tocodecov.yml'sincludeas part of this change — that's a separate,repo-wide decision out of scope here.
Expected Outcome
A regression in
scripts/migrate-selfhost-sqlite-to-postgres.ts's SQL-generation, validation, orconflict-detection logic is caught by
npm run test:coveragelocally and in CI, instead of only beingdiscoverable by an operator running a real (potentially destructive) Postgres migration.
Links & Resources
scripts/migrate-selfhost-sqlite-to-postgres.ts(the file needing tests)test/unit/selfhost-pg-adapter.test.ts(the mock-Pool/PoolClientpattern to mirror)test/unit/selfhost-pg-queue.test.ts/test/unit/selfhost-pg-vectorize.test.ts(siblings using thesame mock convention)
test/helpers/d1.ts(existing in-memorynode:sqliteusage pattern in this repo's test suite)codecov.yml(confirmsscripts/**is excluded fromcoverage.include)