fix(boot): run pending data migrations on every server start

[atomantic]

Summary

• Existing installs hit Stage pipeline-volume-cover-concepts not found (and equivalent migration-shaped errors) on plain pull-and-restart because scripts/run-migrations.js was only invoked from update.sh — npm start, npm run dev, and pm2 restart all skipped it.
• Refactored the migration runner to export runMigrations({ rootDir } = {}) while keeping its CLI behavior, then awaited it at the top of server/index.js before createAIToolkit() reads stage-config.json.
• Idempotent: the applied-list short-circuit makes the no-pending case ~1ms (mkdir + read + readdir). Failures log via console.error with the full stack and let boot continue, matching the existing provider-warmup pattern at line 243.

Why

PR #262 shipped migration 017 for the missing stage entry but couldn't help users who didn't run update.sh — they restarted via pm2 restart or npm start and still saw the route error. Wiring the runner into server boot makes shipped migrations apply automatically on the next launch, regardless of how the process was restarted.

Implementation notes

• CLI guard is a synchronous URL-vs-URL comparison: import.meta.url === pathToFileURL(resolve(process.argv[1])).href. resolve() normalizes a possibly-relative argv[1] (the case when launched as node scripts/run-migrations.js) into the absolute path pathToFileURL requires; URL equality normalizes slash/case differences on Windows. Symlink chains aren't normalized — but the script is launched via node, not a bin shim, so there's no symlink in play in any of the documented entry points (npm run migrations, server boot, manual node ./scripts/...).
• The guard is intentionally kept synchronous so importing this module from server/index.js doesn't make it an async module or trigger filesystem I/O at evaluation time.
• MIGRATIONS_DIR is now a module-level const (was function-scoped) since both the exported function and the CLI shim share it.
• runMigrations() returns the number of applied migrations for callers that want it; current consumers ignore the return value.

Test plan

• Cold start with pending migrations: delete data/migrations.applied.json entries 015–019, run npm start, verify the server applies them on boot and pipeline-volume-cover-concepts resolves on the next request.
• Hot restart with everything applied: pm2 restart ecosystem.config.cjs → server logs ✅ No pending migrations and serves traffic.
• CLI still works: npm run migrations runs through migrations and exits 0 (CLI guard fires when invoked directly).
• Importing the module does not auto-execute (validated locally — await import('./scripts/run-migrations.js') yields exports only, no migration logs).

[Copilot]

Pull request overview

This PR ensures shipped data migrations are applied automatically during server boot, preventing upgraded installs from missing newly-added prompt stages/config (e.g., “Stage X not found”) when users restart via npm start, npm run dev, or pm2 restart without running update.sh.

Changes:

• Refactor scripts/run-migrations.js to export runMigrations({ rootDir } = {}) while preserving CLI behavior via an “invoked directly” guard.
• Invoke runMigrations() early in server/index.js startup, before the AI toolkit loads stage-config.json / providers.json.
• Document the behavior change in .changelog/NEXT.md.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
server/index.js	Runs migrations during boot before AI toolkit initialization to prevent missing-stage errors on restart.
scripts/run-migrations.js	Exposes a programmatic migration runner and adds a CLI-only execution guard.
.changelog/NEXT.md	Changelog entry describing auto-application of pending migrations on server start.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

scripts/run-migrations.js:16

• runMigrations() is now a shared module API and is invoked during server boot, but there are no automated tests covering the refactored runner behavior (e.g., applied-list handling, migration import selection, and the “does not auto-run when imported” contract). Consider adding a vitest unit test (can live under server/ and mock fs/promises + import() via a tiny fixture migration) to prevent regressions that could silently skip migrations or break startup.

export async function runMigrations({ rootDir = DEFAULT_ROOT_DIR } = {}) {
  const appliedFile = join(rootDir, 'data', 'migrations.applied.json');

  // Ensure data/ exists so we can persist applied state (migrationsDir
  // ships in the repo and always exists).
  await mkdir(dirname(appliedFile), { recursive: true });

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.