feat: schema-drift detection + dev auto-reconcile + os migrate (#2186)#2196
Merged
Conversation
The metadata→DB sync was additive-only (create table / add column), so any non-additive metadata change — relax `required`, change type/length, drop or rename a field — silently diverged from an existing database. The physical column won at write time, surfacing a misleading `organization_id is required` 400 even though `/meta` reported the field optional. Root-cause note: that 400 is NOT a validator bug. `record-validator` already reads `required` from metadata and passes once the field is optional; the 400 is a DB NOT NULL violation that `rest-server` paraphrases as "is required". So the fix is reconciling the physical schema + de-misleading the message, not changing the validator's source of truth. P1 — detection + warnings (schema-drift.ts, wired into SqlDriver.initObjects): diff metadata vs physical columns, categorise safe / needs_confirm / destructive (reusing SchemaDiffEntry). Boot warns once per divergence with an actionable `os migrate` hint. rest-server's NOT NULL translation now carries a drift-aware `hint` (VALIDATION_FAILED / fields envelope unchanged for back-compat). P2 — dev auto-reconcile (SqlDriverConfig.autoMigrate:'safe', wired into serve/dev in dev only): auto-applies the loosening subset (relax NOT NULL, widen varchar) so an existing dev DB self-heals on restart. Never destructive; force-disabled under NODE_ENV=production. P3 — os migrate plan/apply (cli/commands/migrate/*): categorised dry-run + reconcile, --allow-destructive gate, confirm prompt, --json. SQLite reconciles via the official table-rebuild (copy→swap, data preserved); Postgres/MySQL alter in place. Only examines objects in the loaded artifact; never auto-drops a table absent from metadata. Tests: driver-sql +12 (incl. #2178 repro self-heal with data preserved, prod guard, destructive gating, PG/MySQL DDL-gen), cli integration (real createStandaloneStack boot → detect → apply → in-sync), rest hint. Full build green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
github-actions
Bot
added
documentation
Contributor
📓 Docs Drift CheckThis PR changes 3 package(s): 23 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| @@ -0,0 +1,256 @@ | |||
| // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. | |||
|
|
|||
| import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'; | |||
xuyushun441-sys
added a commit
that referenced
this pull request
Jun 22, 2026
…ollow-up) (#2205) Follow-up to #2196 — the schema-drift / `os migrate` feature shipped with no docs. - cli.mdx: new "Schema migrations" section — `os migrate plan/apply`, the safe/needs-confirm/destructive categories, `--allow-destructive`, and the dev-only `autoMigrate:'safe'` self-heal (force-disabled in production). - objectstack-platform SKILL: add `os migrate plan/apply` to the CLI cheat sheet. - objectstack-data SKILL: "Schema evolution on an existing database" — the sync is additive-only; relaxing `required` / type changes / drops on a live DB need `os migrate` (dev auto-heals loosening). Names the tell-tale: `/meta` says optional but a write 400s "<field> is required" = stale NOT NULL, not a validator bug. Body-only skill edits (frontmatter unchanged → generated skill docs don't drift). Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2186.
Problem
The metadata→DB schema sync is additive-only: it creates tables and adds columns but never alters/drops existing ones. Any non-additive metadata change (relax
required, change type/length, drop/rename a field) silently diverges from an existing DB, and the physical column wins at write time.Diagnosis correction
The issue attributed the repro's
400 "organization_id is required"to a "two sources of truth" bug in the validator. It isn't:record-validatorreadsrequiredfrom metadata and passes once the field is optional. The 400 is a DBNOT NULLviolation thatrest-serverparaphrases as "is required". So the AC item "validation derives required from metadata" was already satisfied; the real fix is reconciling the physical schema + de-misleading the message.What's here
P1 — detection + warnings. New
schema-drift.tsdiffs metadata vs physical columns (reusingSchemaDiffEntry), categorising safe / needs_confirm / destructive.SqlDriver.initObjectswarns once per divergence with an actionableos migratehint.rest-server's NOT NULL branch now carries a drift-awarehint— theVALIDATION_FAILED/fieldsenvelope is unchanged for back-compat.P2 — dev auto-reconcile. New
SqlDriverConfig.autoMigrate: 'safe'(wired intoserve/devin dev only) auto-applies the loosening subset (relaxNOT NULL, widen varchar) so an existing dev DB self-heals on restart. Never destructive; force-disabled underNODE_ENV=production.P3 —
os migrate plan/apply. Categorised dry-run + reconcile,--allow-destructivegate, confirm prompt,--json. SQLite reconciles via the official table-rebuild (copy→swap, data preserved); Postgres/MySQL alter in place. Only examines objects in the loaded artifact (runos buildfirst); never auto-drops a table absent from metadata.Acceptance criteria
required:true→falsereflects in an existing DB after restart, no reset (P2)os migrate plan/applyreconciles non-additive changes with preview + destructive guardrail (P3)requiredfrom metadata — already true; reframed (see diagnosis)Tests
createStandaloneStackboot → detect → apply → in-syncCaveats
ALTERpath is standard SQL but validated via generated-DDL assertions (no live PG in CI — matches the repo's SQLite-only DDL test posture).os migratedestructive ops assume the full app/plugin set is loaded; a column that looks "orphaned" under a partial build is reported but a table absent from metadata is never dropped.🤖 Generated with Claude Code