fix(migrator): handle orphan versions in shared dev databases (#2780)

.ai/wheels/troubleshooting/shared-dev-databases.md

@@ -0,0 +1,50 @@
+# Shared Development Databases
+
+Short reference for the orphan-migration case. User-facing version lives at
+`web/sites/guides/src/content/docs/v4-0-0/basics/shared-development-databases.mdx`.
+
+## What's an orphan?
+
+A row in `wheels_migrator_versions` whose `version` timestamp has no matching
+file in `app/migrator/migrations/`. Common cause: shared dev DB, peer ran
+their migration first against the shared DB before their file was merged.
+
+## Detection
+
+`Migrator.cfc::$getOrphanVersions()` — returns an array of orphan version
+strings, sorted ascending. Excludes the sentinel `"0"` returned when the
+tracking table is empty.
+
+## Display
+
+`wheels migrate info` marks orphan rows with `[?]` and the literal
+`********** NO FILE **********` (Rails-style). Includes a footer
+explaining the cause. Rendering logic lives in
+`Migrator.cfc::$buildInfoOutput()` so it's unit-testable without the HTTP
+dispatcher.
+
+## Behavior in `migrateTo()`
+
+If `currentVersion > target` ONLY because of orphans (no local file with
+version > target marked migrated), the down branch is skipped. Either:
+
+- Pending local migrations exist → fall through to up branch with a
+  warning naming the orphans
+- Nothing pending → emit "Nothing to do" naming current vs target and
+  return immediately
+
+If SOME DB versions > target are orphans and SOME have local files, the
+down branch runs as usual but emits a warning naming the orphans (they
+get skipped by the existing loop because it iterates files only).
+
+## Related
+
+- Issue #2780 (the original report)
+- PR #2798 (the fix)
+- `vendor/wheels/Migrator.cfc::$getOrphanVersions()`
+- `vendor/wheels/Migrator.cfc::$buildInfoOutput()`
+- `vendor/wheels/tests/specs/migrator/OrphanDetectionSpec.cfc`
+- `vendor/wheels/tests/specs/migrator/MigratorInfoSpec.cfc`
+- Follow-up work (separate PRs):
+  - `wheels migrate doctor` / `forget` / `pretend` for manual reconciliation
+  - Schema enrichment of `wheels_migrator_versions` (add `name` and `applied_at` columns)

CHANGELOG.md

@@ -22,6 +22,7 @@ All historical references to "CFWheels" in this changelog have been preserved fo
 
 ### Fixed
 
+- `wheels migrate latest` no longer takes a misleading "down" branch and silently no-ops when `wheels_migrator_versions` records a version whose migration file isn't in the current checkout (shared dev DB / peer migration not yet pulled). `Migrator.migrateTo()` now diffs the tracking table against `app/migrator/migrations/` via the new `$getOrphanVersions()` helper and branches on orphan-at-top before the existing direction check — applying pending local migrations with a clear warning when all DB versions above target are orphans, emitting "Nothing to do" naming target vs current when none are pending, and warning + letting the existing down loop continue in mixed cases (orphan rows skip naturally because the loop iterates files). `wheels migrate info` also now shows orphan rows with `[?] <version> ********** NO FILE **********` (Rails-style) and a footer explaining the cause (#2780)
 - `tools/test-local.sh` silently aborted with `EXIT=1` (no `/tmp/wheels-test-server.log` written, no diagnostic printed) on every install since the `lucli` → `wheels` rebrand window closed — i.e. anyone whose `~/.lucli/express/` directory never existed. Line 81 ran `LUCEE_LIB=$(find ~/.wheels/express ~/.lucli/express -path "*/lib/ext" -type d 2>/dev/null | head -1)` under `set -euo pipefail`; `find` exits non-zero whenever any path argument doesn't exist (stderr suppressed via `2>/dev/null`, but the exit status survives), `pipefail` propagated it through `head -1`, and the assignment tripped `set -e`. The cleanup trap then fired with no server to clean up, leaving the user staring at "Starting Wheels CLI server on port 8080…" with no further output. Dropped the now-dead `~/.lucli/express` fallback (the rename landed in 3.0 and recent CLI releases extract Lucee Express to `~/.wheels/express/` only) and added `|| true` for defense in depth so a missing directory (e.g. a truly fresh install before `wheels start` has ever run) leaves `LUCEE_LIB` empty and the downstream `[ -n "$LUCEE_LIB" ]` guard skips the JDBC pre-install cleanly (#2796)
 - Routes registered inside `.namespace("foo")` (or equivalent `.scope()` / `.package()`) with a redundant namespace prefix in the controller path — e.g. `to="foo/dashboard##index"` instead of `to="dashboard##index"` — previously silently produced a `foo.foo/dashboard` lookup that downstream flattened to a `Foodashboard`-style class name with an opaque `Wheels.ViewNotFound` error. The Mapper now rejects this at route-registration time with `Wheels.MapperArgumentInvalid`, naming the namespace and the offending value and pointing at the correct shorter form, so users can find the bad route definition instead of chasing the symptom (#2791)
 - `WheelsTest` auto-bind missed user-defined global helpers added via `include` in `app/global/functions.cfm`. The pseudo-constructor used `getMetaData(application.wo).functions`, which only enumerates methods declared directly on the CFC and skips symbols merged in via `cfinclude`. Specs that called custom helpers (e.g. `can()`, `hasRole()`) had to manually rebind each one in `beforeAll()`. The auto-bind now iterates `application.wo` as a struct and binds every UDF via `isCustomFunction()`, preserving the existing public-only filter for declared methods (#2790)

vendor/wheels/Migrator.cfc

@@ -39,14 +39,71 @@ component output="false" extends="wheels.Global"{
 			}
 		}
 
+		// Issue #2780: detect orphan versions (DB rows whose timestamp has no
+		// matching local file). Common in shared dev DBs when a peer applied
+		// a migration whose file isn't yet in this branch. Without this
+		// check, the directional logic below silently took the "down" branch
+		// and emitted a misleading "Migrating from X down to Y" output.
+		local.orphans = $getOrphanVersions();
+		local.orphansAboveTarget = [];
+		for (local.v in local.orphans) {
+			if (local.v > arguments.version) {
+				ArrayAppend(local.orphansAboveTarget, local.v);
+			}
+		}
+		local.isOrphanAtTop = (
+			local.currentVersion > arguments.version
+			&& ArrayLen(local.orphansAboveTarget)
+			&& !arguments.missingMigFlag
+		);
+		if (local.isOrphanAtTop) {
+			// Check whether EVERY DB version above target is an orphan. If
+			// some have local files, the down branch is legitimate (the user
+			// has files to run down() on) — we still emit a warning naming
+			// the orphans but otherwise proceed as before.
+			local.dbVersionsAboveTarget = [];
+			for (local.v in ListToArray($getVersionsPreviouslyMigrated())) {
+				if (Len(local.v) && local.v != "0" && local.v > arguments.version) {
+					ArrayAppend(local.dbVersionsAboveTarget, local.v);
+				}
+			}
+			local.allOrphans = ArrayLen(local.dbVersionsAboveTarget) == ArrayLen(local.orphansAboveTarget);
+			if (local.allOrphans) {
+				local.rv = "Note: database tracks version(s) "
+					& ArrayToList(local.orphansAboveTarget, ", ")
+					& " with no matching file in app/migrator/migrations/. "
+					& "This usually means a peer applied a migration whose "
+					& "file isn't yet in your branch.#Chr(13) & Chr(10)#";
+				if (!local.hasPendingMigrations) {
+					local.rv &= "Nothing to do. Your target version ("
+						& arguments.version
+						& ") is below the database's current version ("
+						& local.currentVersion & ").#Chr(13) & Chr(10)#";
+					return local.rv;
+				}
+				// Suppress the down branch: rewrite currentVersion so the
+				// outer conditional falls through to the up branch, which
+				// applies any actually-pending local migrations.
+				local.currentVersion = arguments.version;
+			} else {
+				// Mixed case: some legitimate down candidates, some orphans.
+				// Warn but let the existing down branch handle the rest;
+				// orphan rows are skipped naturally because the loop only
+				// iterates local files.
+				local.rv = "Note: database tracks version(s) "
+					& ArrayToList(local.orphansAboveTarget, ", ")
+					& " with no matching file. These will be skipped during rollback.#Chr(13) & Chr(10)#";
+			}
+		}
+
 		if (local.currentVersion == arguments.version && !local.hasPendingMigrations) {
-			local.rv = "Database is currently at version #arguments.version#. No migration required.#Chr(13) & Chr(10)#";
+			local.rv &= "Database is currently at version #arguments.version#. No migration required.#Chr(13) & Chr(10)#";
 		} else {
 			if (!DirectoryExists(this.paths.sql) && application[local.appKey].writeMigratorSQLFiles) {
 				DirectoryCreate(this.paths.sql);
 			}
 			if (local.currentVersion > arguments.version && arguments.missingMigFlag == false) {
-				local.rv = "Migrating from #local.currentVersion# down to #arguments.version#.#Chr(13) & Chr(10)#";
+				local.rv &= "Migrating from #local.currentVersion# down to #arguments.version#.#Chr(13) & Chr(10)#";
 				for (local.i = ArrayLen(local.migrations); local.i >= 1; local.i--) {
 					local.migration = local.migrations[local.i];
 					if (local.migration.version <= arguments.version) {
@@ -79,7 +136,7 @@ component output="false" extends="wheels.Global"{
 				}
 			} else {
 				if(arguments.missingMigFlag){
-					local.rv = "Migrating remaining migrations till #arguments.version#.#Chr(13) & Chr(10)#";
+					local.rv &= "Migrating remaining migrations till #arguments.version#.#Chr(13) & Chr(10)#";
 					$removeVersionAsMigrated(local.currentVersion);
 				} else if (local.currentVersion gte arguments.version && local.hasPendingMigrations) {
 					// Out-of-order pending migrations: a migration with a
@@ -89,9 +146,9 @@ component output="false" extends="wheels.Global"{
 					// generator's current-day timestamp). The "from N up to N"
 					// framing reads as a no-op even though new migrations are
 					// about to run, so emit a clearer message. Onboarding F16.
-					local.rv = "Applying pending migration(s) up to #arguments.version#.#Chr(13) & Chr(10)#";
+					local.rv &= "Applying pending migration(s) up to #arguments.version#.#Chr(13) & Chr(10)#";
 				} else {
-					local.rv = "Migrating from #local.currentVersion# up to #arguments.version#.#Chr(13) & Chr(10)#";
+					local.rv &= "Migrating from #local.currentVersion# up to #arguments.version#.#Chr(13) & Chr(10)#";
 				}
 				for (local.migration in local.migrations) {
 					if (local.migration.version <= arguments.version && local.migration.status != "migrated") {
@@ -532,6 +589,104 @@ component output="false" extends="wheels.Global"{
 		}
 	}
 
+	/**
+	 * Returns versions recorded in the tracking table that have no matching
+	 * migration file in the current checkout. Used to detect the "shared dev
+	 * database" case where a peer has applied a migration whose file isn't
+	 * yet in the local branch. See issue #2780.
+	 *
+	 * Result is sorted ascending. The sentinel "0" returned by
+	 * $getVersionsPreviouslyMigrated() on an empty tracking table is excluded.
+	 *
+	 * [section: Migrator]
+	 * [category: General Functions]
+	 */
+	public array function $getOrphanVersions() {
+		local.appliedList = ListToArray($getVersionsPreviouslyMigrated());
+		local.fileVersions = [];
+		for (local.m in getAvailableMigrations()) {
+			ArrayAppend(local.fileVersions, local.m.version);
+		}
+		local.orphans = [];
+		for (local.v in local.appliedList) {
+			if (Len(local.v) && local.v != "0" && !ArrayFind(local.fileVersions, local.v)) {
+				ArrayAppend(local.orphans, local.v);
+			}
+		}
+		ArraySort(local.orphans, function(a, b) {
+			return Compare(a, b);
+		});
+		return local.orphans;
+	}
+
+	/**
+	 * Builds the human-readable info output for `wheels migrate info`.
+	 * Returns an array of lines (caller joins with newlines). Extracted
+	 * from cli.cfm's info handler so the rendering can be unit-tested
+	 * without exercising the HTTP dispatcher. Orphan rows (DB versions
+	 * with no matching local file — see issue #2780) are marked with
+	 * [?] and the literal "********** NO FILE **********", Rails-style.
+	 *
+	 * [section: Migrator]
+	 * [category: General Functions]
+	 */
+	public array function $buildInfoOutput() {
+		local.lines = [];
+		local.migrations = getAvailableMigrations();
+		local.currentVersion = getCurrentMigrationVersion();
+		local.orphans = $getOrphanVersions();
+		local.applied = 0;
+		local.pending = 0;
+		for (local.m in local.migrations) {
+			if (local.m.status == "migrated") {
+				local.applied++;
+			} else {
+				local.pending++;
+			}
+		}
+		ArrayAppend(local.lines, "Current version: " & (Len(local.currentVersion) ? local.currentVersion : "0"));
+		ArrayAppend(local.lines, "Total migrations: " & ArrayLen(local.migrations));
+		if (ArrayLen(local.migrations) || ArrayLen(local.orphans)) {
+			ArrayAppend(local.lines, "  applied: " & local.applied);
+			ArrayAppend(local.lines, "  pending: " & local.pending);
+			if (ArrayLen(local.orphans)) {
+				ArrayAppend(local.lines, "  orphan: " & ArrayLen(local.orphans));
+			}
+			ArrayAppend(local.lines, "");
+			ArrayAppend(local.lines, "Migrations (newest last):");
+			// Merge file rows + orphan rows into one chronological list so
+			// orphans appear in the right position relative to local files.
+			local.combined = [];
+			for (local.m in local.migrations) {
+				ArrayAppend(local.combined, {
+					version: local.m.version,
+					name: local.m.name,
+					marker: local.m.status == "migrated" ? "[x]" : "[ ]"
+				});
+			}
+			for (local.v in local.orphans) {
+				ArrayAppend(local.combined, {
+					version: local.v,
+					name: "********** NO FILE **********",
+					marker: "[?]"
+				});
+			}
+			ArraySort(local.combined, function(a, b) {
+				return Compare(a.version, b.version);
+			});
+			for (local.row in local.combined) {
+				ArrayAppend(local.lines, "  " & local.row.marker & " " & local.row.version & " " & local.row.name);
+			}
+			if (ArrayLen(local.orphans)) {
+				ArrayAppend(local.lines, "");
+				ArrayAppend(local.lines, "Orphan versions are recorded in the database but have no");
+				ArrayAppend(local.lines, "matching file in app/migrator/migrations/. This usually means");
+				ArrayAppend(local.lines, "a peer applied a migration whose file isn't yet in your branch.");
+			}
+		}
+		return local.lines;
+	}
+
 	/**
 	 * F15 Phase 1: detect which system-table naming family this app's database
 	 * already uses, and flip the configured names if needed.

vendor/wheels/public/views/cli.cfm

@@ -157,33 +157,17 @@ try {
 				data.message = migrator.redoMigration(local.redoVersion);
 				break;
 			case "info":
-				// Build a human-readable status block from the data already
-				// populated above (currentVersion, migrations, datasource).
-				// Without this, the CLI's `wheels migrate info` printed an
-				// empty success message (issue #2474).
+				// Build a human-readable status block. The migrations list
+				// is rendered by Migrator.$buildInfoOutput() so the logic
+				// is unit-testable without exercising the HTTP dispatcher.
+				// Issue #2780 surfaced orphan versions (DB rows with no
+				// matching file) — those are rendered with a [?] marker
+				// and an explanatory footer.
 				local.lines = [];
 				ArrayAppend(local.lines, "Datasource: " & data.datasource);
 				ArrayAppend(local.lines, "Database type: " & data.databaseType);
-				ArrayAppend(local.lines, "Current version: " & (Len(data.currentVersion) ? data.currentVersion : "0"));
-				ArrayAppend(local.lines, "Total migrations: " & ArrayLen(data.migrations));
-				if (ArrayLen(data.migrations)) {
-					local.applied = 0;
-					local.pending = 0;
-					for (local.m in data.migrations) {
-						if (local.m.status == "migrated") {
-							local.applied++;
-						} else {
-							local.pending++;
-						}
-					}
-					ArrayAppend(local.lines, "  applied: " & local.applied);
-					ArrayAppend(local.lines, "  pending: " & local.pending);
-					ArrayAppend(local.lines, "");
-					ArrayAppend(local.lines, "Migrations (newest last):");
-					for (local.m in data.migrations) {
-						local.marker = local.m.status == "migrated" ? "[x]" : "[ ]";
-						ArrayAppend(local.lines, "  " & local.marker & " " & local.m.version & " " & local.m.name);
-					}
+				for (local.line in migrator.$buildInfoOutput()) {
+					ArrayAppend(local.lines, local.line);
 				}
 				data.message = ArrayToList(local.lines, Chr(10));
 				break;

vendor/wheels/tests/specs/migrator/MigratorInfoSpec.cfc

@@ -0,0 +1,84 @@
+component extends="wheels.WheelsTest" {
+
+	include "helperFunctions.cfm";
+
+	function beforeAll() {
+		migration = CreateObject("component", "wheels.migrator.Migration").init();
+		migrator = CreateObject("component", "wheels.Migrator").init(
+			migratePath = "/wheels/tests/_assets/migrator/migrations/",
+			sqlPath = "/wheels/tests/_assets/migrator/sql/"
+		);
+	}
+
+	function run() {
+
+		var _isCockroachDB = CreateObject("component", "wheels.migrator.Migration").init().adapter.adapterName() == "CockroachDB";
+
+		var insertOrphan = function(required string version) {
+			queryExecute(
+				"INSERT INTO #application.wheels.migratorTableName# (version, core_level) VALUES ('#arguments.version#', #application.wheels.migrationLevel#)",
+				{},
+				{ datasource = application.wheels.dataSourceName }
+			);
+		};
+
+		describe("Migrator info output", () => {
+
+			beforeEach(() => {
+				for (local.table in ["c_o_r_e_bunyips", "c_o_r_e_dropbears", "c_o_r_e_hoopsnakes"]) {
+					try { migration.dropTable(local.table); } catch (any e) {}
+				}
+				deleteMigratorVersions(2);
+			});
+
+			afterEach(() => {
+				for (local.table in ["c_o_r_e_bunyips", "c_o_r_e_dropbears", "c_o_r_e_hoopsnakes"]) {
+					try { migration.dropTable(local.table); } catch (any e) {}
+				}
+				deleteMigratorVersions(2);
+			});
+
+			it("$buildInfoOutput returns expected lines for a clean state", () => {
+				if (_isCockroachDB) return;
+				migrator.migrateTo("002");
+				var lines = migrator.$buildInfoOutput();
+				expect(lines).toBeArray();
+				var joined = ArrayToList(lines, Chr(10));
+				expect(joined).toInclude("Current version:");
+				expect(joined).toInclude("[x] 001");
+				expect(joined).toInclude("[x] 002");
+				expect(joined).toInclude("[ ] 003");
+			});
+
+			it("$buildInfoOutput marks orphan versions with [?] and NO FILE", () => {
+				if (_isCockroachDB) return;
+				migrator.migrateTo("001");
+				insertOrphan("999");
+				var lines = migrator.$buildInfoOutput();
+				var joined = ArrayToList(lines, Chr(10));
+				expect(joined).toInclude("[?] 999");
+				expect(joined).toInclude("NO FILE");
+			});
+
+			it("$buildInfoOutput summary counts orphans separately", () => {
+				if (_isCockroachDB) return;
+				migrator.migrateTo("001");
+				insertOrphan("999");
+				var lines = migrator.$buildInfoOutput();
+				var joined = ArrayToList(lines, Chr(10));
+				expect(joined).toInclude("orphan: 1");
+			});
+
+			it("$buildInfoOutput omits orphan summary line when no orphans exist", () => {
+				if (_isCockroachDB) return;
+				migrator.migrateTo("001");
+				var lines = migrator.$buildInfoOutput();
+				var joined = ArrayToList(lines, Chr(10));
+				expect(joined).notToInclude("orphan:");
+			});
+
+		});
+
+	}
+
+}