feat(types,cli,verify)!: 只解析 host app 声明过的包 —— NODE_PATH 不再算数,ADR-0093 D5 的墙与启动方式脱钩 (#4719)

.changeset/host-declared-package-resolution.md

@@ -0,0 +1,64 @@
+---
+"@objectstack/types": minor
+"@objectstack/cli": minor
+"@objectstack/verify": minor
+---
+
+feat(types,cli,verify)!: 只解析 host app 声明过的包 —— `NODE_PATH` 不再算数,ADR-0093 D5 那道墙从此与启动方式无关 (#4719)
+
+**问题:契约写下了,但从没被检查过。** `@objectstack/types/node` 的
+`createHostRequire` 返回一个 CJS `createRequire`,而 CJS 解析认 `NODE_PATH`
+(`Module.globalPaths`)。pnpm 生成的 bin shim 第一件事就是
+`export NODE_PATH=<workspace>/node_modules/.pnpm/node_modules`,于是任何被工作区里
+**任意一个包**传递依赖到的包都能"从 host app 解析成功" —— 跟这个 app 声明了什么毫无关系。
+
+实测(cloud `apps/objectos-ee`,当时未声明 `@objectstack/organizations`):
+`pnpm start`(经 shim)boot 成功、插件表里有 `Organizations`、ADR-0093 D5 一声不吭;
+`node node_modules/@objectstack/cli/bin/run.js serve`(不经 shim)则
+`✖ FATAL: tenancy posture 'isolated' was requested…` 并 exit 1。同一个 app、同一份
+`package.json`、同一个 posture,**只因为进程是怎么被拉起来的**,走出两种结果。
+而 D5 的报错一直在教 operator "declare it in the app's package.json" —— 那正是
+CLI 从来没检查过的那件事。
+
+**改法:声明即执行。** 解析前先读 `<hostRoot>/package.json`;只有包名出现在
+`dependencies` / `devDependencies` / `optionalDependencies` / `peerDependencies`
+的 **键**里,才去 host 的 `node_modules` 里查它。仅仅"能被解析到"不再算数 ——
+那正是让契约失效的那个偶然。未声明的包退回到 importing package 自身的解析
+(ESM,不认 `NODE_PATH`),框架自有的包加载路径不受影响。
+
+**两种失败从此分开报。** 今天它们都塌成同一条 `MODULE_NOT_FOUND`,补救办法却相反:
+
+- **未声明** —— 指向"在 app 的 `package.json` 里声明并安装",并说明为什么
+  hoisting / `NODE_PATH` 不被接受;
+- **声明了但解析不到** —— 明确说这是**安装**问题(`pnpm install`、生产 prune
+  砍掉了它、dist 没构建),别再让人回去重看那份已经写对的 `package.json`。
+
+分类经新导出的 `hostImportFailureKind(err)` 暴露给调用方;两种错误都仍带
+`code: 'MODULE_NOT_FOUND'`,`isModuleNotFoundError` 的既有判定不变。
+
+**BREAKING — 哪类部署会从假绿变红,以及怎么修。**
+
+1. **靠 hoisting 苟着的部署。** 一个 app 请求了 walled tenancy posture
+   (`OS_TENANCY_POSTURE=group` / `isolated` 或 `OS_MULTI_ORG_ENABLED=1`)、
+   却没在自己的 `package.json` 里声明 `@objectstack/organizations`,过去经 pnpm
+   shim 启动能正常 boot —— 现在会命中 ADR-0093 D5 并 exit 1。
+   **修法:在那个 app 的 `package.json` 里声明该依赖并安装。**
+   这些部署本来就在未声明状态下运行,红的是一直存在的事实,不是新引入的故障:
+   同一个 app 不经 shim 启动今天就已经是 exit 1。
+   (同样适用于 `@objectstack/service-ai` / `@objectstack/service-ai-studio`,以及
+   `bootStack({ multiTenant: true })`、dogfood 的 enterprise 门。)
+
+2. **`createHostImporter` 的签名变了**,因为它现在需要 host 的**根目录**才能读到
+   那份 manifest,而一个 `NodeRequire` 无法被问出它锚在哪里:
+
+   ```diff
+   - createHostImporter(createHostRequire(hostRoot))
+   + createHostImporter(hostRoot)                     // 省略参数 = process.cwd(),同旧默认
+   ```
+
+   `createHostRequire` 本身保持不变,仍然导出。
+
+新增导出(`@objectstack/types/node`):`HOST_DECLARATION_FIELDS`、
+`HostDeclarationField`、`HostDeclaration`、`readHostDeclaration`、
+`isDeclaredByHost`、`packageNameFromSpecifier`、`HostImportFailureKind`、
+`HOST_IMPORT_FAILURE_KIND`、`hostImportFailureKind`。

packages/cli/src/commands/serve.ts

@@ -20,7 +20,12 @@ import { graftAuthoredRuntimeMembers, isAppPluginLike } from '../utils/graft-run
 import { redactConnectionUrl, describeDriverConnection } from '../utils/connection-display.js';
 // Shared with @objectstack/verify and the dogfood multi-org probes (#4700) —
 // node-only, hence the `/node` subpath rather than the edge-safe root export.
-import { createHostRequire, createHostImporter } from '@objectstack/types/node';
+import {
+  createHostImporter,
+  hostImportFailureKind,
+  isDeclaredByHost,
+  readHostDeclaration,
+} from '@objectstack/types/node';
 import {
   printHeader,
   printKV,
@@ -1544,14 +1549,22 @@ export default class Serve extends Command {
       // instead; the CLI's own resolution stays as the fallback for the
       // framework-owned packages the CLI depends on.
       //
+      // #4719: "resolve from the host root" now means "resolve what the host
+      // root DECLARES". The host lookup was a CJS require, CJS honours
+      // NODE_PATH, and the pnpm bin shim exports NODE_PATH pointing at the
+      // hoisted workspace store — so anything transitively reachable from
+      // anywhere in the workspace resolved as if the app had declared it, and
+      // whether the D5 wall below fired came down to whether `serve` was reached
+      // through that shim. The declaration is the contract; reachability is not.
+      //
       // Defined HERE, above the auth block, because the enterprise organizations
       // load inside it needs it: this helper used to be declared *after* that
       // block, so the organizations load fell back to a bare import, resolved in
       // the framework workspace, never found the cloud-private package, and every
       // walled-posture deployment hit the ADR-0093 D5 fail-fast and exited 1
       // (cloud#1013).
-      const hostRequire = createHostRequire();
-      const importFromHost = createHostImporter(hostRequire);
+      const hostRoot = process.cwd();
+      const importFromHost = createHostImporter(hostRoot);
 
       // 5d. Auto-register AuthPlugin (and paired Security/Audit) when the
       // 'auth' tier is enabled and no auth plugin is already configured.
@@ -1795,16 +1808,34 @@ export default class Serve extends Command {
                 // exact footgun this guard closes.
                 const cause = orgErr instanceof Error ? orgErr.message : String(orgErr);
                 if (!resolveAllowDegradedTenancy()) {
+                  // #4719 — TWO ABSENCES, TWO REMEDIES. Until the host lookup was
+                  // gated on the host's declaration, both arrived here as one
+                  // MODULE_NOT_FOUND and got one piece of advice: "declare it in
+                  // the app's package.json". For an operator who HAD declared it
+                  // and whose install was pruned, that sent them to re-read a
+                  // file that was already correct. The importer now says which
+                  // one it is, so this text can too.
+                  const declaration = readHostDeclaration('@objectstack/organizations', hostRoot);
+                  const remedy =
+                    hostImportFailureKind(orgErr) === 'declared-unresolvable'
+                      ? '      • this app DECLARES @objectstack/organizations ' +
+                        `(${declaration.field}: ${JSON.stringify(declaration.specifier)}) — the\n` +
+                        '        declaration is NOT the problem and re-reading package.json will not help.\n' +
+                        `        Repair the INSTALL in ${hostRoot}: run \`pnpm install\`, check that a\n` +
+                        '        production prune did not drop it, and that its dist is actually built — or\n'
+                      : '      • add @objectstack/organizations (the enterprise multi-org runtime) to THIS APP\n' +
+                        "        — declare it in the app's package.json and install; the CLI resolves it from the\n" +
+                        '          app, not from the framework it is linked out of. Being merely reachable\n' +
+                        '          through NODE_PATH / a hoisted workspace store is deliberately not enough\n' +
+                        '          (#4719) — that made this wall depend on how the process was launched — or\n';
                   console.error(
                     chalk.red(
                       `\n  ✖ FATAL: tenancy posture '${tenancyPosture}' was requested but ` +
                         '@objectstack/organizations could not be loaded,\n' +
                         '    so the organization wall is INACTIVE. Refusing to boot — a deployment that requested\n' +
                         '    multi-organization isolation must not serve traffic without it (ADR-0093 D5).\n\n' +
                         '    Fix one of:\n' +
-                        '      • add @objectstack/organizations (the enterprise multi-org runtime) to THIS APP\n' +
-                        "        — declare it in the app's package.json and install; the CLI resolves it from the\n" +
-                        '          app, not from the framework it is linked out of — or\n' +
+                        remedy +
                         "      • set OS_TENANCY_POSTURE=single (or unset OS_MULTI_ORG_ENABLED) to run single-org, or\n" +
                         '      • set OS_ALLOW_DEGRADED_TENANCY=1 to boot in an explicitly degraded single-org state.\n\n' +
                         `    cause: ${cause}\n`,
@@ -2031,20 +2062,14 @@ export default class Serve extends Command {
       // surface), while MCP and every other capability are unaffected. Gating on
       // a *declared* dep — not mere resolvability — makes this reliable in a
       // workspace/monorepo, where the package stays hoist-resolvable when undeclared.
-      const _fs = await import('node:fs');
-      const hostDeclaresDependency = (pkg: string): boolean => {
-        try {
-          const hostPkg = JSON.parse(
-            _fs.readFileSync(hostRequire.resolve('./package.json'), 'utf8'),
-          ) as Record<string, Record<string, string> | undefined>;
-          return Boolean(
-            hostPkg.dependencies?.[pkg] ?? hostPkg.devDependencies?.[pkg]
-              ?? hostPkg.optionalDependencies?.[pkg] ?? hostPkg.peerDependencies?.[pkg],
-          );
-        } catch {
-          return false;
-        }
-      };
+      //
+      // #4719 — this used to be a local re-implementation of that read. It was
+      // right, and it was the ONLY place in the boot path that asked the question
+      // the right way: the enterprise organizations load two blocks up asked
+      // "does it resolve", which a hoisted store answered yes to regardless. Both
+      // now go through the one owner in `@objectstack/types/node`, so "declared"
+      // cannot mean two different things in one file (Prime Directive #12).
+      const hostDeclaresDependency = (pkg: string): boolean => isDeclaredByHost(pkg, hostRoot);
       // `wantsAiService` is the AUTO (opt-in) signal: the host app listed the base
       // AI service — or the Studio that builds on it — in its OWN package.json. This
       // is a package.json READ (a deliberate authoring act), not a speculative

packages/cli/test/serve-organizations-host-resolution.e2e.test.ts

@@ -76,6 +76,35 @@ export class OrganizationsPlugin {
 let appWithPackage: string;
 /** The same app WITHOUT it — the fail-fast must still fire. */
 let appWithoutPackage: string;
+/**
+ * #4719 — an app that declares NOTHING, run with `NODE_PATH` pointing at a
+ * store that carries the enterprise package. This is not a contrivance: it is
+ * verbatim what a pnpm bin shim does before it execs the CLI —
+ *
+ *     export NODE_PATH="<workspace>/node_modules/.pnpm/node_modules"
+ *
+ * — and CJS resolution honours it, so `serve` used to boot a walled posture for
+ * an app that had never asked for the multi-org runtime. Measured on cloud's
+ * `apps/objectos-ee`: `pnpm start` (through the shim) booted silently, while
+ * `node …/@objectstack/cli/bin/run.js serve` on the same app hit D5 and exited 1.
+ */
+let hoistedStore: string;
+
+function writeOrganizationsPackage(root: string): void {
+  const pkgDir = join(root, '@objectstack', 'organizations');
+  mkdirSync(pkgDir, { recursive: true });
+  writeFileSync(
+    join(pkgDir, 'package.json'),
+    JSON.stringify({
+      name: '@objectstack/organizations',
+      version: '0.0.0-fixture',
+      type: 'module',
+      main: 'index.js',
+    }),
+    'utf8',
+  );
+  writeFileSync(join(pkgDir, 'index.js'), FAKE_ORGANIZATIONS, 'utf8');
+}
 
 function writeApp(prefix: string, opts: { withOrganizations: boolean }): string {
   const dir = mkdtempSync(join(tmpdir(), prefix));
@@ -96,31 +125,19 @@ function writeApp(prefix: string, opts: { withOrganizations: boolean }): string
     ),
     'utf8',
   );
-  if (opts.withOrganizations) {
-    const pkgDir = join(dir, 'node_modules', '@objectstack', 'organizations');
-    mkdirSync(pkgDir, { recursive: true });
-    writeFileSync(
-      join(pkgDir, 'package.json'),
-      JSON.stringify({
-        name: '@objectstack/organizations',
-        version: '0.0.0-fixture',
-        type: 'module',
-        main: 'index.js',
-      }),
-      'utf8',
-    );
-    writeFileSync(join(pkgDir, 'index.js'), FAKE_ORGANIZATIONS, 'utf8');
-  }
+  if (opts.withOrganizations) writeOrganizationsPackage(join(dir, 'node_modules'));
   return dir;
 }
 
 beforeAll(() => {
   appWithPackage = writeApp('os-org-host-ok-', { withOrganizations: true });
   appWithoutPackage = writeApp('os-org-host-missing-', { withOrganizations: false });
+  hoistedStore = mkdtempSync(join(tmpdir(), 'os-org-hoisted-store-'));
+  writeOrganizationsPackage(hoistedStore);
 });
 
 afterAll(() => {
-  for (const dir of [appWithPackage, appWithoutPackage]) {
+  for (const dir of [appWithPackage, appWithoutPackage, hoistedStore]) {
     if (dir) rmSync(dir, { recursive: true, force: true });
   }
 });
@@ -184,4 +201,38 @@ describe('os serve — enterprise organizations resolution (cloud#1013)', () =>
     },
     300_000,
   );
+
+  it(
+    'refuses when the package is reachable only through NODE_PATH — the pnpm shim shape (#4719)',
+    async () => {
+      // The #4719 defect, over a real process, with the launcher reproduced
+      // exactly. Same app as the case above (declares nothing), same posture —
+      // the only difference is the NODE_PATH every pnpm bin shim exports. Before
+      // this change that single environment variable was enough to boot the
+      // organization wall off a package the app had never declared, so whether
+      // ADR-0093 D5 fired came down to HOW the process was started.
+      const port = randomPort();
+      const { stdout, stderr } = await runServe(appWithoutPackage, ['--port', port], {
+        waitFor: /Press Ctrl\+C to stop/,
+        env: { ...SERVE_ENV, NODE_PATH: hoistedStore },
+        timeoutMs: 240_000,
+      });
+
+      const seen = `\n--- stdout ---\n${stdout.slice(-4000)}\n--- stderr ---\n${stderr.slice(-4000)}`;
+      expect(
+        stderr,
+        `NODE_PATH got an undeclared app past the D5 wall — the #4719 defect${seen}`,
+      ).toMatch(/FATAL: tenancy posture 'isolated' was requested/);
+      // …and the remedy is the declaration one, naming why reachability lost.
+      expect(stderr).toMatch(/to THIS APP/);
+      expect(stderr).toMatch(/NODE_PATH/);
+      expect(stdout, `serve served traffic without the wall${seen}`).not.toContain(
+        'Press Ctrl+C to stop',
+      );
+      expect(stdout, `the hoisted package was mounted anyway${seen}`).not.toContain(
+        'Organizations',
+      );
+    },
+    300_000,
+  );
 });

packages/qa/dogfood/test/enterprise-organizations.test.ts

@@ -22,16 +22,28 @@ import { probeOrganizations, MULTI_ORG_ENV, ORGANIZATIONS_PKG } from './enterpri
 
 let hostWithPkg: string;
 let hostWithoutPkg: string;
+/**
+ * #4719 — the package is physically INSTALLED in the app's own `node_modules`
+ * and the app's `package.json` never mentions it. That is what a hoisted
+ * workspace store (or a `NODE_PATH` a pnpm bin shim exported) looks like to the
+ * resolver, and it used to read as AVAILABLE.
+ */
+let hostInstalledButUndeclared: string;
 
-function writeHost(prefix: string, withPkg: boolean): string {
+function writeHost(
+  prefix: string,
+  withPkg: boolean,
+  opts: { declare?: boolean } = {},
+): string {
+  const declare = opts.declare ?? withPkg;
   const dir = mkdtempSync(join(tmpdir(), prefix));
   writeFileSync(
     join(dir, 'package.json'),
     JSON.stringify({
       name: 'dogfood-host-fixture',
       private: true,
       type: 'module',
-      ...(withPkg ? { dependencies: { [ORGANIZATIONS_PKG]: '*' } } : {}),
+      ...(declare ? { dependencies: { [ORGANIZATIONS_PKG]: '*' } } : {}),
     }),
     'utf8',
   );
@@ -60,10 +72,11 @@ function writeHost(prefix: string, withPkg: boolean): string {
 beforeAll(() => {
   hostWithPkg = writeHost('os-dogfood-org-ok-', true);
   hostWithoutPkg = writeHost('os-dogfood-org-missing-', false);
+  hostInstalledButUndeclared = writeHost('os-dogfood-org-undeclared-', true, { declare: false });
 });
 
 afterAll(() => {
-  for (const dir of [hostWithPkg, hostWithoutPkg]) {
+  for (const dir of [hostWithPkg, hostWithoutPkg, hostInstalledButUndeclared]) {
     if (dir) rmSync(dir, { recursive: true, force: true });
   }
 });
@@ -99,4 +112,24 @@ describe('enterprise multi-org probe (#4700)', () => {
   it('does not throw when the run declares the package AND it is there', async () => {
     await expect(probeOrganizations(hostWithPkg, true)).resolves.toEqual({ available: true });
   });
+
+  it('reports UNAVAILABLE when the package is merely PRESENT but not declared (#4719)', async () => {
+    // The gate is the app's declaration, not what happens to be reachable. Under
+    // the old resolver this host answered AVAILABLE — same bytes on disk, same
+    // package.json, and the multi-org gates would run for an app that never
+    // asked for the enterprise runtime. Worse, in a real pnpm workspace the
+    // "present" half arrives via the bin shim's NODE_PATH, so the verdict moved
+    // with the launcher.
+    const probe = await probeOrganizations(hostInstalledButUndeclared, false);
+    expect(probe.available).toBe(false);
+    expect(probe.reason).toContain("package.json");
+  });
+
+  it('THROWS with a DECLARE-it remedy when the run declares it but the app does not (#4719)', async () => {
+    // The remedy has to be the one that works. "Install it" is unfollowable
+    // advice here — it is already installed; the missing act is declaring it.
+    await expect(probeOrganizations(hostInstalledButUndeclared, true)).rejects.toThrow(
+      new RegExp(`declare ${ORGANIZATIONS_PKG.replace('/', '\\/')} in .* package\\.json`),
+    );
+  });
 });