Skip to content

feat(objectql)!: refuse to boot when a data driver fails to connect (#3741) - #3751

Merged
os-zhuang merged 2 commits into
mainfrom
claude/objectql-driver-connection-error-jwicgq
Jul 28, 2026
Merged

feat(objectql)!: refuse to boot when a data driver fails to connect (#3741)#3751
os-zhuang merged 2 commits into
mainfrom
claude/objectql-driver-connection-error-jwicgq

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Closes #3741. 采纳 issue 里倾向的方案 Ainit() 默认 fail-fast,宽容行为退化为显式 opt-in 的逃生阀。

问题

packages/objectql/src/engine.tsinit() 对每个 driver 的 connect() 做 try/catch,失败只 log 一条 error 就继续启动。两个后果,结构性的那个更严重:

  1. 误配静默通过:数据库连不上的服务器"启动成功",然后每个请求 500,报出的错误跟真实原因相距很远。它打印的那句 Operations may recover via lazy reconnection or fail at query time 有一半是假的 —— 全仓 grep,driver-sql / driver-mongodb 里没有任何重连实现,只剩 "fail at query time"。调用点还紧接着做 DDL:ObjectQLPlugin.start()init() 之后立刻跑 syncRegisteredSchemas(),对一个没连上的 driver 建表建索引。
  2. driver 失去了"拒绝启动"的能力:任何启动期致命校验(许可、版本、配置不兼容、能力缺失,不只是网络不可达)都靠 connect() 抛错表达,而它们全被这层 catch 降级成运行时错误。feat(driver-mongodb)!: 显式单租户 —— 检测到多租户模式即拒绝启动 (#3724) #3734 的 driver-mongodb 多租户守卫被迫挪进构造函数,正是为了绕开这一点。

改动

  • init() 默认 fail-fast —— 任一 boot 期注册的 driver connect() 失败即抛 DriverConnectErrorcode: 'ERR_DRIVER_CONNECT'),kernel bootstrap 中止。仍然先尝试完所有 driver,所以一次失败的启动能把问题全列出来。错误消息自包含(每个失败 driver + 原因),因为 CLI 只打印 error.message(stack 仅在 DEBUG 下);首个 cause 挂在 error.cause 上。从 @objectstack/objectql@objectstack/objectql/core 都导出。

  • connect() 成为 driver 否决启动的受支持位置 —— 需要活连接才能做的启动期校验(server version、能力探测)不必再硬塞进构造函数。

  • 删掉 "lazy reconnection" 那句空头承诺(Prime Directive chore: version packages #10:declared ≠ enforced)。

  • 逃生阀 OS_ALLOW_DRIVER_CONNECT_FAILURE=1@objectstack/typesresolveAllowDriverConnectFailure(),默认关)恢复旧的宽容启动,但会打一条 DEGRADED BOOT banner,写明失败的 driver 不会被重试也不会重连、所有路由到它们的查询和 schema sync 在本进程生命周期内都会失败。

    banner 除了 logger 还会直接写 stderros serve 在 boot 期间接管了整个 process.stdout.write(boot-quiet),而 Loggerwarn 送去 stdout —— 只走 logger 的话,这条最该被看见的消息在它唯一存在意义的场景里恰好是隐形的。实测发现并修掉了。

  • 顺手更新 driver-mongodb 里把"init() 会吞掉 connect 失败"写进注释/changeset 的地方(构造函数守卫保留 —— 它失败得更早 —— 但理由已经不同了)。

影响与迁移

配置正确的部署无需任何改动:以前能连上的 driver 现在照样连上。以前静默地在没有数据库的情况下启动的部署,现在会启动失败,并在错误里给出 driver 名和原因。要继续那样启动 —— 明知每个碰到该 driver 的请求都会失败 —— 设 OS_ALLOW_DRIVER_CONNECT_FAILURE=1

changeset 按 #3734 的先例定为 minor + !

验证

  • 新增 packages/objectql/src/engine-driver-connect-failfast.test.ts(8 例):单 driver 失败抛错;多 driver 失败时消息里列全每个 driver 及其原因、failedDrivers / totalDrivers / cause 正确;语义性(非网络)拒绝同样否决启动;全部健康时正常 resolve 且每个 driver 都连上;不会在第一个失败处停下;opt-in 后降级启动并打出 banner;banner 确实落到 stderr;falsy 值(false)视为未开启。
  • pnpm build + pnpm test 全绿(132/132 tasks)。
  • 真实启动验证(examples/app-crmOS_DATABASE_URL 指向不可达的 postgres):
    • 默认:退出码 1,输出 1 of 1 data driver(s) failed to connect — refusing to boot. + • com.objectstack.driver.sql: connect ECONNREFUSED 127.0.0.1:59437
    • OS_ALLOW_DRIVER_CONNECT_FAILURE=1:服务器确实起来并持续运行(降级),DEGRADED BOOT banner 出现在启动输出第 7 行 —— 逃生阀是真能用的,不是又一个 declared ≠ enforced。

Generated by Claude Code

…3741)

`ObjectQLEngine.init()` wrapped every driver's `connect()` in a try/catch,
logged one error line, and carried on. A server whose database was unreachable
"started successfully" and then failed every request with an error that reads
nothing like *the database is down*. The warning it printed — "Operations may
recover via lazy reconnection or fail at query time" — was half fiction: no
reconnection exists in driver-sql or driver-mongodb, so only the "fail at query
time" half was ever real. The caller made it worse: `ObjectQLPlugin.start()`
runs `syncRegisteredSchemas()` immediately after `init()`, issuing DDL against
a driver that isn't there.

The structural half was worse than the operational one — the catch removed a
driver's ability to refuse startup at all. Any fatal startup check (licence,
server version, incompatible configuration, missing capability, not just an
unreachable socket) is expressed by throwing from `connect()`, and every one of
them was silently downgraded to a runtime error. That is why driver-mongodb's
multi-tenancy guard had to be hoisted into its constructor in #3734.

- `init()` now throws `DriverConnectError` (`code: 'ERR_DRIVER_CONNECT'`) when
  any boot-registered driver's `connect()` rejects, aborting kernel bootstrap.
  It attempts every driver first, so one failed boot names all of them. The
  message is self-contained — each failed driver and its cause — because the
  CLI prints `error.message` alone; the first cause is attached as
  `error.cause`. Exported from `@objectstack/objectql` and `.../core`.
- `connect()` is now a supported place for a driver to veto boot.
- Removed the misleading "lazy reconnection" claim.
- Escape hatch `OS_ALLOW_DRIVER_CONNECT_FAILURE=1`
  (`resolveAllowDriverConnectFailure()` in `@objectstack/types`) restores the
  lenient boot, defaults off, and states the degradation in a DEGRADED BOOT
  banner. The banner also goes to stderr: `os serve` swallows all of stdout
  during boot and `Logger` routes `warn` there, so logger-only it would be
  invisible in exactly the deployment the flag is for.
- Refreshed the driver-mongodb comments and changeset that documented the
  swallow as the reason its guard lives in the constructor.

Verified against a real boot: `os serve` with an unreachable postgres URL exits
1 with the driver name and cause; with the flag set it stays up, degraded, with
the banner on line 7 of the boot output.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T9VBCNvg9Zixrf1A9Rnwdd
@vercel

vercel Bot commented Jul 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Jul 28, 2026 1:59am

Request Review

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling size/m labels Jul 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 3 package(s): @objectstack/objectql, @objectstack/driver-mongodb, @objectstack/types.

16 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/concepts/metadata-lifecycle.mdx (via @objectstack/objectql)
  • content/docs/data-modeling/drivers.mdx (via @objectstack/driver-mongodb)
  • content/docs/data-modeling/formulas.mdx (via packages/objectql)
  • content/docs/deployment/migration-from-objectql.mdx (via @objectstack/objectql)
  • content/docs/deployment/vercel.mdx (via @objectstack/objectql)
  • content/docs/getting-started/glossary.mdx (via @objectstack/driver-mongodb)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/objectql)
  • content/docs/kernel/services.mdx (via @objectstack/objectql)
  • content/docs/permissions/authentication.mdx (via @objectstack/objectql)
  • content/docs/plugins/index.mdx (via @objectstack/objectql)
  • content/docs/plugins/packages.mdx (via @objectstack/objectql, @objectstack/driver-mongodb, @objectstack/types)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/objectql)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/driver-mongodb)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/objectql)
  • content/docs/releases/implementation-status.mdx (via @objectstack/objectql, @objectstack/driver-mongodb)
  • content/docs/releases/v9.mdx (via @objectstack/objectql)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

…and driver authors

The drift check flagged `data-modeling/drivers.mdx`; none of its existing claims
were invalidated, but the page had nothing about what happens when a driver
cannot connect — which is the half of #3741 a future driver author needs.

- Operator-facing: connect failure refuses the boot, `serve` exits 1, there is
  no lazy reconnection, and `OS_ALLOW_DRIVER_CONNECT_FAILURE=1` is the loud,
  never-in-production escape hatch.
- Author-facing: throwing from `connect()` is the supported way to veto boot for
  ANY fatal startup condition (server version, capability, deployment mode,
  licence), not just an unreachable socket — with a worked example. Connection-
  free checks can still go in the constructor, which is where MongoDBDriver
  keeps its tenancy guard.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T9VBCNvg9Zixrf1A9Rnwdd
@github-actions github-actions Bot added size/l and removed size/m labels Jul 28, 2026
@os-zhuang
os-zhuang marked this pull request as ready for review July 28, 2026 02:05
@os-zhuang
os-zhuang merged commit 030125b into main Jul 28, 2026
16 checks passed
@os-zhuang
os-zhuang deleted the claude/objectql-driver-connection-error-jwicgq branch July 28, 2026 02:10
os-zhuang added a commit that referenced this pull request Jul 28, 2026
…econnection" claim (#3769) (#3781)

Two related corrections, both from measuring what #3741/#3751/#3765 had only asserted.

**The claim was wrong.** #3751 and #3765 shipped several statements that drivers never reconnect. Measured, both recover: killing a real mongod and restarting it on the same port, the SAME driver instance served the next write in 13ms with no reconnect call from us (the official driver's topology monitor); and a knex/pg pool is not poisoned by an outage — its error tracks live server state, i.e. every acquire opens a fresh connection. The original reasoning grepped this repo for `reconnect`, found nothing, and concluded recovery does not happen — but the recovery lives in the client libraries, not in our code.

**Fail-fast at boot is unchanged; the reason is different.** It is not that the connection can never return — it is that the boot sequence never re-runs. A driver that missed init() also missed syncRegisteredSchemas(), so its tables can simply not exist even after the database comes back. The DEGRADED BOOT banner now says that.

**The real defect underneath.** SqlDriver passed its config to knex untouched, so an endpoint that accepts TCP but never completes the handshake made every query wait out tarn's 30s default, then fail with "Timeout acquiring a connection. The pool is probably full" — pointing an operator at pool sizing instead of the network. SqlDriver now defaults pool.createTimeoutMillis to 10s, matching driver-mongodb's existing connectTimeoutMS. A host that sets its own value is left alone.

Message accuracy needs the dialect-specific knob (pg's connectionTimeoutMillis), which changes the shape of `connection` and would regress serve.ts's startup banner — left open in #3769.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

2 participants